Section 7: Advanced Shell Scripting Tools

Lesson 1: Awk and Sed for Text Processing

Text processing in shell scripts can be efficiently performed using tools like awk and sed. These tools provide powerful features for pattern matching and text manipulation.

1.1 Overview of Awk and Sed 

#!/bin/bash

# Overview of awk and sed

# Using awk to print the second column of a CSV file

awk -F',' '{print $2}' data.csv

# Using sed to replace a word in a file

sed -i 's/apple/orange/g' fruits.txt

1.2 Examples of Text Manipulation

Using Awk

#!/bin/bash

# Text manipulation with awk

# Print lines where the second field is greater than 50

awk '$2 > 50' data.txt

# Print the total sum of the third column

awk '{sum += $3} END {print "Total:", sum}' data.txt

Using Sed

#!/bin/bash

# Text manipulation with sed

# Replace all occurrences of "old" with "new" in a file

sed -i 's/old/new/g' file.txt

# Delete lines containing a specific pattern

sed -i '/pattern/d' file.txt

Lesson 2: Using External Commands and Utilities

Shell scripts can leverage external commands and utilities to extend their capabilities, allowing for more complex tasks.

2.1 Executing External Commands in Shell Scripts 

#!/bin/bash

# Executing external commands in shell scripts

# Run a Python script from within a shell script

python3 script.py

# Invoke a system command to list files

ls -l

2.2 Piping and Redirecting Output

#!/bin/bash

# Piping and redirecting output

# Pipe the output of one command as input to another

cat data.txt | grep "pattern"

# Redirect output to a file

ls -l > file_list.txt

Understanding how to use powerful text processing tools like awk and sed, as well as executing external commands, enhances the capabilities of shell scripts. In the next sections, we'll explore more advanced scripting techniques and real-world applications.