Text Processing

Parsing Data Like a Pro

The true power of Linux lies in chaining small, isolated commands together using Pipes (|). The output of the left command becomes the input of the right command.

Search with grep

grep filters lines of text matching a pattern.

# Find all "ERROR" entries in an apache log file
cat /var/log/apache2/error.log | grep "ERROR"
 
# Search recursively for "API_KEY" in all files within a code directory
grep -r "API_KEY" /var/www/app

Search and Replace with sed

The Stream Editor (sed) can process and alter text on the fly.

# Replace 'localhost' with 'db.remote.com' inside a configuration file
sed -i 's/localhost/db.remote.com/g' config.yml

Column Extraction with awk

awk is a complete programming language designed for text processing. It is phenomenal for extracting specific data columns.

# Print only the 1st (permissions) and 9th (filename) columns from an 'ls' output
ls -la | awk '{print $1, $9}'
 
# Show the 1st column (IP address) from an NGINX access log
awk '{print $1}' /var/log/nginx/access.log

When you combine these tools together, you have absolute power over your system data!