G
GuideDevOps
Lesson 3 of 15

Input/Output & Redirection

Part of the Shell Scripting (Bash) tutorial series.

Standard Streams

The Three Streams

All processes have three standard streams:

# 0 = stdin (standard input)
# 1 = stdout (standard output)
# 2 = stderr (standard error)
 
# Name references
0  = /dev/stdin
1  = /dev/stdout
2  = /dev/stderr

Output Redirection

Redirect stdout

# Send output to file (overwrite)
echo "Hello" > output.txt
 
# Append to file
echo "World" >> output.txt
 
# Redirect stdout explicitly
command 1> output.txt
 
# Redirect to /dev/null (discard)
command > /dev/null

Redirect stderr

# Send errors to file
command 2> errors.txt
 
# Append errors
command 2>> errors.txt
 
# Discard errors
command 2> /dev/null

Redirect Both

# Both to same file
command > output.txt 2>&1
 
# Or using &>
command &> output.txt
 
# Or using |& (pipe both)
command |& grep "pattern"

Input Redirection

Read from File

# Redirect stdin from file
python script.py < input.txt
 
# Basic example
sort < unsorted.txt

Here Documents

cat << 'EOF'
Line 1
Line 2
Line 3
EOF

Here Strings

# Pass string as input
wc -w <<< "This is a test"
 
# Variable expansion
data="Important info"
grep "info" <<< "$data"

Multiple Input

# From multiple files
cat file1.txt file2.txt | sort
 
# From command output
sort < <(ls -la)

Pipes

Basic Piping

# Chain commands
ps aux | grep "firefox" | awk '{print $2}'
 
# Multiple pipes
cat data.txt | sort | uniq | wc -l
 
# Show intermediate results
echo "1 2 3" | tee intermediate.txt | awk '{print $1}'

Process Substitution

# Use command output as file
diff <(sort file1.txt) <(sort file2.txt)
 
# In loops
while read line; do
    echo "Processing: $line"
done < <(cat input.txt)

Named Pipes (FIFOs)

# Create named pipe
mkfifo my_pipe
 
# Write to pipe (background)
cat input.txt > my_pipe &
 
# Read from pipe
cat < my_pipe
 
# Cleanup
rm my_pipe

File Descriptors

Using File Descriptors

# Redirect to specific file descriptor
exec 3> output.txt              # Open fd 3 for writing
echo "Output" >&3               # Write to fd 3
exec 3>&-                       # Close fd 3
 
# Multiple file descriptors
exec 3> file1.txt
exec 4> file2.txt
echo "Line 1" >&3
echo "Line 2" >&4
exec 3>&- 4>&-

Duplicating File Descriptors

# Duplicate stdout to fd 3
exec 3>&1
 
# Redirect stdout to file
exec 1> output.txt
 
# Redirect back to terminal
exec 1>&3
exec 3>&-
 
# Usage - capture both, but still output
exec 3>&1 1> output.txt 2>&1
# ... commands run with output to file
exec 1>&3 3>&-

Advanced Redirection

Redirect All Output

# Trap all output (POSIX portable)
{
    command1
    command2
    command3
} > output.txt 2>&1

tee - Send to File AND stdout

# Show and save output
command | tee output.txt
 
# Append
command | tee -a output.txt
 
# Send to multiple files
command | tee file1.txt file2.txt

exec for Permanent Redirect

#!/bin/bash
 
# All output from this point goes to log.txt
exec >> log.txt 2>&1
 
echo "Starting script..."
echo "This goes to log file"

Swap Streams

# Swap stdout and stderr
command 3>&1 1>&2 2>&3 3>&-
 
# Capture stderr to stdout, discard stdout
command 2>&1 1>/dev/null
 
# Send stderr to stdout
(command 2>&1) 1>/dev/null

Practical Examples

Logging Script

#!/bin/bash
 
LOG_FILE="script.log"
ERROR_FILE="script.error"
 
exec 1> >(tee -a "$LOG_FILE")
exec 2> >(tee -a "$ERROR_FILE" >&2)
 
echo "Starting..."
echo "Some error" >&2

Suppress Output Conditionally

VERBOSE=0
 
if [ $VERBOSE -eq 1 ]; then
    output_redirect=""
else
    output_redirect="> /dev/null 2>&1"
fi
 
eval "my_command $output_redirect"

Read Multiple Files

# Combine multiple inputs
{
    cat file1.txt
    echo "---"
    cat file2.txt
} | sort | uniq

Pipeline with Error Handling

# Check exit code of all commands in pipeline
set -o pipefail
 
if ! cat data.txt | \
    grep "pattern" | \
    sort | \
    uniq > output.txt; then
    echo "Pipeline failed"
    exit 1
fi

Common Patterns

# Discard all output
command > /dev/null 2>&1
 
# Log errors separately
command 2> error.log | process_output
 
# Debug mode - show commands
exec 5>&1          # Save stdout
set -x             # Debug on
# ... script runs ...
set +x             # Debug off
exec 1>&5 5>&-     # Restore stdout
 
# Split output
command 1> >(sort > sorted.txt) 2> >(sort > sorted_errors.txt)

title: "Input/Output & Redirection" description: "Placeholder content for the Input/Output & Redirection lesson within the Shell Scripting (Bash) tutorial series." order: 3

Overview

Welcome to the Input/Output & Redirection lesson! This topic covers the essential fundamentals and practical applications within the Shell Scripting (Bash) ecosystem.

What You Will Learn

  1. Key terminology and core concepts.
  2. Industry best practices.
  3. Command line operations and configuration.

Example Code

Here is a quick placeholder snippet:

# Input/Output & Redirection
echo "Learning Shell Scripting (Bash) at GuideDevOps!"