OS & Subprocess Module - Python for DevOps

Sometimes you need to call a shell command from Python because a native library doesn't exist or isn't as convenient. The subprocess module is the modern way to do this.

1. Running Basic Commands

Use subprocess.run() for the most common tasks.

Simple Command

Action:

import subprocess
 
# Pass command as a list of strings
result = subprocess.run(['ls', '-l', 'server.list'], capture_output=True, text=True)
 
print(f"Exit Code: {result.returncode}")
print(f"Output: {result.stdout}")

Result:

Exit Code: 0
Output: -rw-r--r-- 1 devops devops 24 Apr 10 12:00 server.list

Capturing Errors

If a command fails, you can catch it using check=True.

Action:

import subprocess
 
try:
    # This file does not exist
    subprocess.run(['cat', 'missing-file.txt'], check=True, capture_output=True, text=True)
except subprocess.CalledProcessError as e:
    print(f"Error occurred!")
    print(f"Code: {e.returncode}")
    print(f"Message: {e.stderr}")

Result:

Error occurred!
Code: 1
Message: cat: missing-file.txt: No such file or directory

2. Using the Shell

By default, subprocess.run does not use the shell (it's safer). However, for pipes (|) or redirects (>), you can enable shell=True.

Action:

import subprocess
 
# Dangerous if user input is involved, but fine for hardcoded commands
result = subprocess.run(
    "echo 'hello' | tr '[:lower:]' '[:upper:]'",
    shell=True,
    capture_output=True,
    text=True
)
 
print(result.stdout.strip())

Result:

HELLO

3. Environment Variables

You can pass specific environment variables to the subprocess.

Action:

import os
import subprocess
 
my_env = os.environ.copy()
my_env["DEPLOY_TARGET"] = "production"
 
result = subprocess.run(
    ["bash", "-c", "echo Deploying to $DEPLOY_TARGET"],
    env=my_env,
    capture_output=True,
    text=True
)
 
print(result.stdout.strip())

Result:

Deploying to production

Summary

Use subprocess.run() for almost everything.
capture_output=True is needed to read stdout and stderr.
text=True makes the output a string instead of bytes.
check=True makes the script crash if the command fails (often desired).
Avoid shell=True if any part of the command comes from user input.