Shell Scripting
Automate repetitive tasks with Bash scripts - file processing, CI/CD pipelines, system administration, and developer tooling.
skill: Shell Scripting category: Developer Experience tags: [Shell Scripting, Bash, CLI, Automation, Unix] readingTime: 6
Shell scripting is the glue of the Unix world - combining existing tools with pipes is more powerful than writing custom code for file processing, CI tasks, and system administration. Use shell for orchestration (pipe tools together); use Python when the logic is complex enough to need data structures, error handling, or external libraries.
Workflow
- Write safe, robust scriptsShell scripts silently proceed after errors by default. 'Strict mode' catches errors early and prevents cascading failures.
- Use variables, conditionals, and loopsShell scripting has quirky syntax for variables and conditionals - getting it right prevents subtle bugs like word splitting and empty string comparison.
- Process files with pipes and toolsUnix philosophy: compose small tools via pipes. grep, sed, awk, sort, uniq, cut, xargs - together they handle 90% of text processing.
- Write reusable functionsFunctions make scripts modular and testable - extracting logic into named functions also makes the main script flow readable.
- Handle arguments and inputA script that hardcodes values needs editing for each use; one that accepts arguments is reusable.
Output Format
Bash scripts: strict mode, functions, argument parsing, error handling, and Unix pipeline composition.
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
# Functions
log() { echo "[$(date +%H:%M:%S)] $*" >&2; }
# Main
main() {
log 'Starting'
# ... script logic
log 'Done'
}
main "$@"Examples
Input
Output
Edge Cases & Gotchas
- Spaces in filenames - Unquoted $file splits on spaces - 'my file.txt' becomes two arguments. Always quote variables: "$file". Use -print0 and read -d '' for null-delimited file processing.
- Portable vs. Bash-specific - #!/bin/sh requires POSIX sh compatibility (no [[, no arrays). #!/usr/bin/env bash explicitly requires Bash features. Choose based on portability requirements.
- Exit code propagation in pipes - By default, a pipe exits with the last command's status: false | true exits 0. set -o pipefail makes the pipe exit with the first failed command's status.
- Temporary file management - Scripts that create temp files in /tmp must clean them up to prevent disk exhaustion. Use mktemp and trap cleanup EXIT to guarantee cleanup even on error.
When to Break These Rules
- Use Python instead of Bash when the script needs: complex data structures, error handling with context, external API calls, or logic > 100 lines - Python is more readable and maintainable.
- Use Makefile instead of shell scripts for build tasks - Make's dependency tracking and target caching are built-in without extra scripting.
Reference Files
https://www.shellcheck.net/https://google.github.io/styleguide/shellguide.html