Shell Tools: Essential Command-Line Utilities for Faster Workflows

Mastering Shell Tools: Top Utilities Every Developer Should Know

The command line remains an indispensable environment for developers. Mastering shell tools boosts productivity, automates repetitive tasks, and gives precise control over systems and data. This article highlights the top shell utilities every developer should know, with practical examples and short workflows to start using them today.

1. The Core: bash/zsh and shell built-ins

• What: The shell (bash, zsh, fish) is the command-line interpreter and includes built-ins like cd, export, alias, and functions.
• Why: Built-ins are fastest and available without external binaries; they’re essential for scripting and environment management.
• Quick tips:
  • Use aliases for frequent commands: alias gs=‘git status’.
  • Define small functions for reusable tasks:

    Code
    Copy CodeCopied
    mkcd() { mkdir -p “\(1" && cd "\)1”; } 

  • Use set -euo pipefail in scripts to catch errors early.

2. File and Text Manipulation: find, xargs, grep, sed, awk

• find — locate files by name, type, size, or modification time.
  • Example: find . -type f -name ‘.log’ -mtime +7 -delete
• xargs — build and execute command lines from standard input; combines well with find.
  • Example: find . -name ‘.tmp’ -print0 | xargs -0 rm
• grep (ripgrep/rg) — search file contents. Consider ripgrep (rg) for speed.
  • Example: rg “TODO” src/
• sed — stream editor for substitutions and simple transforms.
  • Example: sed -i ’s/old/new/g’ file.txt
• awk — powerful for field-processing and reporting.
  • Example: awk -F, ‘{sum+=$3} END {print sum}’ data.csv

3. File Viewing and Navigation: ls, tree, less, bat

• ls — list directory contents. Use ls -lah for human-readable sizes.
• tree — visual directory tree: tree -L 2 for limited depth.
• less — paginated viewing with search: less +F logfile.
• bat — a cat clone with syntax highlighting and Git integration (install separately).

4. Archiving and Compression: tar, gzip, zip

• tar — create and extract archives:
  • Create: tar -czf archive.tar.gz folder/
  • Extract: tar -xzf archive.tar.gz
• gzip / gunzip — compress single files.
• zip / unzip — cross-platform archives.

5. Networking and Remote Work: ssh, scp, rsync, curl, wget

• ssh — secure shell for remote access. Use key-based auth and config file (~/.ssh/config).
• scp / rsync — copy files securely. Prefer rsync -avz for efficient syncs.
• curl / wget — download or interact with HTTP APIs.
  • Example: curl -sS https://api.example.com/data | jq ‘.’

6. Process and System Monitoring: ps, top/htop, lsof, du, df

• ps — list processes: ps aux | grep myservice.
• top / htop — live process monitor. htop has a friendlier UI.
• lsof — list open files and network sockets.
• du / df — disk usage and filesystem space: du -sh and df -h.

7. Version Control Helpers: git (and git tools)

• git — essential for source control. Combine with shell tools for productivity.
  • Common shortcuts: git log –oneline –graph –all.
  • Use aliases: git config –global alias.st status.
• Tools like tig give a terminal UI for git history.

8. Parallelism and Job Control: & , nohup, screen/tmux, parallel

• Background jobs: command & and job control with fg/bg.
• nohup — keep processes running after logout.
• screen / tmux — persistent terminal sessions; tmux is widely preferred.
• GNU parallel — run jobs in parallel safely:
  • Example: ls *.jpg | parallel convert {} -resize 800x600 resized/{}

9. Data Manipulation and JSON: jq, csvkit

• jq — query and transform JSON in the shell.
  • Example: curl -s api | jq ‘.items[] | {id, name}’
• csvkit — tools for CSV processing: csvcut, csvstat, csvgrep.

10. Automation and Scripting: make, cron, systemd timers

• make — simple task runner beyond building code. Use Makefiles to encode workflows.
• cron — schedule recurring jobs