Understanding Linux Pipes
What Is a Pipe in Simple Terms
A pipe is a conveyor belt between commands. Instead of saving a command's output to a file and feeding that file to the next command, a pipe connects them directly — output from the left flows immediately into the input on the right.
ps aux | grep nginx | awk '{print $2}' — three commands chained together. ps produces output, grep filters it, awk extracts just the PID column. No temporary files, no intermediate steps.
How It Works
◈ DIAGRAM
+------------------+ +------------------+ +------------------+| ps aux | | grep nginx | | awk '{print $2}' || | | | | || stdout -------> | --> | stdin stdout -> | --> | stdin stdout -> | --> terminal| | | | | |+------------------+ +------------------+ +------------------+ All three run simultaneously in parallel.The kernel buffer (65536 bytes) between each pairbacks up the producer if the consumer is slow.Practical Commands
Bash
## Basic pipeps aux | grep nginx ## Chain multiple pipescat /var/log/nginx/access.log | grep '200' | awk '{print $1}' | sort | uniq -c | sort -rn | head -10## Finds top 10 IPs making successful requests ## Count lines matching a patterngrep 'ERROR' /var/log/app.log | wc -l ## Find most common errorsgrep 'ERROR' /var/log/app.log | sort | uniq -c | sort -rn | head -5 ## Process JSON outputcurl -s https://api.internal/health | jq '.services[] | select(.status == "down")' ## Named pipes (FIFOs) -- persist on filesystemmkfifo /tmp/mypipeecho "data" > /tmp/mypipe & ## write in backgroundcat /tmp/mypipe ## read from itrm /tmp/mypipe ## Check exit code of specific command in pipeline## $PIPESTATUS array holds each command's exit codecat file.txt | grep pattern | wc -lecho ${PIPESTATUS[@]}## 0 0 0 -- all succeeded## 0 1 0 -- grep found no matches (exit 1) ## With set -o pipefail, pipeline fails if any command failsset -o pipefailcat missing-file.txt | grep pattern## Error: cat returns 1, whole pipeline returns 1Troubleshooting
| Symptom | Command | What to Check |
|---|---|---|
| Pipeline always succeeds | echo ${PIPESTATUS[@]} |
Last command exit code hides earlier failures |
| Data lost in pipeline | Check buffer size | Producer much faster than consumer |
| Cannot pipe to sudo | `echo cmd | sudo bash` |
PLACEMENT PRO TIP**Tip:** Use `tee` to send output to both a file and the next pipe simultaneously: `command | tee output.log | grep ERROR`. The file gets everything, grep gets everything, and you lose nothing.
REMEMBER THIS**Remember:** In bash, `set -o pipefail` makes the entire pipeline return the exit code of the rightmost failed command. Without it, `false | true` returns 0 (success) because only the last command's exit code counts. Always use `set -o pipefail` in production scripts.