Understanding the Linux Kernel
What Is the Linux Kernel in Simple Terms
The kernel is the operating system's core. Everything else — bash, nginx, Python, Docker, even the desktop — runs on top of the kernel. The kernel manages physical hardware (CPU, memory, disk, network), schedules which processes run when, and provides controlled access to hardware through system calls.
As a DevOps engineer, you interact with the kernel constantly — through sysctl parameters, kernel modules, system calls, and the /proc and /sys virtual filesystems — even if you never touch the kernel source code.
How It Works
+------------------------------------------+| User Space || nginx, postgres, node, bash, docker |+------------------------------------------+ | system calls (read, write, open) v+------------------------------------------+| Kernel Space || Process scheduler, memory manager || VFS (virtual filesystem layer) || Network stack (TCP/IP) || Device drivers |+------------------------------------------+ | hardware access v+------------------------------------------+| Hardware || CPU, RAM, Disk, Network interface card |+------------------------------------------+Practical Commands
## Check kernel versionuname -r## 5.15.0-1031-aws <- kernel version, build, and platform uname -a## Linux mumbai-prod-node-1 5.15.0-1031-aws #35-Ubuntu SMP ... ## View kernel messages (hardware events, OOM kills, driver messages)dmesg | tail -20dmesg | grep -i 'error\|fail\|oom' ## View kernel messages with timestampsdmesg -T | tail -20 ## Kernel parameters at runtimesysctl -a | grep net.ipv4sysctl net.ipv4.tcp_syncookies ## view one parametersudo sysctl -w net.ipv4.tcp_syncookies=1 ## set immediately ## Persist sysctl settingssudo tee /etc/sysctl.d/99-custom.conf << 'EOF'net.ipv4.tcp_syncookies = 1vm.swappiness = 10EOFsudo sysctl -p /etc/sysctl.d/99-custom.conf ## List loaded kernel moduleslsmod ## Load a kernel modulesudo modprobe br_netfilter ## needed for Kubernetes networking ## Check if module is loadedlsmod | grep br_netfilter ## Make module load at bootecho 'br_netfilter' | sudo tee /etc/modules-load.d/k8s.conf ## View kernel boot parameterscat /proc/cmdline ## Available memory and swapcat /proc/meminfo | grep -E 'MemTotal|MemAvailable|SwapTotal'Troubleshooting
| Symptom | Command | What to Check |
|---|---|---|
| OOM kills happening | `dmesg | grep -i oom` |
| Kernel module missing | `lsmod | grep modulename` |
| High system CPU (sy) | perf top |
Kernel code consuming CPU |
| Network issues | sysctl net.ipv4 |
Kernel network parameters |
PLACEMENT PRO TIP**Tip:** `dmesg -T | grep -i oom` is your first command when applications are dying unexpectedly. The OOM (Out of Memory) killer logs every process it kills, including why — the process name, PID, and memory usage. This immediately tells you if memory pressure is causing your application crashes.
REMEMBER THIS**Remember:** sysctl changes made with `sysctl -w` are immediate but temporary — lost on reboot. Always write them to `/etc/sysctl.d/99-custom.conf` and run `sysctl -p` to make them persistent. Kubernetes setup guides often require specific sysctl values (like `net.bridge.bridge-nf-call-iptables=1`) that must persist across reboots.