The /proc filesystem is a goldmine of information about running processes, and it lives entirely in RAM. It’s not a real filesystem — the kernel generates these files on the fly. Want to know what a process is doing? Don’t install another tool. Just read /proc.
Here’s the thing: ps, lsof, strace — they all read from /proc. You can do the same thing directly. Understanding /proc gives you a way to debug that doesn’t rely on specific tools being installed or having the right permissions.
The Structure
Every running process gets a directory named after its PID:
$ ls /proc1 cmdline devices filesystems mounts swaps10 cpuinfo diskstats interrupts partitions sys2 crypto dma iomem pressure sysvipc3 cgroups domainname ioports sched_debug thread-selfacpi cgroup_rw exec-domains kallsyms schedstat timer_list...The numeric directories are processes. Let’s look inside one:
$ ls -la /proc/1234/total 0dr-xr-xr-x 9 root root 0 Mar 1 10:45 .-r--r--r-- 1 root root 0 Mar 1 10:45 attr-rw-r--r-- 1 root root 0 Mar 1 10:45 autogroup-r--r--r-- 1 root root 0 Mar 1 10:45 cgroup--w------- 1 root root 0 Mar 1 10:45 clear_refs-r--r--r-- 1 root root 0 Mar 1 10:45 cmdline-r--r--r-- 1 root root 0 Mar 1 10:45 comm...Each file tells you something about the process.
The Essential Files
1. cmdline — The Command That Started the Process
$ cat /proc/1234/cmdline | tr '\0' ' '; echo/usr/bin/python3 /opt/myapp/server.py --config /etc/app.conf --debugThis is the exact command line. Useful when ps output is truncated or you need the full arguments.
2. environ — Environment Variables
$ cat /proc/1234/environ | tr '\0' '\n' | head -n 10PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/binHOME=/rootUSER=rootLANG=en_US.UTF-8APP_DEBUG=1DATABASE_URL=postgresql://localhost/mydbSee what environment variables the process inherited. Useful for debugging why a config setting isn’t being picked up.
3. fd/ — Open File Descriptors
$ ls -la /proc/1234/fdtotal 0lrwx------ 1 root root 64 Mar 1 10:45 0 -> /dev/pts/1lrwx------ 1 root root 64 Mar 1 10:45 1 -> /dev/pts/1lrwx------ 1 root root 64 Mar 1 10:45 2 -> /dev/pts/1lrwx------ 1 root root 64 Mar 1 10:45 3 -> socket:[12345]lrwx------ 1 root root 64 Mar 1 10:45 4 -> /var/log/app.loglrwx------ 1 root root 64 Mar 1 10:45 5 -> /etc/app.conflrwx------ 1 root root 64 Mar 1 10:45 6 -> /dev/nullEach number is a file descriptor. The symlink target shows what it points to. Socket fds show socket:[inode]. This is exactly what lsof reads under the hood.
4. status — Process State and Limits
$ cat /proc/1234/statusName: python3Umask: 0022State: S (sleeping)Tgid: 1234Pid: 1234PPid: 1000TracerPid: 0Uid: 0 0 0 0Gid: 0 0 0 0FDSize: 256VmPeak: 123456 kBVmHWM: 56789 kBVmRSS: 45678 kB...Process state (sleeping, running, zombie), memory usage (VmRSS is resident set size), and more. The FDSize tells you the max file descriptors.
5. maps — Memory Layout
$ cat /proc/1234/maps55555555b000-55555555c000 r-xp 00000000 08:05 9999999 /usr/bin/python355555575b000-55555575c000 r--p 00001000 08:05 9999999 /usr/bin/python355555575c000-55555575d000 rw-p 00002000 08:05 9999999 /usr/bin/python37ffff7dd5000-7ffff7dff000 r-xp 00000000 08:05 8888888 /lib64/libc-2.31.so...Every memory region the process has mapped. Useful for understanding what libraries are loaded and memory layout during debugging.
6. cgroup — Control Group Info
$ cat /proc/1234/cgroup12:freezer:/11:memory:/system.slice/myapp.service10:cpuacct:/system.slice...Which cgroups the process belongs to. Useful for understanding resource limits and systemd service behavior.
7. limits — System Limits
$ cat /proc/1234/limitsLimit Soft Limit Hard Limit UnitsMax cpu time unlimited unlimited secondsMax file size unlimited unlimited bytesMax data size unlimited unlimited bytesMax stack size 8388608 unlimited bytesMax core file size 0 unlimited bytesMax resident set unlimited unlimited bytesMax processes 31767 31767 processesMax open files 1024 65536 filesThe resource limits for this process. This is what ulimit shows.
Real-World Debugging Scenarios
What files is my app actually opening?
$ ls -la /proc/$(pgrep myapp)/fd | awk 'NF>9 {print $NF}' | sort | uniqHow much memory is my process using?
$ grep VmRSS /proc/$(pgrep myapp)/statusVmRSS: 45678 kBIs my app running with the right UID/GID?
$ grep Uid /proc/$(pgrep myapp)/statusUid: 1000 1000 1000 1000What command was used to start this process?
$ cat /proc/$(pgrep myapp)/cmdline | tr '\0' ' '; echoSee all open network sockets:
$ cat /proc/net/tcpsl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode0: 0A0A0A01:0050 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 123456Not human-readable, but it’s there. That’s what ss parses.
The Key Insight
/proc is the single source of truth for process information. Every process monitoring tool reads from /proc. When tools fail or aren’t installed, /proc is still there.
You don’t need ps to see processes — they’re in /proc. You don’t need lsof to see file descriptors — they’re in /proc/[pid]/fd. You don’t need special tools to see environment variables — they’re in /proc/[pid]/environ.
Learning /proc gives you a foundation that works on any Linux system, with or without tools. And honestly, understanding how the kernel exposes this data makes you a better systems engineer.
Next time something goes wrong, don’t reach for another tool. Start with /proc. The answers are already there.
