Measuring Real Memory Pressure on a Linux Server

Understanding Memory Reporting

When troubleshooting performance issues on Linux, the free -h command is often the first stop. However, it can be misleading. Linux kernel developers have spent decades optimizing the system to use 'unused' RAM for page caching. Consequently, a server with very little 'free' memory is often operating exactly as intended. To identify actual memory pressure, we must look at how the kernel manages resources under contention.

The Problem with 'Free' Memory

In modern Linux systems, memory is divided into several categories: active, inactive, slab, and buffers. When you see low free memory, it usually means the kernel is holding onto file caches to speed up disk I/O. If a process needs memory, the kernel will instantly reclaim that cache. True memory pressure occurs only when the kernel cannot reclaim enough memory to satisfy new allocation requests, forcing it to enter 'direct reclaim' or trigger the OOM (Out of Memory) killer.

Using Pressure Stall Information (PSI)

Introduced in kernel 4.20, Pressure Stall Information (PSI) provides the most accurate metric for system-wide resource contention. It tracks the percentage of time tasks are delayed because they are waiting for memory.

You can view these metrics in the /proc/pressure/memory file:

cat /proc/pressure/memory

The output will look like this:

some avg10=0.00 avg60=0.00 avg300=0.00 total=12345
full avg10=0.00 avg60=0.00 avg300=0.00 total=0

• some: The percentage of time at least one task was stalled waiting for memory.
• full: The percentage of time all non-idle tasks were stalled simultaneously. This is the critical metric; if this is greater than zero, your system is experiencing severe memory starvation.

Analyzing Slab Usage

The slab allocator manages memory for kernel objects. Sometimes, a memory leak in a kernel module or a high number of dentry objects can cause the slab to consume significant RAM that cannot be easily reclaimed.

To inspect slab usage, use the slabtop command:

sudo slabtop -s c

This will sort by cache size. If you see a specific object type consuming gigabytes of RAM that isn't shrinking under pressure, you have identified a potential kernel-level memory leak.

Monitoring with vmstat

If you need to see if the kernel is actively struggling to reclaim memory, vmstat provides insight into the si (swap-in) and so (swap-out) columns. Frequent swapping is a clear indicator of memory pressure:

vmstat 1 5

If the si and so columns are consistently non-zero, your system is likely thrashing, meaning it is spending more time moving data to disk than executing code.

Conclusion

Stop relying solely on free to determine if your server needs more RAM. By monitoring PSI metrics, you can identify if your applications are actually being stalled by memory contention. If full pressure is consistently zero, your system is likely managing its memory efficiently, regardless of how small the 'free' value appears to be.