Build1 publisher3 min readPublished
A pinned vCPU disappears into a 12.5 percent average on an eight-vCPU guest
A dev.to runbook for slow Linux virtual servers orders the read-only evidence, putting timestamps, per-CPU sampling and /proc/pressure ahead of any load test, because utilisation hides the waiting that users feel.
The Engineer · Build desk

What happened
- A dev.to incident runbook opens on the case where a dashboard reads 20% CPU with memory still available while a user reports an application that is slow or intermittently unreachable.
- It lists six things one CPU graph cannot separate: storage waits, memory reclaim, packet loss, cgroup stalls, contention for hypervisor time, and a degraded network path.
- Before any metric it asks for four written facts, including the exact UTC start and end time and whether all clients were affected or only one location or ISP.
- The opening stages are read-only, and it confines fio and iperf3 to an approved window, against endpoints you control, with explicit rate limits.
- One single-threaded worker can max a vCPU while the average across eight vCPUs stays near 12.5%, which per-CPU mpstat output exposes.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision Escalating to the host provider now turns on several sustained steal intervals correlated with application timing, so an on-call who files after one spike has no case to hand over.
- constraint Anyone running containers cannot read the VM's memory total as the service ceiling, so the cgroup path has to be found before the memory number means anything.
- cost The first pass costs sampling time and nothing else, because the load generators that would prove a storage or network limit sit behind an approved window and someone's permission.
Utilisation and delay are separate measurements, and the runbook keys on the second. The PSI files /proc/pressure/cpu, /proc/pressure/memory and /proc/pressure/io each report "some" and "full" lines describing how much time tasks were delayed because a resource was unavailable [15]. According to the runbook, that is often more informative than utilisation, because it measures waiting experienced by workloads [15]. Memory is where the gap shows up first. Linux deliberately uses free RAM for cache, so the free column alone is not an alarm [16], while a rising memory PSI means tasks are stalling during reclaim [17].
The 12.5 percent figure is a claim about someone else's guest. It holds when two things are true: the guest has eight vCPUs, and the hot path is single-threaded [13]. On a two-vCPU guest the same pinned worker shows up as 50 percent of the aggregate [24], which no operator would call healthy. mpstat -P ALL is the command that distinguishes the two cases [11][13].
Steal time is the other number that survives a low average. The st column in vmstat counts time when the virtual CPU was ready and the hypervisor scheduled something else [12]. Sustained steal that correlates with latency is strong evidence of host-side contention, and the runbook is explicit that one sample is not proof [14].
The cheapest split in the sequence is the paired HTTP request. Time the call from an external client with curl [9], then repeat it against localhost or the private service address [10]. Fast locally and slow externally points away from the application process and toward the proxy, firewall, network or client path [10].
The numbers also lie about scope, in two places. Inside containers, host memory can look comfortable while the service is hitting its cgroup limit, so on cgroup v2 the comparison is memory.current against memory.max in the relevant scope under /sys/fs/cgroup [18]. On disk, a filesystem with plenty of free space can still respond slowly, so capacity and inodes come first with df -hT and df -i [19]; await from iostat -xz, the average time for an I/O request to complete, comes after [20][21].
All of it depends on the timestamps written down before the first command. The runbook names the failure this prevents: comparing a user report from 14:05 with metrics from 14:40, after the system has already recovered [8]. Its identity bundle starts with date -u -Is [7].
The published text breaks off in the storage section at the definition of await [23], so the packet-loss and network-path stages named in the introduction are not spelled out. What is there separates guest-level faults from upstream ones and ends in a compact incident package for an application team or infrastructure provider [22].
What to watch
- Whether your metrics agent samples /proc/pressure at all; if it stores only utilisation, the waiting this runbook keys on is gone once the guest recovers.
- Whether an infrastructure provider treats several correlated steal-time intervals as evidence, or asks for host-side proof a guest cannot produce.