Build1 publisher3 min readPublished
One column in ollama ps separates a driver fault from a VRAM shortfall
A dev.to walkthrough argues that the PROCESSOR field and the startup inference compute log line narrow a slow Ollama to about five named causes, and each of the three states PROCESSOR can print points at a different fix.
The Engineer · Build desk
What happened
- A dev.to walkthrough puts the whole diagnosis in the PROCESSOR column of ollama ps, which prints one of three states while a model is loaded: 100% GPU, a CPU/GPU split, or 100% CPU.
- The second check is the server log line matching "inference compute", which on a healthy NVIDIA box names the device, library=CUDA, compute capability and driver version it found at startup.
- Ollama bundles its own CUDA runtime libraries, so the CUDA toolkit is not required, but that runtime still needs a driver new enough to talk to it.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Because detection runs once at server start, retrying a prompt proves nothing about a driver or environment change; the service has to come back up before any fix is even visible.
- decision A partial split forces a choice about what to give up, and the post orders it: quant precision first, then context length, then offloaded layers.
- exposure Anyone running the HSA_OVERRIDE_GFX_VERSION line on a gfx1031 or gfx1032 card is operating outside official support until they delete it.
- cost Reinstalling drivers for a fault the post attributes to a missing Unix group leaves the machine exactly as slow as it was.
Device detection happens once, when the server starts [8]. A driver installed after the service came up is therefore invisible to it, and the post's instruction after any driver update is `sudo systemctl restart ollama` [8]. `OLLAMA_LLM_LIBRARY` belongs in the same category: a pin left in the environment is one of the listed causes of a 100% CPU reading, so the check is whether it is set anywhere at all [2].
A partial split is a capacity problem you can compute before you change a setting. Weights plus the KV cache for your context window have to fit in VRAM, and a 7B model at Q4 runs roughly 4 to 5 GB before any cache [9]. On an 8 GB card that leaves about 3 to 4 GB for the cache and everything else [10]. Give it an 8K context and the cache eats into that; the post puts 32K context on an 8 GB card as most layers staying on the CPU [11].
The order of the three remedies matters. Dropping Q8 to Q4 halves the weight size for a modest quality cost, `num_ctx` dominates cache size, and `num_gpu` caps how many layers get offloaded at all [11]. That last one is the trap worth checking first in a shared setup, because a `num_gpu` below the model's layer count guarantees a split no matter how much VRAM is free, and the post's advice is to unset it if someone put it in a Modelfile or an API call [11][12].
There is a reverse case. A model reporting `100% GPU` that still runs slowly may be swapping over system RAM, and the check is `ollama ps` SIZE against the VRAM you actually have [13].
On AMD under Linux, three things have to hold: a ROCm build, which the official install script bundles; membership of the `render` and `video` groups, which is what grants access to `/dev/kfd` and `/dev/dri`; and a supported GPU or an override [14]. The post calls the missing group the most common "Ollama not using GPU on Ubuntu" forum post, and notes it survives driver reinstalls because the group membership is the fault [15].
The override is where the support boundary sits. Unsupported RDNA2 consumer cards, gfx1031 and gfx1032, fail detection even with a working ROCm stack, and the workaround is `Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"` in a systemd drop-in, followed by a restart [16]. The post describes that as unsupported but widely used, and says removing the line puts you back in official support territory [16]. On Windows, it says AMD support is narrower [17].
Treat the "about five causes" count as a claim about the failures this author has seen, not a measured population [5]. For it to transfer you need roughly their install path: a systemd service on Linux, or Windows and WSL, on consumer NVIDIA or AMD hardware [14][17]. A container started without GPU access is in the list, but the diagnosis there ends at the runtime flags [2]. What does transfer is cheap to run: `ollama run llama3.2 "hello"` in one terminal, `ollama ps` in another, and a grep of the server log for `inference compute` [19][3].
What to watch
- If Ollama ever re-probes devices on every request, the restart step drops out of this diagnostic.
- Supported Ollama builds for gfx1031 and gfx1032 would retire the HSA_OVERRIDE_GFX_VERSION workaround and shorten the AMD checklist.
- The Windows AMD section is where a specific supported-card list would settle how much of this transfers off Linux.