Build1 publisher3 min readPublished
A backend that detects the AMD GPU can still leave operations on the CPU
ROCm sits under PyTorch, vLLM and SGLang; Vulkan is what llama.cpp-class engines compile shaders against. The engine you already run narrows the choice to one, and the rest of the work is reading logs.
The Engineer · Build desk
What happened
- A dev.to hosting guide argues ROCm and Vulkan sit at different layers: ROCm is the compute platform under PyTorch, vLLM and SGLang, while Vulkan is the portable API llama.cpp-class engines use for quantized models.
- ROCm 10.0.0 formally adds RDNA 4 support for gfx1200, covering the RX 9060, 9060 XT and 9050, and gfx1201, covering the RX 9070 series and the Radeon AI PRO R9700 series.
- The guide's stated failure mode is a backend that detects the GPU, leaves some operations on the CPU, and looks healthy while performing badly, so it prescribes log inspection and an end-to-end prompt test.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision Existing weights settle the backend argument before anyone benchmarks. A GGUF shelf makes llama-server, Ollama, LM Studio or LocalAI the cheap path; batching and tensor parallelism make ROCm the only path.
- exposure An acceptance check that only asserts the GPU was found will pass on a build doing part of the work on the CPU, and throughput ends up low with nobody able to explain why.
- capability Teams with mixed AMD, Intel and Nvidia hardware can standardise on one Vulkan build of a llama.cpp-class engine and skip per-vendor machine-learning stacks entirely.
Vulkan is implemented by the GPU driver. An inference engine ships or compiles compute shaders, and those execute through a Vulkan implementation: Mesa RADV on Linux, the vendor driver on Windows [13]. ROCm is the compute stack underneath a framework, with the HIP runtime, compiler, math libraries, collective communication, profilers and framework packages in it [6]. In local LLM hosting the two sit at different layers [2].
If an application only exposes a HIP or PyTorch path, Vulkan will not help. If the application is already llama.cpp with GGUF weights, installing the whole ROCm stack may solve a problem you did not have [16]. The engine decides. Vulkan is not a drop-in PyTorch platform [14]. Vulkan gives a llama.cpp-style engine one backend design that runs on AMD, Intel and Nvidia hardware with no vendor machine-learning stack installed [15].
A library of hand-picked GGUF quantizations points at llama-server, Ollama, LM Studio or LocalAI. A deployment built on tensor parallelism, continuous batching and framework-native weights points at ROCm with vLLM or SGLang [18]. Model inventory pushes the same way the engine does. Neither backend formally implies a file format, and llama.cpp reads the same GGUF from either its HIP or its Vulkan build [17].
That makes llama.cpp the place to compare the two. The model file and the HTTP client do not change, and the tokenizer, sampling settings, chat template and quantization can stay fixed [19]. The current build documentation uses `GGML_HIP` for ROCm and `GGML_VULKAN` for Vulkan. Guides that still recommend `GGML_ROCM` or the removed Makefile flags are describing options that are gone [20]. The Vulkan side wants `libvulkan-dev`, `glslc`, `spirv-headers` and `vulkan-tools`, then `vulkaninfo --summary` to confirm the driver enumerates the card you meant [21].
With ROCm, six things have to form a compatible set: host driver, GPU target, user-space libraries, framework wheel, kernel version, container image [8][22]. ROCm 10.0.0 shipped on August 26, 2026, built on TheRock, AMD's build and release system since ROCm 7.14 [9]. It validates PyTorch 2.13, vLLM 0.27 and SGLang 0.5.15 [10]. It formally adds RDNA 4 support for gfx1200 (RX 9060, 9060 XT, 9050) and gfx1201 (RX 9070, 9070 XT, 9070 GRE, Radeon AI PRO R9700 series) [11]. For a specific GPU and operating system pair, the compatibility matrix is the authority, whatever a forum post about a card with the same marketing name says [12].
A backend can detect the GPU, leave some operations on the CPU, and look healthy while performing badly. The dev.to guide therefore asks for log inspection and an end-to-end prompt test before any decision is final [5]. Its own comparison table is described as a starting policy, not a benchmark result [4]. For anyone else's throughput number to transfer to your box, the quantization, the tokenizer, the sampling settings and the GPU target would all have to match yours. llama.cpp is where those can be held fixed [19].
What to watch
- Whether ROCm 10.x extends its validated set beyond PyTorch 2.13, vLLM 0.27 and SGLang 0.5.15.
- Whether the compatibility matrix widens the operating systems listed for gfx1200 and gfx1201.
- Whether llama.cpp renames its backend CMake options again, as it did when GGML_ROCM went away.