Build1 publisher3 min readPublished
A vsock hop keeps the model on Metal while the agent runs in Ubuntu
Apple's Virtualization.framework gives Linux guests a 2D framebuffer and no path to Metal. So a sandboxed coding agent has to send every request back across a vsock channel to an inference server running natively on macOS.
The Engineer · Build desk
What happened
- Apple's Virtualization.framework does not expose the host Metal GPU to Linux guests, which receive only a 2D paravirtualized virtio-gpu framebuffer.
- Running the model inside the guest pushes inference onto virtualized CPU cores, and the post reports generation speeds falling by more than 80%, which it says makes multi-turn agentic loops unusable.
- Velo Workspaces runs Ollama or Apple MLX natively on macOS with full unified memory bandwidth and Metal access, and keeps the agent framework in an isolated Ubuntu guest.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint The isolation covers writes and shell execution. Every agent turn still depends on a host process the guest can reach over vsock, so the bridge sits inside the trust boundary and has to be modelled that way.
- cost The memory bill is paid on the host. A fifth to a quarter of RAM is held back for the OS and compositor, and the guest allocation is padded against pip and npm cache growth. That leaves less room for weights than the machine's spec suggests.
- decision Since only the listening port changes downstream, MLX versus Ollama stays reversible; the choice that is hard to undo is putting inference on the host in the first place.
- exposure The optional LAN forward makes the guest's web UI reachable from other machines on the network, so a VM built to contain an agent needs its own access controls on that port.
An 80% drop leaves at most a fifth of native generation speed, so call it a 5x slowdown [19]. The post gives that figure without naming the model, the quantization, or the host chip [22]. For the number to transfer, your weights and your guest core allocation would have to match the setup behind it. Why it happens is clear enough. Virtualization.framework hands the Linux guest a virtio-gpu 2D framebuffer and no compute path to Metal [3], and Docker Desktop, OrbStack and Colima all sit on Linux guest kernels with the same gap [5].
So the model stays outside the sandbox. What happens when the agent calls it:
1. The agent posts an OpenAI-compatible HTTP request to 127.0.0.1:8080 for MLX, or 127.0.0.1:11434 for Ollama, inside the VM [10]. 2. A local socat proxy takes that payload and routes it across vsock [11]. 3. A Swift bridge on the macOS host accepts the connection and relays it to whichever engine is set as the workspace's host provider [11].
The transport is vsock through hypervisor memory buffers instead of a virtualized NAT stack, and the post puts the bridge overhead in single-digit milliseconds [9].
File modification and terminal execution stay inside the Ubuntu guest's sandboxed filesystem [8]. That is the surface a prompt injection would otherwise use to read dotfiles or pull secrets out of the macOS Keychain [2]. The guest also holds a standing channel to a host process, by design. The guide's MLX server is launched bound to loopback [16], so the only listener the guest reaches is the bridge.
Weights and KV cache live in macOS unified memory. So do the OS and the display compositor. The guide's floor is to keep at least 20 to 25% of total host RAM for operating overhead [14]. On a 32 GB machine that is 6.4 to 8 GB withheld before you load a single weight [20]. The VM itself needs only enough to run the code the agent writes [21]. Long sessions grow. Temporary files, compiled dependencies, pip and npm caches and execution logs accumulate, and the advice is to over-allocate guest memory so the Linux OOM killer does not terminate the task [15]. On a machine with a discrete GPU, the compositor and the KV cache would never have been drawing on the same budget.
Engine choice is the least sticky part. The downstream steps (VM setup, the vsock bridge, agent configuration) are identical for MLX and Ollama, substituting the port the engine listens on [18]. The external forward runs host port 8081 to the guest's web UI on 4096, chosen so it does not collide with 8080 or 11434 [13]. The post also warns that MLX and Ollama conversions can lag a new model release by days to weeks [17].
What to watch
- Whether a benchmark naming model, quantization and chip puts the drop anywhere near 80%.
- Any Apple move to give Virtualization.framework guests a GPU compute path would collapse the split back onto one machine.
- Whether single-digit millisecond bridge overhead holds across an agentic loop making hundreds of calls per task.