The pip story is the good news on this axis. On NVIDIA hardware JAX's wheels bring the CUDA libraries with them, so there is nothing to build [13]. Everything above that layer is where the aarch64 tax shows up. The engine in this rig is the repo's own Gemma 4 port, driven by a JAX generation loop behind an OpenAI-compatible FastAPI server under systemd [14]. Writing your own decode loop is a real line item, and it is roughly what you pay for a pairing the guide describes as one almost nothing in the ML ecosystem targets [4].
The Hugging Face token, incidentally, is written to Secrets Manager under the path `vllm/hf-token` [18], which on a pure-JAX rig reads like a fossil of the attempt before this one.
Then the memory budget. The T4G reports 15,360 MiB of device memory rather than the nominal 16 GB, and the guide says to budget against the measured number [11]. The weights are 6.155 GB [6]. Read as decimal gigabytes, that is about 5,869 MiB resident, leaving roughly 9,491 MiB for KV cache, activations and XLA scratch [3]. Read it as GiB instead and about 430 MiB of that goes away [3]. Workable, and not roomy, and the T4G has no fp8 [3], so the usual lever for buying cache space back with a smaller numeric format is absent.
Now read the default. The rig runs `g5g.2xlarge` at 1 GPU, 8 vCPU and 16 GiB, and that instance type is hardcoded into the MCP server's environment block [10][17]. The guide's own note that the first fitting fractional instance, `g6f.4xlarge` at $0.9500, is 1.7x this rig [7] puts the configuration you actually launch at about $0.56 an hour [2], which is around 6% above the $0.5260 `g4dn.xlarge` that the family undercuts on paper [5]. It is a bigger box than the `g5g.xlarge` that sets the floor, so this is not a like-for-like comparison. It does mean $0.4200 is the family's entry price and not the rig's.
For the price table to be your price table, three things have to hold. The same region and the same on-demand basis, since the guide's launch example passes `spot=True` [22]. G instance vCPU quota in us-east-1, because the default wants 8 vCPU and the tooling checks the limit before launching rather than after a failure [23]. And a subnet, security group and instance profile you already own, because the rig requires all three explicitly and will not create them for you [20].
