Start with the arithmetic, because it carries the case. A pod that requests `nvidia.com/gpu: 1` and then uses a tenth of the card is sitting on the other nine tenths [1]. Ten workloads of that shape could in principle share one accelerator; the whole-device request model gives them one each [2]. A faster model server does not touch this, and the CNCF post says so directly: what is needed is a stack that allocates accelerators without stranding or endangering capacity, and isolates tenants well enough that packing them together holds [11].
Here is what platform teams tell themselves: the accelerator problem is a scheduling problem, so a version bump fixes it. Here is what the drawing actually shows. Most layers are Kubernetes native or CNCF projects, with a few OSS and vendor pieces at the points that decide density and separation, including NVIDIA's MIG, vCluster and Dynamo [12]. Two years ago these same teams already had containers, RBAC, autoscaling and policy working, and no clean answer for accelerators or for keeping tenants apart on one node [16]. The parts arrived separately, from different owners. NVIDIA's framing, "infrastructure for the full AI lifecycle, from data preparation through training, fine-tuning, and high-volume inference", describes the scope rather than the plumbing [2].
The provisioning layer is where the honesty lives. Before a node counts, something inventories its GPUs, checks ECC state, records InfiniBand GUIDs and NIC MACs, network boots an image with the driver, CUDA and NCCL baked in, then applies a BIOS profile chosen for baseline, performance or confidential compute [13]. Burn-in runs it under load, an NCCL test proves the GPUs talk at full bandwidth, and the result lands in a source of truth such as NetBox, which also holds the IP assignments [14]. Retirement runs the same loop backwards: disks wiped, the remote-management login reset, the node handed back clean [15]. A fleet count and a validated fleet count are two different numbers, and only one of them can be scheduled.
The forcing function worth putting on the GPU intake form has two axes: where the tenant's isolation requirement comes from, and how much of a device the workload fills. A contractual or regulatory boundary plus a full-device job gets its dedicated block, with the idle hours priced back to the tenant that required them, since that default is safe under strict trust and wastes most of the hardware [9]. A contractual boundary with a small job goes to a separate pool running the confidential-compute BIOS profile, because a slice does not soften a boundary. A habitual boundary with a small job is the one cell where MIG partitions under a virtual control plane earn their keep. A habitual boundary with a full-device job belongs in the shared pool, with quotas doing work they already do well.
Naming the origin of the boundary is the uncomfortable part, because "we have always had our own cluster" arrives on a capacity request looking exactly like a compliance requirement. What settles the argument a quarter later is validated device-hours per tenant, split into allocated and used, because that split prices your tenancy policy instead of your hardware.
