Skip to content

Build1 publisher2 min readPublished

Topograph turns fabric discovery into Kubernetes labels and Slurm's topology.conf

NVIDIA's open source Topograph discovers cluster topology from five named clouds or from ibnetdiscover on-premises, then republishes it as node labels, Slurm config or Slinky ConfigMaps whenever a watched part of the cluster changes.

The Engineer · Build desk

Illustration accompanying Topograph turns fabric discovery into Kubernetes labels and Slurm's topology.conf

What happened

  • Google Cloud, Lambda, Nebius, Nscale and OCI have working provider integrations, and NVIDIA lists further cloud and colocation providers as in development.
  • On-premises clusters discover their topology through the InfiniBand provider with ibnetdiscover, or through NetQ for Spectrum-X and Multi-Node NVLink domains.
  • Topograph regenerates the topology view on request and whenever a watched part of the cluster changes, so the scheduler reads current data.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • cost Misplacement costs twice: lost throughput, and provisioned power that GPUs keep drawing while they wait on data.
  • constraint Anyone who cannot enable an alpha feature gate loses the Node Feature Discovery output and has to write scheduling rules against plain node labels.
  • decision NVIDIA did not publish a measured throughput difference for a placement change, so approving a rollout means running one job twice on your own fabric and comparing.
  • capability Because the provider interface is open, an operator whose fabric has no integration yet can write discovery for it and send it upstream.

Topograph discovers and publishes topology; it does not schedule anything. It splits into providers and engines: a provider discovers topology and normalizes it into one canonical model, and an engine translates that model into Slurm configuration, Kubernetes labels, Slinky ConfigMaps, Node Feature Discovery resources, or an instance-oriented topology JSON [2]. The toolkit is open source [3]. The integration surface is therefore a config artifact the scheduler already reads.

Slurm and Kubernetes both already support topology-aware allocation, and both can only act on the topology they observe [15]. Five components keep Topograph's view current, and the first one I would size is the API Server, which validates requests, aggregates duplicates and dispatches discovery [18].

The bandwidth gap here is wide. NVIDIA puts fifth-generation NVLink at 1.8 TB/s of bidirectional bandwidth per GPU on Blackwell parts such as GB200 and GB300, and sixth-generation at 3.6 TB/s per GPU on Vera Rubin, over a dedicated NVLink Switch fabric [12]. It puts a modern Quantum InfiniBand port at up to 800 Gb/s [11]. In the same units, 1.8 TB/s is 14,400 Gb/s, which is 18 of those ports, and 3.6 TB/s is 36 of them [19]. For that ratio to describe a job of yours, the collective has to be bandwidth-bound and all-to-all, which is the traffic pattern NVLink's non-blocking design is built for [13].

Two lines in the support matrix cost real work to satisfy. The Slurm engine will run inside Kubernetes, but it requires a writable volume for its configured topology.conf output path [9], so the scheduler's topology file becomes a mounted artifact that someone has to own. The Crusoe provider reads fabric and accelerator-domain labels from Crusoe Managed Kubernetes nodes, which means Topograph itself runs in Kubernetes there [8]. The matrix is dated to upstream main as of September 16, 2026, and NVIDIA says requirements can vary by Topograph version, environment and provider configuration [7].

NVIDIA argues the case on power, describing AI factories as power-limited systems in which GPU workload placement is a key optimization [22]. The post itself is a deployment walkthrough covering Kubernetes, Slurm and Slinky [21]. Inside the DSX OS orchestration layer, the placement decisions stay with Dynamic Resource Allocation and the KAI Scheduler, which consume the published topology for gang scheduling [17].

What to watch

  • NodeFeatureGroupAPI graduating past alpha, which would put the NFD engine within reach of managed clusters.
  • A provider contributed from outside NVIDIA landing in upstream main for a fabric the matrix does not yet cover.
  • A published throughput comparison for the same job placed with and without Topograph's topology data.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories