Skip to content

Build1 publisher3 min readPublished

A provisioning control plane that knows only IP addresses cannot tell your host from its replacement

One developer's fix: a world-readable manifest the machine writes about itself, plus a read-only probe where half the possible outcomes are refusals.

The Engineer · Build desk

Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction

What happened

  • The application is a control plane that provisions plain VMs over SSH; the author writes that a wrong answer does not render badly, it reinstalls a database daemon on a machine that is serving traffic.
  • The author admits the only question the platform could actually answer about a host was "is there a row in my database with this address?"
  • That question is true of three completely different machines: one the team bootstrapped and still administers; one somebody rebuilt on the same address (same IP, brand new disk); and one belonging to a different control plane entirely.
  • Those three cases need three different answers, and the database cannot tell them apart; only the machine can.
  • The bootstrap pipeline now writes a small JSON file onto every node it touches, at /opt/control-plane/node.json.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

A developer publishing a daily build log admitted something about the control plane he maintains: the only question it could actually answer about a host was whether a row existed in its database with that address [2]. It provisions plain VMs over SSH, so a wrong answer does not render badly, it reinstalls a database daemon on a machine that is serving traffic [1]. That row, according to his write-up, is equally true of three different machines: one the platform bootstrapped and still administers, one somebody rebuilt on the same address with a brand new disk, and one that belongs to a different control plane entirely [3]. The database cannot separate them. Only the machine can [4]. So the bootstrap pipeline now writes a small JSON file onto every node it touches, at /opt/control-plane/node.json [5]. The database says it believes node X lives at an address; the file says "I am node X, administered by this control plane" [6]. The manifest carries a node uuid, a provider uuid, the control plane, stack and agent versions, a components map and an optional bootstrap timestamp [9], and it is world-readable on purpose, holding no private key, no password and no control-plane token, so an operator already on the box can answer "what is this machine?" without sudo [7]. The more interesting omission is the SSH host key fingerprint. He argues it belongs on the control plane's side, as trust state about the machine; write it onto the machine and anything that can edit the file gets to choose what it will be compared against [8]. Parsing is where the default gets set. The manifest's fromJson returns null both for a missing file and for a file it cannot read as its own, and the caller is required to treat both as not recognised [10]. A corrupt manifest is not an absent one, but it is equally not a match, and returning null for both is what forces the caller to surface it instead of quietly reinstalling over a live node [11]. Ahead of the pipeline, from every entry point that can start one, a single read-only probe reads the manifest, checks the database and returns an outcome enum [12]. Six cases: bootstrap, reconnect, upgrade, refuse-unknown-node, refuse-address-claimed, refuse-foreign-control-plane [13]. Three of them are refusals [15], which is to say half the possible answers are stop [25], and the enum owns the mayProceed predicate itself rather than asking three call sites to each remember which case is which [14]. The refusals are written to be acted on rather than merely counted. Address-claimed means a row says node X is here but the machine answering carries no manifest: rebuilt, or the address recycled, and the fix is to retire the node record [16]. Unknown-node means the machine carries this platform's manifest naming a node it has no record of, and the fix is to find out where that record went [17]. Emitting "unknown node" for the first case sends an operator hunting for a record that exists; if every refusal says the same thing, the author's phrase for what you have built is a shrug [18]. The foreign-control-plane check runs before the uuid lookup, because two control planes rotating credentials on one host is how both lose it [19]. The third piece is a version ledger, so an upgrade becomes a diff instead of re-running fourteen steps on a host that is serving traffic [20][21]. Worth watching: whether the probe holds its position at every entry point [12], since the realistic failure here is not a wrong refusal but a code path that never reaches the probe. The rest of the same log is the argument for pessimistic defaults. An SSL suite was green while exercising a path that could not reach a certificate authority, because the fake client never encodes anything, and every real issuance died malformed [23].

Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories