A versioned symbol reference is resolved when the program loads, before any of its own code runs. That is why the crash loop happened before the orchestrator read a single line of configuration [1]. Correct config was never reachable.
So "official release binary" is a claim about the machine that built it, and it transfers only if the receiving host's glibc is at least as new as the build host's. The controller node was newer and took the release without complaint [3]. The second node runs Ubuntu 20.04, ships glibc 2.31, and could not load the same file [2]. One host set the floor for every prebuilt artifact in the cluster [6].
The fix required a rebuild on a matching host, not a configuration flag. The `rust:1-bullseye` image is Debian 11 with glibc 2.31, so a binary compiled there loads on the 2.31 node, with `protobuf-compiler` installed in the same container invocation [4][5]. Cheap once, permanent after: upgrades of that orchestrator on that host are now a compile rather than a copy [7].
The cgroup failure was arithmetic. On this Virtuozzo container `cgroup.subgroups_limit` reads 100, and a directory count under `/sys/fs/cgroup/systemd` returned 101 [9][10]. At the cap, no new unit starts regardless of the binary. Of those 101 directories, 96 held no processes at all [12]. Remove them and the live count is 5, leaving 95 slots free [20]. The author's own repeated logins to debug the box had been spending the budget he needed to start the service [12].
The third failure is the one worth copying into review notes. The release added a startup path that registered each service, and it treated services pinned to the controller's own hostname as remote placeholders, so the controller waited indefinitely for a node that was itself, without adopting the already-running containers or registering their routes [15]. According to the author, this is the same blind spot as a redeploy bug he fixed in the previous instalment: the controller's self-registration matching a placement lookup that should exclude it, now in a later code path his fix did not cover [16]. Pulling the pins out of config did not undo it, because the placeholder registration had persisted a stop-mark that the reconciler still honoured [17], and the release had also dropped the CLI `start` verb [18].
Each of the three failures traced to a different root cause [19]. What they do share is sequence: the loader runs before systemd and systemd runs before the reconciler, so each layer failed with an error naming itself rather than the upgrade that triggered it.
