Skip to content

Build1 publisher2 min readPublished

Rejected candidates have to count as zeros before a stalled agent shows up at all

In LaunchDarkly's agentic demo, a flag set tighter than the drummer can satisfy rejects every pattern it generates. The other two agents keep publishing, so the only instrument that sees the stall is the publish rate.

The Engineer · Build desk

Illustration accompanying Rejected candidates have to count as zeros before a stalled agent shows up at all

What happened

  • LaunchDarkly's walkthrough argues that an agentic pipeline ending at a vague monitor step leaves one lever, a deploy-level rollback that takes back every other change along with the bad one.
  • The example repo runs three music agents, a drummer, a bassist and a keys player, through a generate-validate-deploy loop about eleven times a minute while a conductor holds tempo and key.
  • The demo breaks on purpose with a boolean flag, strict-mix-gate, set tighter than the drummer can satisfy, so every drum pattern is rejected, the previous bar replays, and nothing crashes.
  • An agent told to make its part more interesting can comply by turning itself up and still pass every check, so the demo puts a gain ceiling in the variation and reads the peak gain of what shipped.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint Only changes shipped as a flag or an agent config variation can be pulled back one at a time, so the scope of any future revert is fixed by how the change was authored.
  • exposure The demo has no local fallback, so LaunchDarkly's reachability decides whether the agents produce anything at all.
  • decision Picking the metric that guards a rollout becomes a per-change decision. It moves from whoever owns the dashboard to whoever ships the change.
  • cost Making the guard work costs instrumentation in every validator: each rejection has to emit an event, or the process metric only moves when the change works.

The parser in the demo asks one question about each generated pattern: whether it is well-formed [10]. On a no, the last good pattern keeps looping and that agent publishes nothing new [6]. A metric built from successful publishes alone cannot see this, because a rejection produces no event, so the ratio holds steady while one producer has stopped producing.

The size of the signal depends on how many producers you have. With three agents publishing at the same rate, one dead instrument takes roughly a third off the total [20]. A third is a big step in a series with ordinary variance. A fleet with hundreds of producers hands the same guard a much smaller number, so before this design transfers, the thing to check is the variance of your own publish rate at the window length the guard uses.

Reverting one change at a time works here because both of the things that change are runtime objects. An agent's model, parameters and system message are a variation in an AgentControl config; the code around the agent, including the limit its validator enforces, sits behind a boolean flag, and neither needs a restart [9]. Anything compiled into the build is outside that set.

There is a coupling that comes with it. The musical vocabulary is not in Shipping Floor at all: personas, gain limits and groove recipes all arrive from LaunchDarkly prompt snippets at runtime [7]. For a walkthrough that is a fair simplification. A production copy of this shape needs a cached last-known-good config on the client, because the prompt is now on the serving path.

About eleven loop cycles a minute is about 660 an hour, and near 15,800 a day if it ran continuously [19]. LaunchDarkly's own contrast is the pipeline where a person wrote every change, review was the quality system, and a few changes a day could each be read before they shipped [22].

The walkthrough is LaunchDarkly's, and the fix runs on LaunchDarkly's flags and configs [7][9]. The engineering claim under it is separable and, in my view, correct: gate each artifact on an invariant you can compute, measure the process for drift, and let a change that moves the process be taken back without a person [16]. The post's framing is that a factory gates in three places on three different clocks, and only the first is a runtime gate [18]. The third is a release control, so it can catch a change every individual gate approved [16].

What to watch

  • Whether the repo adds a cached last-known-good config so the agents keep running when LaunchDarkly is unreachable.
  • Whether LaunchDarkly publishes the threshold and window the guarded rollout uses to trigger a revert without a person.
  • Whether anyone runs this shape on a workload where the runtime invariant is more expensive to compute than well-formedness.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories