Build1 publisher3 min readPublished
Why an AI bug pipeline reclassifies tickets by integration-layer count before dispatch, not after
James Coombs's pipeline now re-tiers any ticket touching three or more integration layers before an agent starts, a rule that came out of one ticket filed as an error handling improvement that ran 5 hours and 17 iterations.
The Engineer · Build desk

What happened
- Its title looked like Tier A work, which the pipeline budgets at 70 minutes of wall clock, while the integration boundaries made it Tier B, budgeted at a full day.
- Classify now sends any ticket touching three or more integration layers to Tier B before dispatch, whatever complexity the ticket text claims.
- The CI failure pattern library started empty and reached seven recurring patterns over eight runs, with the agent applying known fixes on first encounter by run five.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision Supervision level and time budget are now committed while the agent is still idle; the revision used to come two hours into a run that was scoped at 70 minutes.
- constraint Copying the layer rule requires a classifier that can resolve a ticket to files and label those files by layer before dispatch; without that mapping the threshold has nothing to count.
- exposure The agent's own pass signal covers only the tests it chose to run, so type errors in unmodified dependent files and builds broken by stale codegen reach the PR unless a separate gate re-runs everything.
A ticket title describes a symptom. The layer count is a property of the fix, and Coombs's Classify stage gets it by reading the ticket and then examining the code [3]. The threshold is a single rule: three or more integration layers, meaning backend ORM plus API schema plus frontend component, and the ticket becomes Tier B whatever complexity was stated [4]. Five hours is 300 minutes. That is about 4.3 times the 70-minute Tier A budget, and 17 iterations spread across 300 minutes averages close to 18 minutes each [22][23]. Tier B keeps the ticket in the pipeline. The agent still implements and a human reviews the PR, and Coombs says most of the backlog lives in that middle band [13]. A Tier A misclassification therefore hands a full-day supervised job a 70-minute unsupervised budget, and the pipeline used to find out about two hours into the run [12]. The classifier has to resolve a ticket to files and know which files are the ORM, which are the API schema and which are the frontend, before any agent starts [3][4]. Where those boundaries follow directory conventions, that is a lookup; anywhere else, the threshold has nothing to count. There is engineering in the later stages too. Dispatch preloads build commands, CI failure patterns and cross-layer rebuild sequences into the implementation skill, because a generic agent finds them by trial and error at 3 to 5 CI failures per unfamiliar pattern [7]. At that rate, the seven-pattern library saves the next run 21 to 35 CI failures [24]. Seven patterns over eight runs could mean a well-behaved codebase or just a short sample [8]. The post does not report how many tickets the pipeline has classified or how often it assigns the wrong tier [25]. The case for putting the engineering into triage rests on one expensive ticket and a rule that would have caught it. Coombs wrote that "the hard problem isn't getting AI to write the fix. It's knowing which tickets to give it, how much supervision they need, and when to stop the agent before it burns hours on something that needs a human" [19]. In my view the placement carries more weight than the taxonomy: the check blocks dispatch [21]. The stage I would build first is Monitor. It re-runs the critical gates independently once the agent finishes, on the working assumption that the injected context was ignored [9]. Coombs wrote that the agent is optimizing for task completion, and that it will report that all tests pass on the strength of the subset of tests it ran [10]. The full gate is the type checker, the build and lint, and it catches tests that pass in isolation and fail in the full suite, type errors in files the agent never modified but that depend on interfaces it changed, and builds broken by stale codegen after a GraphQL schema change [11]. Two of those three cross a layer boundary. Coombs rates the discovery output above the fixes, calling it "the highest-value output of the pipeline, higher than any individual fix" [17]. During one Tier B run the agent fixed the reported bug and then found API endpoints across the codebase swallowing inner exceptions and returning raw error details to clients, none of it mentioned in the ticket [15]. A later UI alignment fix surfaced undersized form controls from a design system migration regression and redundant wrapper components across five or more files, and that one run generated three tickets of follow-on work [16]. Tier C tickets get investigation only: a plan, the affected files, a scope estimate, which Coombs puts at 30 to 60 minutes saved even when a human writes the code [14].
What to watch
- A published count of tickets classified, with per-tier error rates, would show whether three layers is the right cut or just the threshold that caught this one ticket.
- Whether the CI failure pattern library grows past seven after run eight, which tells you if the failure space saturates or the eight-run sample was simply short.
- Whether the Tier C investigation output holds up in a human's hands, and whether the claimed 30 to 60 minute saving survives a plan that names the wrong files.