Build1 publisher3 min readPublished
Confluent's AI pipeline analyzed all 4,700 alerts, escalating about 5% for review
Over 30 days Confluent's detection stack made about 4,700 alerts; the agent pipeline read every one, escalated roughly 5 percent, and reported more than 250 true positives with no human baseline published beside it.
The Engineer · Build desk
What happened
- Confluent's detection stack produced roughly 4,700 alerts over a 30-day run of an agentic SOC its engineers built on a stream.
- Priority-based routing sent about 3 percent of those alerts to on-call, the practice the post describes as sorting by severity and abandoning the lowest-priority items.
- The automated pipeline analyzed every alert instead, escalating roughly 5 percent and surfacing more than 250 true positives for analyst review.
- The pipeline is a central triage coordinator, specialized evidence agents, an adversarial evaluator that checks the findings, and a knowledge base that updates itself.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Reading everything raises what humans must read: about 1.7 times as many cases land in an analyst queue whose capacity Confluent says has not moved.
- contradiction Five percent of 4,700 is about 235, below the 250-plus true positives claimed, so a reader cannot tell whether escalation is the only route to a human or the percentages are loose.
- cost Full coverage buys a few minutes of agent work on every alert rather than seconds on a handful, and the post prices none of it, so an adopter discovers the per-alert cost on their own bill.
- decision Anyone choosing between this pipeline and another analyst is missing the comparison figure, because Confluent reports no count for what the human queue found in the same month.
The number that sets the staffing question is 5 percent. Roughly 5 percent of about 4,700 alerts is about 235 cases sent up for analyst review [2], against the roughly 141 that priority-based routing paged to on-call in the same month [1]. That is about 1.7 times as many cases arriving at a queue whose capacity did not change [3]. Confluent's post names the constraint plainly: "The core bottleneck remains human ability because investigation capacity stays fixed while alert volume scales relentlessly" [5].
The two headline figures do not sit flush. More than 250 true positives went to analysts [3], and 5 percent of 4,700 is about 235 [2]. Either the percentages are rounded loosely, or true positives reached analysts by some path other than escalation [4]. Confluent does not report false-positive counts, any sampling of the alerts the pipeline closed on its own, or what the 3 percent on-call queue caught over the same 30 days [18]. Without that last figure, 250-plus true positives is not the same claim as 250-plus true positives that priority routing would have dropped.
Cost control sits before the models. Duplicate alerts are filtered out first, and what survives is enriched with the team's existing institutional knowledge [8]. A triage coordinator then decides whether the alert can be resolved immediately; if not, it dispatches specialists scoped to evidence domains such as endpoint forensics or cloud audit trails, which run deep queries and return compact findings [9]. A separate evaluator agent reads those findings looking for untested hypotheses and unsupported conclusions, and sends the case back to the coordinator when it finds gaps [10]. The post does not say how many times a case can go around that loop.
The design decision worth copying is the axis of specialization. An agent per detection source is fast to stand up and, in Confluent's words, becomes "a permanent tax on your engineering team" [12]. Specializing by evidence domain instead pushes each source's quirks into data, so onboarding a new source is a configuration file and the knowledge base learns the quirks within a few alerts [13]. Past findings come back out through hybrid vector-plus-keyword search [11].
For the 4,700 to mean anything on someone else's stack, the alert mix has to look like this one. Confluent's was heavy on the signals that pile up unread: DLP alerts, high in volume and mostly benign, sitting in a backlog nobody reviews by hand [14]. The evidence also has to be reachable by query, because the unit of work is an investigation and not a classification. "An alert never contains its own verdict," the post says [6], and it budgets a few minutes per alert rather than seconds [7]. At three minutes each, 4,700 alerts is 14,100 minutes, about 235 hours, or roughly 7.8 hours of investigation a day across the month [6]. Deduplication removes an unreported share of that, and the remainder only fits because cases run in parallel [8].
Confluent weighed the three options every security team weighs, hiring more analysts and raising alert thresholds among them, and chose to automate [15]. Detection was never the bottleneck on their stack [17]. Thirty days of reading the whole stream produced more than 250 true positives for people to work [3].
What to watch
- Whether Confluent publishes what the 3 percent on-call queue caught in the same 30 days, which is the only way to size the coverage gain.
- A false-negative measure: sampling the roughly 95 percent of alerts the pipeline closed without escalating.
- Cost per investigated alert, and how often the evaluator agent sends a case back to the coordinator for a second pass.