Salesforce rebuilt Security Center's agent around state, because incidents outlast conversations

Salesforce has moved the AI layer in Security Center from a standalone conversational interface to what it calls a stateful investigations platform for managing long-running security incidents across multiple telemetry systems ^[4]. The useful part of the account, published as a Q&A with Zac Taylor, a software engineering architect on the Trusted Services team, is not the model; it is the admission that chat was the wrong container for the work ^[1].

The first version did what most agent projects do first: let customers ask questions about security findings and telemetry ^[7]. Taylor says it became clear quickly that real incident response requires significantly more than a standalone chat interface, because investigations behave more like long-running operational systems than transient conversations, with a single incident spanning triage, investigation, escalation, communication, and resolution ^[7][8]. The response was scaffolding rather than prompting: investigation lifecycle management, remediation tracking, auditability, and visualization ^[9], delivered as investigation-focused workflow hubs and state-management capabilities for running an incident end to end ^[10]. Taylor's stated conclusion is that conversational AI alone is not sufficient for enterprise-grade incident response ^[11].

The second problem is verification, and it is the one worth copying. Identical prompts can produce different responses across executions, which breaks deterministic testing, and Taylor notes this matters most under the high-pressure conditions of an active incident ^[12]. Salesforce's answer is AI-driven evaluation pipelines in which AI systems validate other AI systems: large collections of simulated customer interactions are replayed through the agent, and LLM evaluators judge the output ^[13]. The design choice that makes it workable is that evaluators do not check exact wording; they check whether a response satisfies the intended investigative behavior while remaining accurate and contextually grounded ^[14]. Salesforce reports this increased testing throughput by roughly 10-20X and allows continuous evaluation of new scenarios, prompt variations, and interaction patterns ^[15]. The same release is credited with accelerating engineering velocity by 2-3X ^[3].

Treat both multiples as vendor arithmetic. They appear in Salesforce's own engineering blog with no baseline, timeframe, or measurement method attached ^[16]. The published text also stops mid-sentence in the middle of the evaluation discussion, so the account of the pipeline is incomplete ^[17]. More materially, the post does not say how investigation state is persisted, what a passing grade from an evaluator means, how many scenarios the suite contains, or how evaluator judgments are checked against human reviewers ^[18]. An evaluator that scores intent rather than strings is the right idea and also a second non-deterministic system sitting in the test harness, and nothing here explains who audits it.

The transferable lesson does not depend on the numbers. If your agent is meant to carry work that outlives a session, the engineering is lifecycle state, remediation tracking, and an audit trail, and the model call is one step inside that ^[9][10]. Salesforce reached that by shipping the chat version first and watching it fail to hold an incident ^[7].

What to watch: whether the auditability layer becomes a customer-visible artifact usable as evidence in a post-incident review, or stays internal telemetry ^[9]; whether Salesforce publishes any agreement rate between its LLM evaluators and human security reviewers ^[13][18]; and whether the Salesforce-specific telemetry gap Taylor describes as unaddressed by traditional tooling gets defined concretely enough for buyers to test ^[5].