Red Hat clocks sandbox isolation at under 5 percent of an agent request's latency

Sizing a cluster for agents usually starts with a number taken from the wrong test. Red Hat's post says most inference benchmarks send a raw request to vLLM and count the tokens coming back ^[1], while a production request first goes through a harness that assembles context, manages sessions, injects tool schemas and, if you want isolation, provisions a sandbox before any token is generated ^[2].

The sub-5 percent figure is for bringing that sandbox up. Red Hat lists sandbox startup as one of its three analysis areas and defines it as the cost of provisioning isolated execution environments ^[13]. Tool calling was off in every run and every request was a single turn, a choice Red Hat made to separate framework overhead from the variable latency of tool invocations and accumulating history ^[6]. So the sandbox was provisioned with nothing to execute, and the figure does not cover what isolation costs while a container is actually running model-generated code.

C=10 here means ten drivers taking turns. Each agent is an independent OpenClaw instance driven by a dedicated GuideLLM worker sending 500 prompts per dataset to the streaming /v1/responses endpoint ^[9], and each prompt goes only after the previous response completes ^[10], so the backend sees at most ten requests in flight. Ten agents at 500 prompts is 5,000 requests per dataset in that run ^[14]. Between runs Red Hat restarted the vLLM serving runtime to clear the prefix cache ^[11].

Red Hat reports significant GPU spare capacity at that concurrency and concludes that the optimization opportunity is in the harness pipeline, not necessarily in faster hardware ^[5]. If sandbox provisioning is under 5 percent of end-to-end latency, everything else accounts for more than 95 percent, and token generation sits inside that remainder ^[15]. Splitting harness time from generation time is the job of Figure 1 and the tables; the text names context assembly, sandbox initialization and response processing among the five OpenTelemetry spans ^[12], and lists prompt expansion and SSE forwarding as framework overhead elsewhere ^[6].

For a team applying this to its own deployment, the split worth making is two per stage in the trace: whether the cost recurs once per session or once per turn, and whether it lands on the GPU or on CPU and I/O. Sandbox startup is once per session and off the GPU. Context assembly is per turn, on CPU, and grows as history accumulates. Generation is per turn and on the GPU. A run with tool calling disabled measures the once-per-session column cleanly and holds the per-turn column at one ^[6].

Red Hat says it published its methodology, tooling and results so engineering teams can benchmark and size their own OpenShift AI deployments the same way ^[18].