AgentJIT compiles a traced agent run into deterministic Python after one warmup call

Call the decorated function a second time and the model is not in the loop. The trace from the first run has already become a Python abstract syntax tree, with dynamic tool dispatch replaced by hard-wired, type-checked calls and nested outputs resolved out of the data-flow graph the tracer recorded ^[6][7]. The second call consumes no tokens because there is nothing left to sample ^[10]. The design follows the pattern the post cites in V8 and in torch.compile: run once, profile, compile the hot path ^[15].

A trace records one path. The guards AgentJIT inserts check the structure of the inputs, and a violation bails out to the original LLM agent ^[8]. That covers product_id arriving as an integer, but not fetch_product returning an empty record on run 40, where an agent would have taken a different branch. The worked example in the post is a straight line: fetch_product, then calculate_vat, then generate_invoice ^[19]. Compiling that is defensible. A triage agent that picks its next tool from what the last tool returned is a different case, and the post describes guards on inputs only ^[8].

Take the 90 percent premise at face value ^[2] and the uncompiled tenth determines mean latency. Using the midpoint of the post's own 15-to-45-second range, one call in ten at 30 seconds and nine at 0.1ms average about 3 seconds, so mean latency falls around 10x, not the 300,000x the two per-call figures imply ^[17][21]. At 50,000 invocations a day, that tenth is 5,000 model runs ^[18].

Reliability compounds the same way. A 98 percent per-step success rate across four steps leaves 92.24 percent, so about 7.8 percent of chains fail ^[16][5].

What I like here is concrete: one library, no mandatory external dependencies, a pip install, @jit on the agent and @trace_tool on the tools ^[11][12]. The generated code is readable off the function object at runtime ^[13]. Printing the compiled plan is the first thing I would want, because that is what reviewing it requires. The author wrote that the guarded fallback gives "Zero crashes, zero regressions, pure speedup" ^[9]. Deopt has a price: a guard failure pays the guard check and then the full 15-to-45-second chain ^[8][3].

The post reports a 100-iteration benchmark in Google Colab against a simulated uncompiled multi-step LLM chain ^[14], so the baseline is whatever delay the simulator was configured to wait. The post gives no dataset for the 90 percent figure and calls it "the unspoken truth of agent workflows" ^[22]. For the ratio to transfer, your real chain's p50 has to sit near that simulated delay, and the repeat rate of tool sequences in your own traces has to be as high as 90 percent ^[2].