Loading today’s stories
Build1 distinct publisher3 min readPublished
A dev.to walkthrough puts deterministic tool tests at the base of a four-layer agent pyramid and saves live Gemini runs for the top. It works only if you own the adapter that turns framework events into a contract.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
`summarizeTrajectory` is the load-bearing piece here, and the author is explicit that it belongs to your application rather than the framework [12]. It consumes the array of events collected from `runner.runAsync` and returns four fields: toolCalls, blockedActions, clarificationRequested, finalText [11][12]. The example behavioural test reads two of them, the tool call names and blockedActions [13]. finalText stays in the type and goes unasserted, demoted to documentation after being the thing that turned the suite red in the first place [2][1].
One layer down, the contract is a schema you can read off the page. `city` is `z.string().min(2)`; `maxNightlyPriceUsd` is `z.number().positive().optional()` [7]. Those bounds are exercised in vitest with the hotel gateway mocked and no model in the process [8]. That is the author's argument about permissions, data mapping, error normalization and idempotency: none of them turn probabilistic because a model chose the tool [9].
Two artefacts in the listing read like guards and are actually prompt text. The tool description says the tool never creates a booking [7]. The agent instruction says never call `book_hotel` without explicit confirmation [10]. Both are strings the model consumes.
That is where I would push at review. The trajectory test asserts `blockedActions` equals `[]` [13], and because the reducer is yours, that array holds whatever you decide to put in it [12]. If it is derived only from events the model already emitted, a green test says the model behaved on that run, not that `book_hotel` was unreachable. The source's line about the prohibited side effect not varying [14] holds to the extent your runtime enforces it, and the walkthrough's enforcement point is a sentence in the instruction block [10].
The cheapness claim also belongs to the bottom half of the diagram. Layers one and two run against plain TypeScript [8][9]. The trajectory layer constructs an `LlmAgent` on `gemini-2.5-flash` and drives it through `InMemoryRunner` [10], so every row in the scenario table with its `requiredTools` and `forbiddenTools` [16] is a live model call. Human-reviewed evals sit above that [4]. Two of four layers are deterministic [2], which makes "most tests near the bottom" [5] a budget instruction as much as a design one.
For this to transfer, your harness has to hand you tool calls as structured events, not just a final message. ADK does, because events are runtime primitives alongside sessions and orchestration [3]. If your framework returns text and a token count, layer three is unavailable and you are back to matching sentences. The pattern is not really about Gemini; it is about whether the event stream is rich enough to reduce into a contract, and whether you keep the reducer small enough that one upgrade touches one file.
The scenario table is the part I would copy first [16]. Naming a case, its input, its required tools and its forbidden tools turns policy into rows, and rows are reviewable by someone who does not read TypeScript. The author's framing is that production failures sit at the boundary between a plausible request and an unsafe action [15], and a table is where that boundary can be enumerated instead of imagined.
Ranked by verification strength, evidence, and original report placement.
The article's opening assertion: the fastest way to make an AI-agent test flaky is to assert the final sentence.
Worked example of the flake: the test expects "I'll help you find hotels in Paris." and the agent returns "Sure - I can look for hotel options in Paris."; the behavior is correct but the test is red.
Google's Agent Development Kit represents agents, tools, orchestration, sessions, events, evaluation and deployment as code and runtime primitives; the author says this does not make the model deterministic but gives better places to establish deterministic contracts around it.
The proposed agent test suite has four layers, bottom to top: deterministic unit tests for tools and adapters; runtime contracts covering policy, state and schema; end-to-end trajectory scenarios; a small set of human-reviewed evals.
The author advises that most tests should live near the bottom of the pyramid because they are fast, cheap and deterministic, and that live-model evaluations should be used deliberately rather than for every assertion.
ADK TypeScript tools can be expressed with FunctionTool plus a Zod parameter schema, and the business function underneath remains ordinary TypeScript that should be tested as such.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 31, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
Model-generated tool arguments cross the same trust boundary as an HTTP request1 distinct publisher
build
The A2A card said 0.0.0.0:8080, and the only caller it broke was Google's1 distinct publisher
build
Armenian ASR leaderboard: closed models take the top eight, then lose the domains that matter1 distinct publisher
build
The refund that fired three times: tool calls are a systems problem, not a prompt problem1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Self-verifying code, unverified framework
The code in this story is its own witness — you can read the Zod schema, the vi.spyOn call, the loop over runAsync — and that internal consistency is real. What no one has checked is whether ADK's TypeScript surface behaves as rendered, since Google's documentation never enters the reporting and no ADK version is pinned. Nor is there a single test run, flake count or incident to show the suite works. Strong as a design argument, thin as verification.
No adoption signal
One tutorial's illustrative code is not uptake. Nothing in this reporting names a team, a repository, a shipped agent or a version of ADK, and there is no telemetry, download figure or third-party account to stand next to it. We would rather say nothing than read a walkthrough as traction.
Modest claims, quietly load-shifted
dev.to hedges where it matters — it says plainly that ADK does not make the model deterministic, and later that schema validity is not semantic correctness. The overreach is structural rather than rhetorical. A pyramid sold on determinism spends its top two layers on live model calls, and the showcase test asserts that blockedActions is empty while the policy machinery that would populate it is never written. Small gap, but it points the same direction: the hard half is left to the reader.
Reputation, not revenue
Nothing is being sold here. There is no vendor byline, no product placement, no pricing, and the incentive is the ordinary one of a developer-platform post: publish a clean, quotable pattern and be seen doing it. That pressure shows up as single-framework framing — Google's kit throughout, no alternative considered, no ADK rough edges named beyond the model's nondeterminism.
Believable pattern, single witness
We are fairly sure what the piece says and reasonably sure the pattern is sound engineering; we are much less sure the framework details survive contact with a current ADK release. One publisher, one author, no corroboration, and no adoption to triangulate against — enough to act on as a design idea, not enough to cite as fact about ADK.