Build1 publisher3 min readPublished
Missing description strings hid 21 working handlers from an in-app AI agent
One team's in-app AI agent never saw 21 working handlers because its tool manifest drops any entry without a description string. The text has to be written beside each handler's definition, so the team now runs gap tests that fail when a feature ships blank.
The Engineer · Build desk
What happened
- An in-app AI agent with working auth and write permission could not find half the handlers it needed, even though every one of those handlers existed.
- Twenty-one handlers across the AI pipeline and prompt store features had no description string, so they ran for humans in the UI but never reached the agent.
- Team members who added documentation in the app that mounts the features saw the gap tests keep failing, because the packaged features still shipped blank.
- The team runs its gap lint from a CLI against a whole app config, so a consumer feature shipped without descriptions fails before any demo.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Description text now controls what the agent can do, so writing it cannot wait for a later documentation pass without removing tools from the model.
- decision Agent visibility sits with whoever writes the packaged feature; teams that only mount it have no place to supply a description the package left blank.
- cost A missing string produces no error, so detection has to be paid for with a test per agent-visible feature and an app-wide lint.
- constraint The judgement about which writes are too dangerous for standing approval moves out of the operator's memory and into the handler definition, where a test can hold it in place.
An HTTP API is the useful comparison. According to the post, anyone who knows the path can still call an endpoint that has no OpenAPI description [5]. An agent-facing handler with no description cannot be called at all, because the model never sees a tool to choose [5]. "We treated descriptions as documentation polish. For the agent they are the gate," the author wrote [4].
The gate sits where it does because of what the agent is given. It never reads the TypeScript. It reads a manifest assembled from every feature the app mounts, covering handlers, screens and entities. Each entry takes its description from the place where the handler is defined [2]. An entry without that string is dropped, and the agent never learns the name exists [2]. Keeping the text beside the handler means every app that mounts the feature inherits it [6]. The post says half the handlers the agent needed were missing [1], but it does not give the product's total handler count.
In my view, dropping undescribed entries is the right default when the manifest is the model's only view of the product. The cost is that nothing breaks loudly. The agent just looks underpowered, and, as the author put it, nobody opens a ticket that says "missing string on handler 14" [11].
Visibility is one control. The second is what the agent may do without review. Writes default to mid risk and queries to low, and at those levels the UI can offer "always allow" [7]. The two exceptions carry `agent: { risk: "high" }` in their definitions [8]. The delete-golden handler permanently removes a golden fixture used for dry-runs. Prompt edit saves text that the owning AI feature then uses verbatim as its system prompt [8]. Neighbouring writes such as set-policy, rollback and prompt revert append a restorable revision and keep the default [9]. "Restorable side effects and irreversible ones share a form in the code; they don't share a permission ceiling," the author wrote [15]. The post's model is phone permissions, where camera access can be "always" and wiping storage cannot [10].
The tests are small. One asserts that findAgentDocGaps returns an empty list for createAiPipelineFeature([]) [12]. The other pins delete-golden to high and set-policy to mid, so a refactor cannot quietly make a hard delete eligible for "always allow" again [12].
The string does two jobs in this design. It admits a tool to the manifest [2]. In the pipeline features it also names the follow-up queries the agent will need, such as revisions before activate [14]. For the simple case, the post asks for one sentence that says what changes and what stays reversible [14].
What to watch
- Whether the team publishes findAgentDocGaps and resolveAgentExposure, so others can see how a gap and a risk level are actually computed.
- Whether a third write gets the high mark, for example if set-policy or rollback ever stops appending a restorable revision.
- Whether the lint extends to description content, given that the pipeline features rely on descriptions naming follow-up queries such as revisions before activate.