Build1 distinct publisher3 min readPublished
An MCP server writes its own danger label, so a harness gating approval on annotations is asking the suspect for its record. Airlock exercises the tools first, then puts the resulting policy in front of every call.
The Engineer · Build desk

build
TrueFoundry open-sources an agent harness and calls managed agents a lock-in play2 distinct publishers
invest
Google's bank agent can read FactSet. The licence, not the model, decides how much.1 distinct publisher
build
Rate limit your MCP servers, because a retrying agent turns one error into a billing incident1 distinct publisher
product
A 2x LLM bill is not a bug report: token spend is an observability problem1 distinct publisher
Compiled by The EngineerSomething wrong?How this is made
Airlock's audit produces a connector, and that connector is what enforces the findings rather than just describing them. For an approved case, Airlock emits one that points at a per-case enforcing proxy instead of back at the server it just audited, and the policy is applied before the call reaches the upstream tool [7]. That is code in the path rather than a sentence in a system prompt [7]. Annotation gating and prompt gating fail the same way, because each asks the component under evaluation to describe or restrain itself [2]. The concrete case in the writeup is a tool called `export_report`: it declares itself read-only, returns a normal result, and writes a file outside the operator's declared scope, and a harness that trusts the declaration runs it without an approval card [5]. Airlock's counter is to open a case, inventory the declared tools, exercise them under a capped budget, and compare declaration against observation [6], while stating that a missing finding is not proof of safety [4].
The fixture numbers are claims about a workload the author built. Six tools and 36 checks is six checks per tool [1]; 24 probes across six tools is four per tool [2]. Five behaviours were planted and all five were caught, in seven findings [10], which means at least two behaviours tripped more than one check [4]. "7 of 36" counts findings, and it does not translate into a detection rate you can carry to your own server. For the dishonest result to say anything about your deployment, your server would have to fail in the five ways this one was built to fail: a filesystem write behind `readOnlyHint: true`, scope escape, undeclared egress, injected instructions, canary exfiltration [9]. The zero-finding fixture, not the planted one, is the number worth carrying forward. Same six-tool surface, zero findings across the same 36 checks [11], which is what distinguishes a detector keyed on behaviour from one keyed on unfamiliarity. The deployed HTTPS target, by the way, was probed harder per tool than the fixture was, five versus four [3][2].
Here is how the state machine behind those results works. Every check resolves to `finding`, `no_finding_observed`, `not_tested` or `sensor_failed`, and severity is reported on its own axis [12]. Airlock then splits the empty cases into `capability_absent`, where no sensor existed, and `evidence_missing`, where a sensor could have answered and saw nothing [15]. Those are different engineering problems and they read identically in any tool that collapses them.
Because many MCP servers are launched as commands rather than exposed at a URL [18], the command binding is the part that can hurt the operator. The command never comes from a case argument, a tool result, or a model-generated string [19]. The operator configures a fixed map of names to argument arrays, a case may select a name, and names are looked up rather than parsed into shell commands [20]. The child starts in a throwaway working directory with an explicit environment [21]. The binding is revalidated before each connection, so repointing a configured name revokes an open case rather than silently running something else [22]. The documented boundary is the kind I trust: Airlock's HTTP response cap does not apply to the MCP SDK's stdio transport, so a large line can reach the SDK before it is bounded, though the audit deadline still holds [23].
The driving agent gets six control tools, and three of them are always approval-gated: `probe_tool`, `seal_case` and `emit_policy` [24][25]. The human is offered Block, Approve selected, or Approve all [26]. The audit's own dangerous verbs sit behind the same gate as the target's, which is the correct place to put them.
Ranked by verification strength, evidence, and original report placement.
Airlock reports what it observed, and absence of a finding is not proof of safety.
The author deliberately avoided a single safety score, on the grounds that an average can hide the one tool that lied and encourages readers to treat missing visibility as a passing result.
Launching a stdio server means executing the code Airlock exists to distrust, so the command never comes from a case argument, a tool result or a model-generated string.
The operator configures a fixed map of names to argument arrays; a case may select a name, and names are looked up rather than parsed into shell commands.
The child process starts in a throwaway working directory with an explicit environment.
Airlock revalidates the whole command binding before each connection, so repointing a configured name revokes an open case instead of silently running a different command.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 29, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Precise numbers, single witness
Every figure in this story — 24 probes, 7 findings of 36 checks, 6 tools and 30 probes against ContextFirewall, 302 passing tests — comes from the author of the tool, published on dev.to, and no one else has run it. What raises it above a launch announcement is the shape of the disclosure rather than its volume: a negative control that returns zero, a named blind spot where the HTTP response cap does not reach the stdio transport, and a self-audit that failed on Airlock's own open_case schema and was published anyway. Strong internal hygiene, zero external corroboration.
The author's runs, and no one else's
The only hands on Airlock so far belong to its author. We can see two fixture runs, one deployed HTTPS target, an inventory of five unnamed stdio servers and a test suite — and not one user, integration or deployment outside the project. The fixtures are the project's own, and the single external target is a ContextFirewall server whose relationship to the author is never stated, which makes even that data point hard to read as outside contact.
Unusual restraint, one unsupported leap
The central promise is narrower than the genre usually allows — policy applied on the wire before the call lands, rather than a sentence in a system prompt — and the language polices itself: no aggregate safety score, not_tested treated as a real state, absence of a finding explicitly refused as proof of safety. The overreach sits in a single inferential step. From ContextFirewall's silence the piece concludes that a harness resolving @write and @destructive would wave remember and forget_memory through without approval. No harness was tested; that sentence reads with the same confidence as the measured counts beside it.
Builder writing up his own product
This is the maker describing his own tool on a platform with no editorial check, and the supporting cast comes from the same orbit: a TrueForge agent driving the audit, a ContextFirewall server as the external target, Qodo credited with reviewing every substantive change, a hosted page carrying the case results. The commercial logic is legible in one line — annotations cannot be trusted, therefore you need something in the call path. That is also the strongest reason to want the ContextFirewall finding to read as alarming.
Believable, unverified, well-bounded
We would stand behind the design claims — how cases work, which operations are gated, how stdio commands are bound — because a builder is authoritative about his own code and this one names his limits unprompted. The measured outcomes are a different matter: one run, one witness, four to six probes per tool, and the external evidence amounting to a single HTTPS endpoint. Enough to take the argument seriously; not enough to repeat the numbers as settled.