Build1 publisher2 min readPublished
Swapping every leaf value for a marker string clears the schema on 42% of 265 MCP tools
A fault-injection scan of 31 popular MCP servers found only 3% of their 265 tools declare an output contract that would reject a well-typed but wrong response. The failing 97% split into two different problems.
The Engineer · Build desk

What happened
- A developer built mcp-drill, a fault-injection harness that speaks MCP, and scanned 31 popular servers covering 265 tools, among them Microsoft Learn, Hugging Face, Cloudflare and DeepWiki.
- The corruption probe keeps a response's structure and types intact and replaces every leaf value with mcp-drill-corruption, -999999999 or an out-of-range value; a schema that still validates is scored vacuous.
- Ten tools carried the x-fastmcp-wrap-result marker, left by the FastMCP Python SDK default that wraps a return value as {"result": string} and treats it as the tool's output contract.
- The enforceable share held as the sample grew, reading 3% at 18 servers, 3% at 22, 2% at 26 and 3% at all 31, remote servers included.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint With a vacuous schema in place, the automatic checks reduce to transport success, so validating tool output falls to whoever writes the agent.
- contradiction 30 of the 31 servers answered unknown methods, unknown tools and missing arguments with a proper error, so a healthy error path tells a buyer nothing about the success path.
- decision Teams piping tool A's output into tool B's input have to choose between writing value-level constraints into schemas they may not own and checking results in their own client code.
A vacuous verdict is narrower than it sounds. Holding structure and type constant means the probe asks one question: does the schema constrain values? A schema that lists required keys and their types validates the corrupted payload, because nothing about the payload's shape changed. That same schema would still reject a response that lost half its fields mid-stream. The clearest case in the sample is a tool declaring `outputSchema: {type: "object"}`, which validates any object, including a corrupted one [18].
Two numbers sit under the 3% figure. 56% of the 265 tools declare no output schema, which leaves 44% declaring one [5]. Of those, about 7% reject the corrupted payload: 3 divided by 44 [1]. In absolute terms, 3% of 265 tools is roughly 8 [2].
For that ratio to transfer to your servers, your schemas have to have been produced the way theirs were. The scan uses no model and is deterministic; the author says every number is a property of the server [7]. If you hand-wrote `enum`, `pattern`, `format` or bounds constraints into your output schemas, the scan's number is not about you, and those value-level constraints are the author's stated condition for closing the gap [9].
Writing on dev.to, mcp-drill's author said that "coverage is a vanity metric here" [17]. FastMCP infers schemas from return type hints, which pushes coverage toward 100% while enforceability stays near 0% [10].
Where a server declares nothing, the client has no contract to check against, and the only automatic guard left is that the transport succeeded [11]. Where it declares a vacuous schema, validation runs and the corrupted response passes. The author's example of the second case is a `fetch` tool that returns `{"result": "ok"}` with status 200 even when the tool name is wrong [12]. "Your agent cannot tell a bad result from a good one," he wrote [16].
This is a different question from the one security scanners ask. According to the post, mcp-scan asks whether a server can be abused to do something evil, while mcp-drill asks whether a server can be trusted when it returns a result [15]. Generic HTTP fault injectors cannot run either test, because MCP is JSON-RPC over stdio or Streamable HTTP with bidirectional notifications [4]. `mcp-drill scan --json` writes the probe results for your own servers to a file, and `--badge` emits a shields.io endpoint [14].
What to watch
- Whether FastMCP makes its wrap-result default opt-in, or emits inferred schemas with value-level constraints.
- Whether the enforceable share moves off 3% as the live scorecard adds servers beyond the 31 scanned.
- Whether MCP clients start treating a vacuous outputSchema as absent instead of validating against it.