Topic
Testing the advertised schema and the actual payload across a real transport instead of asserting on in-process return types.
Google, Anthropic and OpenAI shipped four model releases in 12 days, and each one's own docs list ways that code written for the earlier version now fails. A swap of the model ID is a dependency upgrade and needs contract tests at the provider boundary before it ships.
Reality
Three of the four layers in a dev.to testing pyramid for MCP servers are plain pytest checks on schemas, error envelopes and session expiry. Only the fourth puts a model in the loop, and the available text breaks off before describing it.
Reality
A dev.to walkthrough maps a fully green Karate suite back to its OpenAPI paths and finds half the customer operations with no scenario behind them, including both destructive methods on the resource.
Reality
A dev.to case study freezes four error codes, with their HTTP status, retry flag, message key and log level, into one JSON file, hash-checks it in CI, and then lets the generator rewrite the mapper as often as it likes.
Reality
The new tools spec lets structuredContent be any JSON value. A client hard-coded to read structuredContent.result keeps compiling, keeps passing, and starts finding nothing on the wire.
Reality