155 green specs, zero working requests: what a mocked model boundary cannot tell you

A mock is a written record of what its author assumes an API does, which is the developer's own description of the artifact his suite consulted 155 times without once disagreeing with him, in an account he published on dev.to for its Summer Bug Smash ^{[4] [15]}. It was accurate about everything he knew. It was silent about the two things that decided the feature: which structured-output configurations get refused at validation time, and how many optional fields the service will accept ^{[6] [7]}.

The second one is worth the arithmetic. The request asked for 66 optional fields against an allowance of 24 ^[7], which is 42 fields over the line, or 2.75 times the ceiling ^[1]. No amount of local reasoning gets you to 24. It is not a property of contracts or of wizards; it is a number somebody at a vendor picked, and the mock had no opinion about field counts because its author had none when he sat down to write it ^[8].

The shape of the failure is the expensive part. Three walls stood stacked, each invisible until the one in front of it came down ^[9]. Clearing the validation refusal did not produce a working feature, it produced the next refusal, and flattening the schema only bought passage to a timeout ^{[9] [10]}. A debugging run in that shape gives you no signal about how much is left.

The third wall is where the week goes. The policies step carries twenty interlocking fields and never came back inside the client's sixty second timeout, which was configured with a single retry ^[10]. A comfortable reading was available, which is that twenty interlocking policy fields are genuinely hard to reason about and the model wants longer than a minute, and that reading points straight at the dial ^[11].

He sent a control instead: one field, trivial content, same code path. Just as slow ^[12]. That ends the difficulty story. If a one-field request and a twenty-field request burn the same wall clock, the cost is not in reasoning about content, it is in forcing the output to satisfy the shape ^[13]. The schema was the bill.

The repair touched neither the model nor the prompt. Three changes, all to the shape of the conversation, chief among them one call per wizard step, each seeing only its own fields rather than the whole sixty-odd input vocabulary in a single request ^{[14] [5]}. Look at what that does to the number that broke everything: the largest step described, policies at twenty fields, now sits under the 24-field allowance with four fields of headroom ^[3]. The schema ended up sized to the limit that exists rather than to the layout of the wizard.

And 155 was always a count of spec files, never a measurement of the provider seam. That seam had exactly one branch that mattered, the line handing back a mock whenever NODE_ENV was test ^[3], and nothing on the far side of it could be reached by adding another spec. The instrument that found the truth was a single request with one field in it.