Build1 publisher2 min readPublished
Nine test cases pin an error taxonomy before an agent rewrites one except clause
A dev.to worked example freezes the escaping exception type, the return shape, the status integer and the WARNING count for each input to a messy dispatcher, and leaves the log message strings out of the pin.
The Engineer · Build desk
What happened
- A dev.to post argues that the error taxonomy of a mixed API is the real product surface, and that one except clause should change only after a characterization pin on that taxonomy stays green.
- Its example dispatcher has three outcomes: ValueError for bad payloads, None after a send failure, and {"ok": False, "status": 429}, and downstream code already checks all three shapes.
- The recommended fixture records four fields per input: the escaping exception type or a sentinel, the return shape, the integer status when a mapping returns, and the count of WARNING-or-higher log records.
- The post says full-file AI cleanups prefer one error type and prefer raising over returning None, and calls that preference style, not evidence.
- It says loop-style agent edits unify handlers because duplication looks sloppy, and that the duplication in those branches is "the published contract."
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint The fourth field only measures something in a codebase whose warnings already go through named loggers, so teams logging to the root have a prerequisite before they can pin anything.
- decision Recording four observable fields settles in advance what a refactor may change: branch structure is negotiable, and the caller-visible tuple is frozen until someone talks to the callers.
- capability With the pin green a reviewer can accept a large agent diff on evidence, because a collapsed shape fails an assertion; the happy-path suite would have passed the same diff.
Three of the nine cases in the worked example record the same four values [20]. A payload with no `id` returns None and logs one WARNING; so does a send that raises TypeError, and so does a send that raises TimeoutError [17]. The pin freezes the mapping from input to those four values, and each fixture is identified by its input. An agent that decides send failures ought to raise fails three tests at once.
The send call sits inside a bare `except Exception`, so a TypeError from a broken client comes back to the caller as None with one log line [14]. The post says to keep that row and calls it the trap, on the grounds that a cleaner `except` often drops it [10]. TypeError is not a subclass of TimeoutError or ConnectionError, so narrowing the clause to those two lets it escape to a caller that only checks `if result is None` [22].
The bad-JSON row runs the other direction. `"{"` fails inside `json.loads`, while `"[]"` and `"null"` parse to a list and to None, and neither is a dict, so all three arrive at the same `ValueError("bad json")` [23] with zero WARNING records [16]. Two branches, one caller-visible outcome. The empty string is separate: it raises RuntimeError before any parsing, also with zero records [12][19]. Fold everything upstream of the send under a single validation error and the empty-body caller catches something new.
Nine cases collapse to five distinct recorded outcomes [21]. Five outcomes is the whole published contract for that function, and nine assertions plus two test helpers is what it costs to write it down [20][15]. The fourth field is the brittle one. `_records` filters on `r.name == "events"`, so records from any other logger are not counted [15][24]. Anyone who has watched a test fail over a reworded log line will accept the post's advice to skip message strings on the first pass [8].
The post's diagnosis is an unrecorded taxonomy, not missing types [3]. For the recipe to transfer, two things have to hold in your code: callers really do branch on all three shapes [5], and everything they depend on is visible in exception type, return shape, status integer and WARNING count [7]. The post is explicit that its table is a labeled example and not a production trace [11]. In my view the four-field scope is the right one when the callers live in the same repository, because it records what a caller can observe and leaves the branch structure free.
What to watch
- Whether the author publishes the same four fields measured from a real dispatcher rather than the labeled example.
- Tooling that generates the pin from the call sites, which would cut the nine-assertion cost per function.
- Whether coding agents start reading characterization tests in the repo before proposing full-file handler rewrites.