Careful review passed, tsc passed, and the default dispatcher threw on every call

The author of the TypeScript package Beaver-Auth says he had put every module through multiple rounds of deliberate review before he wrote any tests, covering enumeration protection, hashed tokens, refresh rotation and TOTP replay defense ^[1]. Then 238 tests against the actual code turned up real bugs, and the two he documents in detail surfaced only because a test exercised the real dependency chain instead of a hand-built mock of his own logic ^[3][5].

The mechanism he describes is the useful part, because it is a claim you can check against your own repository. Passing tests prove only that the code does what the tests expect; if the tests were written from the same mental model as the code, both agree on the same wrong assumption and the suite happily certifies a bug as correct behaviour ^[4]. More assertions do not move that. Different inputs do.

The dispatcher bug is the clean case. Beaver-Auth defines a TaskDispatcher interface whose dispatch method takes taskName, payload, handler, and an optional onFailure ^[6]. The default implementation had drifted to a three-parameter version with payload missing entirely ^[7]. Callers still passed four arguments in the interface's order, so the payload object landed in the handler slot, the real handler function landed in onFailure's slot, and the real onFailure was silently dropped ^[8]. Calling handler() then throws TypeError: handler is not a function on every dispatch, in every deployment using the default dispatcher, which is the zero-config default ^[9]. That means every verification email and every password reset email ^[10].

According to the author, tsc --noEmit reported zero errors and no warnings, because TypeScript checks method parameters bivariantly by default and a parameter typed unknown structurally accepts anything ^[11]. A compiling signature was not evidence that the shapes agreed at the call site. The correction was one line, plus a regression test that runs the path through the real dispatcher rather than a test double ^[12].

The second bug is a reachability failure. LoginEngine exposed logoutJwtSession(familyId), which revokes a refresh-token family, and it was well tested in isolation ^[13]. But no public API path ever returned a familyId to a caller: login returned a status, the user, an accessToken and a refreshToken ^[14]. The feature was fully implemented and completely unreachable ^[15]. A unit test cannot find this, because writing one means handing the function a familyId you already have, which is precisely what a real consumer cannot do ^[16]. An integration test that role-played a consumer, logging in and then trying to log out with only what the login response provided, hit the gap on the first run ^[17]. The fix threads familyId through the LoginResult type ^[18].

One caution about the source itself: its headline says 238 tests found 4 real bugs, while the body of the same post says 8 ^[2][3]. Those cannot both be right ^[19], so treat the count as unverified. The two defects described in detail do not depend on it.

Worth watching in your own code: whether any test in the suite is constructed from a value the public API never hands out, and whether the zero-config default path, the one most users will run, is covered by a test that touches the real implementation rather than a double ^[9][12].