Four scrapers, thirty passing tests, zero rows: what a green suite actually certified

Every gate in that pipeline was reading something the scaffolder itself had written. The build counter flipped an Actor to "built" the moment `src/models.py` existed, and `models.py` is a file the scaffolder emits ^[8]. The input, output, dataset and manifest schema checks passed because the scaffolder writes valid schemas ^[7]. The linters and type checker passed because a placeholder is well-formed code ^[6]. The tests passed because they were written against the placeholder and asserted what the placeholder does ^[5]. One tool produced the artifact and the evidence, and the evidence agreed with itself.

That is how the board came to read 2 of 5 built with tests green when the delivered count was zero ^[9], and how the wave's real yield came out at four stubs from four attempts ^[3]. The only component that objected was the dataset schema, which refused the row shape the stub produced ^[2]. That check had been added for an unrelated reason, which the author calls luck rather than process, and notes that a more permissive schema would have passed a placeholder emitter straight through to the Store ^[10].

Which is roughly what happened to the fifth one. It validated input, charged the start fee, logged, and exited zero ^[12]. Eleven customer runs in thirty days, every one recorded SUCCEEDED ^[13]. On the success-rate dashboard that put it at 100 percent while the Actors under active triage sat between 69 and 89 percent ^[14], so the no-op led the fleet by at least eleven points ^[1]. A status-only metric will always rank it first, because the failure modes it measures are things that only happen to code that tries.

The gap in the account is worth reading closely. The team now also checks whether an Actor has any runs of their own; that one had none, its mandatory pre-publish QA had never executed, and all twelve of its runs belonged to customers ^[15]. Twelve total against eleven in the last thirty days leaves one run outside the window ^[4], which is to say the first paying customer arrived before the QA step that was supposed to gate the listing.

The fix is where the engineering actually is. Grepping for `placeholder` was rejected because working scrapers in the fleet talk about placeholders: a comment describing selectors as "not a placeholder hypothesis", a documented placeholder-slug URL pattern, a query string built with a `{query}` token ^[16]. The replacement walks the syntax tree and flags three shapes: a placeholder-keyed dict literal, a `run()` that yields a dict whose keys are not the Actor's declared row fields, or an explicit stub banner in the module docstring ^[17]. The middle rule is the one described as load-bearing, because it encodes the requirement itself and survives a future scaffolder that stops using the word ^[18]. The first draft walked every `return` in the module and flagged a dozen good Actors, since a helper returning a headers dict is structurally a dict literal coming out of the row module; narrowing it to yields inside `run()` fixed that ^[19]. The stated reason for the care: a gate that cries wolf gets switched off, which is worse than not having it ^[20].

The same class of defect showed up in billing. A trailing batch flush charged an event called `result` while the pricing manifest declared `result-row`, and Apify does not error on the mismatch, it logs an ignored-charge line ^[21]. Another output nobody was comparing against its own declaration.