Haiku labels three of eight patched twins as reachable vulnerabilities

The patched PHP twin casts `$_GET["id"]` to int and runs the query through `mysqli_prepare` with a bound parameter ^[3]. The function name, the `$_GET["id"]` read and the `SELECT` string are identical in the vulnerable half ^[4]. So the pair holds constant every string a keyword matcher would fire on, and varies one control ^[8].

Twin Gap is vuln accuracy minus patched accuracy, where zero means the model respected the fix and a positive number means it over-flagged ^[5]. Only the patched half can cost you anything: a detector that answers `reachable_vuln` unconditionally takes full marks on the vulnerable twins and nothing on the patched ones, for a gap of 1.0 ^[28]. "an alarm that never stops ringing doesn't help", the author wrote ^[20].

Cost per full triage run across the seven locked models runs from about $0.004 to about $0.18, which the write-up describes as roughly a 50x span ^[11][10]. Dividing the ends gives 45 ^[25]. Gemma's $0.007 run is 3.9% of the $0.18 top end ^[26], and the heavy Pro tier did not beat Flash or Gemma on this probe ^[16].

The two labels the models overturned were an escaped-input filler marked safe, which was really patched under the author's own prompt definition, and a deserialization twin where `pickle.loads` had been replaced by `json.loads`, which was really safe ^[17]. Those items held every model at 0.917 until the gold was re-adjudicated with an HMAC-gated pickle and a `len()`-only filler, after which the top cluster reached 1.000 ^[18]. The 0.083 between those two scores came from the answer key ^[27]. Published figures come from `art-label-triage` v6, on the adjudicated gold ^[23].

Eight pairs is small, and the write-up says so: one miss moves Twin Gap by 12.5%, and the author disclaims any large-N ranking ^[7]. The twins are synthetic so no model can win by recalling a CVE write-up ^[8]. The classes are modelled on WordPress-plugin-style PHP and Flask or Django request handling, and the author's stated intent is that failing a patched twin maps to over-flagging a real fix in those ecosystems ^[9]. The write-up does not report a false-positive rate from a production triage queue, so for the 0.375 to transfer, your candidate sinks would have to look like these: one function, one control, no cross-file reachability.

Two bookkeeping notes carry more weight than the table. Sonnet's proof-marker cell sat at 0.0 across retries because the provider returned an empty completion on 86 prompt tokens ^[19]. "Read the transcript before ranking a model on a single-shot cell", the author wrote ^[21]. And `qwen3-next-80b-a3b-instruct` was attempted, hit heavy-load 429s, and was swapped for Gemma, with the substitution recorded in the repository's `MODELS.md` ^[22].