Three chunking revisions, RRF and a reranker, all shipped without one retrieval number

The two columns that should have told different stories told the same one to four decimal places: hybrid retrieval and vector-only both landed on 0.9799 ^[15]. Reciprocal Rank Fusion cannot manufacture a difference out of a single non-empty list, so the fusion step and the reranker sitting above it had been dressing up a plain vector search for however long that build had been out ^[3]. The writeup's own term for the prior state is architecture taste, with the useful observation attached: taste does not fail loudly ^[23].

That reading spreads across the whole grid. Each track ran four configurations, vector only, FTS only, hybrid, and hybrid plus rerank ^[7]. Three of the four were not measuring their labels ^[20]. FTS-only measured an empty set, hybrid measured vector search, and hybrid plus rerank measured vector search with a reranker on top. One configuration out of four survived contact with the harness.

The mechanism is unremarkable, which is how it lasted. The gse tokenizer cuts the query 埃及有哪些民族？ into five tokens, question filler and question mark included, and FTS5 matches with AND semantics, so a document has to contain every one of them ^[16]. An article about Egypt contains 埃及 and 民族 and will never contain 哪些, so any query phrased as a question resolved to nothing, every time ^[17]. Neither component misbehaved. The defect lived in the seam between two correct ones ^[22], which is why the maintainer's first instinct was that the eval was broken and he had to disprove that before accepting the product was ^[21].

Against that, count what had shipped on rationale alone: five decisions, taking each revision of the chunking contract separately and adding the fusion and the reranker ^[19]. The exam that caught the fault carried three fixed metrics against fixed qrels and no LLM judge anywhere ^[8], under a rule that the same fingerprint must reproduce metrics bit for bit across runs, fresh instance and different port included ^[9]. A mock embedding with no semantics went first, to confirm the scores hugged the random baseline before any real model was trusted ^[10].

The fix is the part worth copying with care. It filters the query side, stripping punctuation, single-character function words and question fillers, and the stopword list is deliberately conservative, because over-filtering kills keyword queries in the same silence the bug did ^[18].