Debug the index before you swap the model: the RAG checklist works, the 73% does not

Bad chunks do not page anyone. A fixed-size splitter that cut a table mid-row still returns results with plausible similarity scores, and the generator still writes fluent prose over them ^[7][8]. That is why the model takes the blame: it is the only component in the chain whose output a human reads and can judge wrong on sight. Everything upstream fails quietly, which is also why the author's first three moves (bigger model, tuned prompt, a bolded instruction to stay in context) changed nothing ^[1][2].

The 73% figure is carrying more weight than it can hold. Take it at face value and retrieval failures outrun generation failures about 2.7 to 1 ^[21], which is a strong claim about where the first hour of a postmortem should go. The post credits it to "industry analysis in 2026" and does not name the analysis or say what counted as a failure ^[18][16]. A number with no denominator and no failure taxonomy cannot distinguish "retrieval returned nothing relevant" from "retrieval returned the right section with the wrong half of it".

The same thinness runs through the two upgrades the post recommends. Semantic chunking is credited with lifting accuracy meaningfully over fixed-size on the same dataset in a published comparison ^[19], and prepending a heading or short summary before embedding is said to measurably improve recall ^[20]. Neither comes with a figure.

The ordering still holds, and it holds on repair cost rather than on any percentage. Pulling ten random chunks and reading them cold needs no infrastructure and no eval harness ^[12]. The pass condition is legible to anyone: can this chunk answer a question by itself, or does it only make sense beside its neighbour ^[10]. A model swap, by contrast, raises the price of every request you will ever serve and does nothing about an index that never contained the answer.

The decoupling instruction is the one item with hard arithmetic behind it. Indexing can take minutes per document ^[3]; the query path is meant to finish in under about three seconds end to end ^[4]. At the friendliest reading of "minutes", the offline work for a single document consumes twenty times the entire online budget ^[17]. Share one system between them and every re-chunk becomes downtime, so chunking becomes the layer nobody touches ^[5][6], and chunking is the layer most likely to be broken ^[7]. Naive RAG was a prototype, and this is the part of it that ossifies first ^[14].

The honest version of this checklist has no percentage in it at all. Cheap diagnostics go first because they are cheap, and the cheap ones happen to sit in retrieval. The 73% is decoration on a decision your own repair bill already makes.