Build1 distinct publisher3 min readPublished
A dev.to postmortem argues most RAG failures happen in retrieval. The debugging order it recommends survives scrutiny. The headline percentage it leans on does not.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Bad chunks do not page anyone. A fixed-size splitter that cut a table mid-row still returns results with plausible similarity scores, and the generator still writes fluent prose over them [8][9]. That is why the model takes the blame: it is the only component in the chain whose output a human reads and can judge wrong on sight. Everything upstream fails quietly, which is also why the author's first three moves (bigger model, tuned prompt, a bolded instruction to stay in context) changed nothing [1][2].
The 73% figure is carrying more weight than it can hold. Take it at face value and retrieval failures outrun generation failures about 2.7 to 1 [18], which is a strong claim about where the first hour of a postmortem should go. The post credits it to "industry analysis in 2026" and does not name the analysis or say what counted as a failure [3][19]. A number with no denominator and no failure taxonomy cannot distinguish "retrieval returned nothing relevant" from "retrieval returned the right section with the wrong half of it".
The same thinness runs through the two upgrades the post recommends. Semantic chunking is credited with lifting accuracy meaningfully over fixed-size on the same dataset in a published comparison [11], and prepending a heading or short summary before embedding is said to measurably improve recall [15]. Neither comes with a figure.
The ordering still holds, and it holds on repair cost rather than on any percentage. Pulling ten random chunks and reading them cold needs no infrastructure and no eval harness [14]. The pass condition is legible to anyone: can this chunk answer a question by itself, or does it only make sense beside its neighbour [12]. A model swap, by contrast, raises the price of every request you will ever serve and does nothing about an index that never contained the answer.
The decoupling instruction is the one item with hard arithmetic behind it. Indexing can take minutes per document [4]; the query path is meant to finish in under about three seconds end to end [5]. At the friendliest reading of "minutes", the offline work for a single document consumes twenty times the entire online budget [20]. Share one system between them and every re-chunk becomes downtime, so chunking becomes the layer nobody touches [6][7], and chunking is the layer most likely to be broken [8]. Naive RAG was a prototype, and this is the part of it that ossifies first [17].
The honest version of this checklist has no percentage in it at all. Cheap diagnostics go first because they are cheap, and the cheap ones happen to sit in retrieval. The 73% is decoration on a decision your own repair bill already makes.
Ranked by verification strength, evidence, and original report placement.
The author's first confidently wrong RAG answer led him to swap in a bigger model, tune the prompt, and add "only answer from the context provided" in bold; the answer got no better.
The author concluded the model was faithfully summarizing the context it was handed; retrieval had returned the wrong chunks, so the system answered the wrong question fluently.
The offline indexing path parses the source, cleans the text, chunks it, optionally enriches each chunk with context, embeds it, and writes to a vector store and a keyword index; it can take minutes per document and runs in the background.
The online query path runs on every user request under a latency budget the post puts at under roughly 3 seconds end to end, covering optional query rewriting, candidate retrieval, reranking, prompt assembly with citations, generation, and trace logging.
The post calls coupling the indexing and query paths the most common architectural mistake: if re-indexing forces the query path offline, you cannot iterate on chunking or swap embedding models without downtime, so you stop iterating.
The post's first check is whether you can re-chunk and re-embed the whole corpus without taking live search down, and it says to decouple the paths before anything else if you cannot.
Follow any of these and your For You feed starts watching them — no settings page required.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
One self-reported account; key numbers uncited
The cluster is a single practitioner post with no second source, no reproducible artifact, and no dataset. Its architectural description and self-audit steps are internally coherent and independently checkable by a reader, but its three quantitative claims — the 73% retrieval-failure share, the semantic-chunking accuracy lift, and the recall gain from contextual embedding — name no study, dataset, or metric. The supplied text also cuts off mid-section, so later pipeline links cannot be assessed.
No adoption signal in supplied material
The supplied source reports no release, deployment, benchmark run, pricing or licence change, or usage disclosure. The author's own unnamed system is the only implementation mentioned, with no scale, corpus size, traffic, or outcome data, so adoption cannot be measured without inventing facts.
Sound checklist, overstated numbers
Positive but moderate. The procedural core — debug the index before swapping the model, decouple indexing from serving, fix chunking, add a keyword index — is standard, low-risk practice and is arguably understated rather than oversold. The overstatement sits in the numbers: a precise-sounding 73% presented as a converged industry finding with no study behind it, plus two unquantified 'measurably improves' claims. A false-precision statistic doing the rhetorical work of the piece, with zero adoption evidence anywhere in the cluster, is the gap.
Reputation-building post, no visible vendor tie
The supplied text contains no product pitch, sponsor, affiliate link, or vendor attribution; tooling is named generically (BM25, RRF, vector store, keyword index) rather than by product. The remaining incentive is the ordinary one for a first-person developer-platform post: engagement and authorial credibility, which rewards a memorable round statistic and a confident checklist framing over careful sourcing. That pressure is visible in the uncited 73% but is weak compared with commercial promotion.
Confident on procedure, weak on quantities
High confidence in what the post says and in the soundness of its debugging order, which matches widely documented retrieval practice and is verifiable by the reader's own checks. Low confidence in every number it leans on, and no adoption evidence at all. Single publisher, single item, truncated body, so the assessment cannot be triangulated.
build
BM25 lost worst on the queries full of file paths. In 54% of them, the path is not in the answer1 distinct publisher
build
The model was never the bottleneck: inside a Kannada RAG rebuild that blames retrieval1 distinct publisher
build
Agent memory poisoning: 1.2% of the corpus, and both shipped defenses caught nothing1 distinct publisher
build
Inference outside the Django app is what makes a multi-tenant RAG support product operable1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 24, 2026