Your RAG Cannot Find SKU-4471, And A Bigger Embedding Model Will Not Help

A dev.to write-up by Arvid Andersson, surveying retrieval architectures as of June 2026, puts a name to the RAG bug that survives every demo ^[3]: most projects start vector-first, embedding the documents and retrieving by similarity, which works until a user searches for an exact product code, error number or specific name and gets a miss ^[1][2]. That matters because the reflex response, buying a bigger embedding model, is budget aimed at the wrong failure ^[9]. The mechanism is not subtle. An embedding turns text into a point in a space where nearby points mean similar things, which is what you want for "find documents about return policies" and exactly what you do not want for "find SKU-4471" ^[4]. The model encodes that string as something close to other codes that look and read like it, so the literal one the user typed can sit just outside the top results ^[5]. Error codes, part numbers, ticket IDs, acronyms and rare proper nouns all fail the same way ^[6]. Exact match is a different job from semantic similarity, and scaling the model that does the second does not buy you the first ^[9]. It hides well. Semantic queries pass, the demo looks finished, and the defect shows up when a real user types the one exact thing they expected to match and loses trust in the whole system ^[7]. Chat interfaces make it worse, because users phrase things literally and expect literal matches ^[8]. The architectural fix is hybrid retrieval: run a BM25 keyword query and a vector query together and merge the two ranked lists with rank fusion, which recovers exact tokens without giving up semantic recall ^[11]. The post calls this the biggest single win for most projects and the first rung to climb, with the advice to climb only as far as your evaluation says you need, because each rung costs more ^[10][11]. The merge is usually Reciprocal Rank Fusion, and it is worth understanding because it is almost trivially small ^[12]. RRF throws away the raw scores, which are not comparable anyway, and adds 1/(k + rank + 1) per list with k defaulting to 60 ^[13]. Since it never compares a BM25 score to a cosine similarity, the score-normalisation problem disappears ^[14]. The consequence is worth stating in numbers: at k=60, the top hit in a single list contributes about 0.0164, while a document sitting tenth in both lists contributes about 0.0286 ^[16]. Agreement between the two systems beats a first place in either one, and the spread across a single list's top ten is only about 15 percent ^[17]. You can buy this two ways. Weaviate and Qdrant handle BM25-based sparse plus dense vectors natively, and Pinecone pairs dense vectors with its own sparse model ^[18]. Or you take search-engine ergonomics: Typesense and Meilisearch combine full-text and vector search with typo tolerance, and Azure AI Search offers full-text, vector and hybrid in one managed service ^[19]. The choice turns on whether you already run a vector database or want faceting, typo tolerance and geo alongside retrieval ^[20]. Hybrid widens recall but leaves the ordering approximate, which is what rerankers are for: a cross-encoder reads the query and each candidate together and reorders them, and it is the next biggest quality jump once recall is good but precision lags ^[21]. Two things to watch. First, licensing: Jina AI publishes multilingual reranker weights on Hugging Face under CC-BY-NC, so commercial use goes through the API ^[23]. Second, consolidation. Jina has been part of Elastic since October 2025 and Voyage AI now sits inside MongoDB, meaning both named reranker vendors are now owned by infrastructure companies ^[24][26].