Fuse ranks, not scores: a retrieval contract that refuses to guess in code review

A dev.to post on code-review retrieval sets out a short prescription: run lexical and embedding search as independent candidate generators, fuse their ranks rather than their raw scores, rerank only a small merged pool, and refuse to emit a finding unless the surviving evidence still points to an accessible, current policy passage ^[1]. The author's framing is the part worth keeping: retrieval success is not the same as review success, because a missed policy lets a risky change pass while a duplicated delivery posts the same finding twice ^[3].

The score-merging objection is mechanical, not aesthetic. A lexical relevance score and a vector distance have different meanings, different ranges, and drift in calibration, so the post treats any weighted sum of the two as unsound ^[8]. The alternative is reciprocal rank fusion over the two ranked lists, deduplication by stable chunk identity, and reranking of the leading candidates against the actual diff and the review question ^[9]. Each branch is required to return document identity, chunk identity, revision, access scope, rank, and an excerpt, which is what makes deduplication and later revision checks possible at all ^[7].

The suggested sizing is 40 candidates from each branch, a fused pool capped at 60, and 12 inputs to the reranker, presented explicitly as configuration rather than a benchmark ^[10]. That is 80 gross candidates narrowed to 12 for the expensive stage, roughly 15 percent ^[17], with the cap discarding at least 20 of the 80 before reranking begins ^[18]. The author declines to defend those cutoffs in the abstract and says they should be resolved against an offline evaluation set containing exact-token cases, paraphrases, stale revisions, forbidden documents, and changes that should produce no finding at all ^[11].

Order is load-bearing. Tenant and repository authorization runs before either search, retrieval runs concurrently, fusion and deduplication follow, reranking uses the diff and the question, and revision, authorization, and evidence are validated before any generation happens ^[12]. The two search branches also have different jobs: keyword search carries exact strings such as AbortSignal, package-lock.json, rule IDs, error codes, and configuration keys, while embedding search carries paraphrases, matching "stop work after the caller disconnects" to a cancellation-propagation policy ^[6].

The operational half comes from the author's stated history of being paged for missed jobs and duplicate deliveries in cron and queue infrastructure, from which the carried-over invariant is that every stage needs an identity, a bounded retry policy, and evidence that survives replay; a high similarity score is not evidence ^[4]. So a finding is a derived record keyed by a hash of repository, commit SHA, policy revision, rule ID, and normalized code location, upserted at the sink, and explicitly not keyed by a freshly generated request ID, which identifies an attempt rather than the work ^[14]. Returning a list of strings demos well and fails in a runbook, because an excerpt cannot say which revision produced the finding, whether the caller was allowed to read it, or whether a retry reproduces it ^[13].

Watch the instrumentation advice, since it is the cheapest part to skip: per-branch candidate counts, fused and reranked counts, rejected-evidence reasons, index and policy revisions, and end-to-end duration, with an empty lexical branch kept distinct from an authorized query that simply had no lexical match, and a generator that found nothing kept distinct from a retrieval stage that silently lost its candidates ^[15].