Build1 publisher3 min readPublished
A Postgres default of 40 caps how many neighbors your vector search can return
pgvector ships hnsw.ef_search at 40, the size of the candidate list its graph walk keeps in flight, and a top-20 query with a tenant filter can come back with two rows and no error to explain it.
The Engineer · Build desk
What happened
- A support bot kept saying a documented answer was not covered; the row sat in Postgres with its 1536-dimension embedding, and BM25 ranked that passage second while vector top-20 missed it entirely.
- The cause was not the embedding model but a Postgres GUC left at its default, hnsw.ef_search, which bounds the candidate list the index walk keeps in flight.
- Metadata post-filtering compounds it, because a tenant predicate applied after the walk discards most of the 40 candidates and can reduce a top-20 query to two results.
- Build-time settings set a hard ceiling, and the writeup's rule is that recall still plateauing at ef_search = 500 means rebuilding with higher m and ef_construction.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost Recall trades off against latency: the sweep shows recall flattening while latency keeps rising, and someone has to choose the crossing point.
- exposure Five silent failures per hundred queries land on the wrong team: the symptom points at ingestion, and the ingestion code is fine, so the debugging time is spent before anyone reads the GUC.
- precedent Any engine migration now needs a recall baseline on both sides, or a beam-width difference gets written up as an embedding regression.
A graph walk is different from a lookup. The query vector enters the HNSW index at the top layer, where nodes are few and hops are long, moves greedily to whichever neighbor sits closer, drops a layer when it cannot improve, and at the bottom layer runs a best-first search over a candidate list [18]. That list has a fixed size, and the size is `ef_search` [2]. A chunk gets skipped when the walk never enters its neighborhood, or enters and evicts it from a list already full of nearer-looking wrong neighbors, and both get likelier as the corpus grows and the embeddings cluster [19].
You cannot return more results than you kept candidates for [21]. With the 40-candidate default and a `LIMIT` of 20, the walk carries two candidates per requested row into the final ranking [1][1]. Now apply a tenant predicate after the walk: most of those 40 are discarded, and the dev.to writeup reports top-20 collapsing to top-2 [5]. Two rows out of twenty requested is a 90 percent loss on that query [2].
Raising the beam has a ceiling that query time cannot reach. `m` and `ef_construction` are baked in when the index is built, at 16 and 64 in pgvector's defaults, and low `m` on a high-dimensional clustered corpus leaves poorly connected regions the walk struggles to enter [9][10]. The writeup gives a threshold for telling the two failures apart: if recall still plateaus below what you need at `ef_search = 500`, rebuild with higher `m` and `ef_construction` instead of adding beam [11].
Cross-engine defaults differ and cannot be compared directly. Weaviate ships `ef: -1` and derives the beam from the query limit at runtime; Qdrant takes `hnsw_ef` per search request [12]. According to the same post, a migration that looks like "worse embeddings" is very often just a different default beam width [13].
The measurement needs no second system. Run the same queries twice, once through the index and once with `SET LOCAL enable_indexscan = off`, and compare the ID sets [6]. Then sweep `ef_search`, watch the recall curve flatten while the latency curve keeps climbing, and stop where they cross [7]. Below a few hundred thousand vectors, the exact scan is milliseconds and the graph is optional [8].
"Recall@k is a number you own, not a guarantee your vector DB provides," the author wrote [17]. That is the load-bearing claim here, and it is an incident report from one corpus. It is not a measured population. For the 40 to be your bug too, you need enough vectors that the walk matters, embeddings that cluster, and a filter applied after the walk instead of inside it [19][5].
Deletes compound all of it. Most HNSW implementations tombstone rather than surgically repair the graph, so a table that has churned hard for months spends candidate slots on dead nodes. Periodic reindexing then becomes a recall operation, not only a disk-space one [14].
What to watch
- A recall sweep on your own corpus size and filter selectivity, which is the only thing that shows whether 40 is the binding constraint for your workload.
- Any pgvector release that changes the ef_search default or how candidates survive filtering after the walk.
- Whether a reindex on a long-churned table measurably recovers recall, which would confirm the tombstone cost.