A review says Tiger Data's 42-hour pgvector build time is inflated by HNSW settings
Tiger Data says pg_vectorsearch built a 100M-vector index in 49 minutes where pgvector's HNSW took 42 hours on the same 512 GiB machine. A DEV Community review calls the query win real and the build comparison tilted. The extension is still alpha.
The Engineer · Build desk
Key figures: rekall's HNSW config sets ef_construction to 200, which the reviewer says costs a lot more build time. pgvector's default ef_construction is 64. pg_vectorsearch built its 100 million-vector index in 49 minutes on the same 512 GiB machine where pgvector's HNSW took 42 hours.
| Figure | Value | Note | Claim |
|---|---|---|---|
| rekall HNSW setting the reviewer says costs more build time | 200 ef_construction | 17 | |
| pgvector default | 64 ef_construction | 16 | |
| pg_vectorsearch index build, 100 million vectors | 49 minutes | same box as 42-hour HNSW build | 15 |
What happened
- HNSW did not finish building on the 128 GiB machine; an index built on the big box and moved over took over a second per query at 95% recall and never reached 99%.
- Tiger Data's rekall benchmark tool ships an HNSW config with ef_construction set to 200, against a pgvector default of 64.
- The launch post has no QPS, p99, concurrency, filtered-search or IVFFlat numbers, and does not publish the embedding dimension.
Why it matters
- What it limits constraint A RAG table filtered by tenant or date cannot count on PRISM returning k rows today. The docs' interim answer is a partial index on the filter, and streaming filtered search waits for GA.
- Who has to decide decision A trial needs only the extension and one index on an existing column. Teams still have to pick the HNSW settings and memory limit to benchmark it against.
- What it costs cost With no QPS or multi-client results published, sizing a production fleet on these numbers means running the concurrency tests that are listed as GA work.
PRISM is a partition index, which the README calls "conceptually similar to pgvector's IVFFlat" [1]. HNSW has to keep a big graph in memory to be fast. PRISM keeps only a small tree there and scans compact posting lists for the rest [3]. A query goes down a hierarchical centroid tree to a few clusters, candidates are filtered on RaBitQ codes of about one bit per dimension, and the survivors are reranked with full-precision vectors from the heap [2].
That structure explains the memory gap. Tiger Data's "6 ms at 100M on 4x less memory" figure sets PRISM on a 128 GiB machine against HNSW on 512 GiB, and both reach 95% recall [7]. The DEV Community reviewer thinks that pairing is fair [7].
The build-time ratio is the weaker number. 42 hours is 2,520 minutes, and 2,520 divided by 49 is about 51 [20]. The launch post does not say which HNSW config produced its table [17], or what maintenance_work_mem it ran with [18]. The review says the higher ef_construction setting buys a better graph at a large cost in build time, and that this inflates the gap [17]. pgvector's README says that once the graph stops fitting in maintenance_work_mem, the build "will take significantly more time" [18].
The 1M-vector results show how far that goes. HNSW took 21.7 hours to build on a 4 GiB machine, with a 2.05 GB index [19]. That is about 52% of the 42-hour build, for a table one hundredth the size [21]. The review reads it as an out-of-memory edge case, one that still backs Tiger Data's argument [19]. "HNSW falls over when it doesn't fit in memory" is exactly the point Tiger Data is making, the review says [8]. If your HNSW index already fits in RAM with headroom, the 1M table does not describe your setup [8].
For the 49-minute figure to transfer, your HNSW build would need the same ef_construction and enough maintenance_work_mem to hold the graph. For the 6 ms figure, your queries would need to match the post's unfiltered runs, on embeddings of unstated dimension [9]. Every figure in it is p50 latency at recall@10 against brute force [5]. On Hacker News, esafak asked: "Shouldn't you lead with your benchmarks if you're fast? qps and precision/recall, please." [11]
We think the query-side result will travel further than the build ratio will. The review worked from the post, the repo, the tuning docs, the benchmark config files and the Hacker News thread, not from a rerun [13]. Without a pgvector run at its default setting, how much of the 51x comes from settings is unknown [20]. The reviewer's call is that the extension is alpha and production is off the table for now [12].
What to watch
- Whether Tiger Data publishes the HNSW config and maintenance_work_mem behind the 42-hour run, or reruns pgvector at its default ef_construction.
- QPS, p99 and multi-client throughput results, which are listed as GA work.
- Streaming search with WHERE filters, which is on the GA roadmap.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence50
- Adoption
- Not scored
- Hype gap+30
- Incentives60
- Confidence60
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
The extension adds an index access method called PRISM (Partitioned Routing Index for Similarity Matching); its README calls it "conceptually similar to pgvector's IVFFlat".
- [2]
PRISM routes each query through a hierarchical centroid tree to a few clusters, stores vectors in those clusters as RaBitQ codes of about one bit per dimension, filters candidates on the compressed codes, then reranks them with full-precision vectors from the heap.
- [3]
HNSW has to keep a big graph in memory to be fast, while PRISM only keeps a small tree in memory and scans compact posting lists for the rest.
- [4]
The vec32 and vec16 types are binary-compatible with pgvector's vector and halfvec, so a PRISM index can be put on an existing table with CREATE EXTENSION pg_vectorsearch and CREATE INDEX ... USING prism.
- [5]
All figures in the launch post are p50 latency at recall@10 against brute force, and HNSW ran on pgvector's 16-bit halfvec.
- [6]
HNSW did not finish building on the 128 GiB box; when the index built on the big machine was moved over to it, queries took over a second at 95% recall and never reached 99%.
- [7]
The headline "6 ms at 100M on 4x less memory" is the 128 GiB PRISM number against the 512 GiB HNSW number; both hit 95% recall and one machine costs a quarter of the other, which the reviewer thinks is fair.
- [8]
The numbers are not dishonest: "HNSW falls over when it doesn't fit in memory" is exactly the point Tiger Data is making, but if an HNSW index already fits in RAM with headroom, the 1M table does not describe that setup.
- [9]
The launch post has no QPS, no p99, no concurrency results (multi-client throughput testing is listed as GA work), no filtered search, no IVFFlat numbers, and no published embedding dimension.
- [10]
Per the tuning docs, the executor applies the WHERE clause after the index has emitted its top-k and the scan inflates k based on estimated selectivity, so filters anti-correlated with vector distance can still return too few rows; the docs suggest a partial index on the filter, and streaming search with WHERE filters is on the GA roadmap.
- [11]
On Hacker News, esafak asked: "Shouldn't you lead with your benchmarks if you're fast? qps and precision/recall, please."
- [12]
The reviewer concludes that the query-side win is real and big, the build-time comparison is tilted, and the extension is still alpha, so production is off the table for now.
- [13]
The reviewer went through the launch post, the repo, the tuning docs, the benchmark tool's config files and the Hacker News thread.
- [14]
Tiger Data (formerly Timescale) published the pg_vectorsearch launch post on October 13, 2026.
- [15]
On a 64 vCPU, 512 GiB machine, pgvector took 42 hours to build an HNSW index over 100 million vectors; on the same box pg_vectorsearch built its index in 49 minutes and answered queries in half the time.
- [16]
Tiger Data's benchmark tool, rekall, ships an HNSW config (hnsw-m16.toml) with m = 16 and ef_construction = 200, while pgvector's default ef_construction is 64.
- [17]
Building with ef_construction 200 buys a better graph and costs a lot more time, which inflates the build-time gap; the post does not say which config produced the published table, so the reviewer is reading the repo, not the run.
- [18]
The post does not publish maintenance_work_mem; pgvector's README warns that once the HNSW graph no longer fits in maintenance_work_mem, the build "will take significantly more time".
- [19]
In the 1M-vector results, a 2.05 GB HNSW index on a 4 GiB machine took 21.7 hours to build and queries took 122 ms; the reviewer says this is what hitting the maintenance_work_mem limit looks like and would not call it normal HNSW performance.
- [20]
The reported 42-hour versus 49-minute build is about a 51x gap.
- [21]
The 21.7-hour HNSW build on 1M vectors is about 52% of the 42-hour build on 100M vectors, for a table one hundredth the size.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- dev.topg_vectorsearch vs pgvector HNSW: 42 Hours vs 49 Minutes. Here's the Catch
1 article · October 16, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.