Qdrant's rescore pass goes from 0.3 ms to 39.1 ms once the originals fall out of cache

The placement is set by one config line. Since v1.19, Qdrant sets memory placement per structure with `memory`, replacing the deprecated `on_disk` and `always_ram` flags ^[11]. Three placements exist. `cold` loads lazily from disk, so the first request that needs a page waits for it. `cached` enters the page cache when the collection loads, and the kernel may evict it later. `pinned` stays in RAM, so the structure has to fit ^[13]. Only the quantized copy can be pinned, because Qdrant reads the originals through a memory map, which leaves them `cold` or `cached` ^[14]. Set both placements explicitly, because the quantized copy defaults to following the originals ^[14]. Nobody wants a compressed copy on disk. Qdrant's recommendation is the quantized copy pinned with the originals `cold` ^[20].

Step one is the footprint. Qdrant's estimate is number of vectors x dimensions x 4 bytes x 1.5. The extra 50% covers metadata, indexes, point versions and temporary segments created during optimization, and Qdrant says the result is a starting estimate; it is not a container limit ^[7]. Put a million 768-dimension `float32` vectors through it: 1,000,000 x 768 x 4 x 1.5 comes to 4,608,000,000 bytes, about 4.29 GiB resident ^[2]. TurboQuant rotates each vector before compressing it, which spreads the error evenly across coordinates, and its `bits` parameter runs from `bits4` down to `bits1` ^[3]. At `bits4`, which Qdrant calls a good default for many workloads at eight times compression ^[4], those same raw vectors compress to 384 MB ^[3].

With rescoring off, the query ran within half a millisecond of itself at both memory limits, so the cost is in rereading original-vector pages ^[17]. Data moves between disk and RAM in fixed-size pages, typically 4 KiB on Linux ^[12]. At `oversampling` 2 with a limit of 10, the prefetch collects 20 candidates from the quantized copy, scores them against the originals and returns the best 10 ^[9]. A 768-dimension `float32` vector is 3,072 bytes, so if those 20 candidates land on 20 distinct pages, the rescore pass waits on 80 KiB of scattered reads ^[4]. In Qdrant's own measurement, rescoring added 0.3 ms at a 12 GiB cap and 39.1 ms at 4 GiB ^[16], about 130 times as much ^[1].

In hybrid search the dense and sparse reads share one page cache, and Qdrant advises rerunning your full query before you size a deployment or set a latency budget ^[6]. The sparse vector index takes the same placements and defaults to `pinned`, holding RAM the quantized copy needs ^[18]. For the 4 GiB figure to describe your deployment, dense vectors have to dominate your footprint the way they do in a single-vector collection. With a late interaction model the multivectors dominate instead, at one vector per token ^[21].

`float16` shrinks the same vectors a different way. Quantization adds a compressed copy beside the originals, while a datatype changes the originals themselves, and Qdrant says that difference decides whether anything full-precision survives to rescore against ^[5]. `rescore` and `oversampling` are query parameters, so a request can change them without touching the collection ^[10]. A collection stored as `float16` holds only the `float16` vectors, so that pass has no full-precision copy to read ^[5].

Step 4 in Qdrant's sequence needs a labeled set. Compare `nDCG@k` with `k` set to the number of results you return, pick the configuration on one part of the set, then confirm it on queries that took no part in the selection. Use `Recall@k` against exact search to explain a loss ^[19].