Hashing chunk IDs into five shard keys widens a DynamoDB vector search to 500 candidates

The 100 is not a ceiling on the index. When you create the table you declare the vector configuration: the attribute that holds the vectors, and the partition key for the vector index ^[6]. A query names one partition key value, and the search runs inside that partition only ^[7]. The cap applies to that call. How many partition key values exist is a design decision.

The shard key in the post is sha256 of the chunk ID, read as an integer in base 16 and reduced modulo the shard count; that spreads items evenly across five groups ^[9]. Even distribution matters twice over, because DynamoDB defines vector index throughput per partition key value, and the documentation says throughput scales linearly across those values ^[8]. The hash is deterministic, so a re-ingest puts every chunk back into the shard it came from ^[10]. Five keys go out as five queries through a thread pool sized to the number of keys ^[11], and the returned records are deduped and sorted by distance, with chunk_id breaking ties ^[12].

Five shards at 100 results each is 500 candidates ^[22], which the post calls the equivalent of a TopK=500 query ^[13]. Both searches in the comparison read the same items, because the table carried two vector indexes over that data, one with a shard key and one without ^[15].

The fan-out is not free on the caller. The author's diagnosis is client-side: five concurrent queries put five lots of client work on one CPU at the same time, and without enough of it the threads wait their turn, which is no better than running them in sequence ^[17]. "Parallel search demands a beefier caller than a single search does," the author wrote ^[18].

For that five-times figure ^[16] to transfer, your caller has to do roughly as much per-result work per core as this one: five responses of up to 100 records each, deserialised and merged in the same process. A caller with more CPU per concurrent query, or one that fans out with non-blocking I/O instead of a thread pool, is bounded somewhere else. The post reports the measurement at 1,024 MB and does not give figures for other memory sizes.

The S3 Vectors ceiling is a long way out at this shape. Ten thousand results at 100 per query needs 100 partition key values, and therefore 100 concurrent queries per user question ^[21]. If latency kept scaling the way it did at five shards on a 1,024 MB function, that search would take about 100 times a single query ^[24]. The shard count is also baked into the hash, so going from 5 to 100 means re-registering every item ^[19]. In my view five is a sensible width for a sources list, and the setting I would argue about at review is the caller's memory.