Embedding Inference Infrastructure

Google's vLLM TPU embedding numbers: 83,996 tokens/s, and a 0.999 cosine gate

The throughput figure is one configuration point. The parity thresholds are the number that decides whether your index is allowed to move off GPUs.