LLM Inference Cost Optimization

Your LLM cache probably never fires, because it hashes spelling instead of meaning

A dev.to writeup traces a support bot's dead cache to hash-based lookup and replaces it with cosine similarity over embeddings at a 0.92 threshold. The lever sits in retrieval, not model choice.