HackerNoon

A semantic cache hit saves five times what a prompt cache hit saves

The provider cache discounts a repeated prefix, and the worst case is bounded arithmetic. The semantic cache deletes the call outright, but a miss there means a wrong answer, not a rounding error, so the threshold sweep matters more than the hit rate.