prompt_cache_key

Traffic past 15 requests a minute can route a prompt away from its cached prefix

OpenAI's guide prices reused input tokens at a discount of up to 90 percent. It also says cached key-value states sit on individual machines, so an unchanged prefix can still miss the cache when routing overflows.