KV Cache and Serving Stack Optimization

Cache hit tiering across HBM and host memory, incremental tokenization, context parallelism, routing and scheduling in production inference engines.