Low-latency AI

Jev collapses structured output into a single forward pass

Sean Goedecke argues that a model emitting only typed answers is fast because it skips autoregression, and that most of that speed is already available from any LLM you can prefill and constrain to a list of choices.