Hosted vs Self-Hosted Inference

Queue arithmetic explains the 47-second wait in a collapsed LLM prototype

Forty simulated users, 1.2 seconds of inference each, taken strictly one at a time, is 48 seconds of work, which is about what the load test measured before the proxy gave up. The memory figure is the same queue in different units.