Topic

Model inference serving

The business and engineering of running trained machine learning models in production, covering latency, throughput, model quality and the cost of each request.

Current clusters

build1 publisher

Baseten buys Blaxel to put the agent sandbox on the same network side as the model call

Baseten's case for owning the agent runtime is that latency and idle compute accumulate at every boundary an agent crosses between the model call and its sandbox, on numbers Blaxel reported itself.

Publishers:runtimewire.com

Reality

Evidence42
Adoption30
Hype gap+26
Incentives78
Confidence52