Topic

Serverless LLM inference

Running language-model generation inside short-lived, per-invocation cloud functions billed by memory and duration, instead of on dedicated GPU servers or through a hosted model API.

Current clusters

build1 publisher

A million serverless briefings for $48 implies a Lambda rate of $0.000004 per GB-second

A dev.to walkthrough puts a quantized Llama 3.2 3B on up to 10,000 concurrent Lambdas and writes a million daily briefings for $48. The timing math is internally consistent. The per-GB-second price behind the $48 is unstated.

Publishers:dev.to

Reality

Evidence24
Adoption: Insufficient
Hype gap+55
Incentives60
Confidence55