Build1 publisher2 min readPublished
A million serverless briefings for $48 implies a Lambda rate of $0.000004 per GB-second
A dev.to walkthrough puts a quantized Llama 3.2 3B on up to 10,000 concurrent Lambdas and writes a million daily briefings for $48. The timing math is internally consistent. The per-GB-second price behind the $48 is unstated.
The Engineer · Build desk
What happened
- A dev.to walkthrough prices a daily briefing for a million free-tier users on a managed API at $0.01 per prompt and response, or $10,000 every morning, before tokens-per-minute throttling stretches the run into hours.
- Its alternative dumps a million rows of user context into S3 at 7:50 and has a Step Functions Distributed Map state request up to 10,000 concurrent Lambdas, each with 4GB of RAM on ARM64 Graviton.
- Inside each function's container image from ECR is llama.cpp and a quantized Llama 3.2 3B that compresses to about 2GB, well under Lambda's 10GB container limit, loaded into local memory.
- At 15 to 20 tokens a second a 50-token briefing takes about 3 seconds per user, so a batch of 30 runs 90 seconds and about 3.3 waves of 10,000 Lambdas finish a million users in roughly 5 minutes.
- The post totals 3,000,000 seconds of execution, or 12,000,000 GB-seconds at 4GB, and puts the morning's inference cost at $48 with a claimed 99% saving.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost The published calculation never names the per-GB-second rate the saving rests on. The bill scales linearly with that rate, so the finance case is only as good as the rate on the buyer's own invoice.
- constraint Output length becomes a scheduling parameter for whoever writes the product copy, because six times the tokens is six times the compute bill and six times the wall clock against a ten-minute delivery window.
- capability Batch generation becomes a capacity question inside one AWS account, and the ceiling to raise is a concurrency quota instead of a vendor's tokens-per-minute allowance.
- decision Before adopting this, someone has to decide whether 50 tokens from a quantized 3B model on a CPU is the same product the free tier was getting from a managed API.
Start with the price of a Lambda GB-second, since the whole result rests on it. The post's cost math ends at 12,000,000 GB-seconds of compute for the million users [15], and its stated result is $48 [16]. Divide one by the other and the implied rate is $0.000004 per GB-second [1]. Anyone quoting the saving should check that against the ARM rate on their own AWS bill, because the total moves in step with it. As printed, $48 against $10,000 [3] is a 99.5% cut [5], a little better than the 99% the post claims [17].
The second figure to test is 50 tokens. At 15 to 20 tokens a second on a 4GB Graviton Lambda, a 50-token insight takes about 3 seconds of compute per user [11][12]. Wall clock and bill both track output length. A 300-token briefing takes about 18 seconds per user, pushes the run from about 5 minutes [14] to about 30, and takes the compute bill to roughly $288 [3]. The pipeline starts with a 7:50 dump into S3 for an 8:00 send [4][1], which leaves ten minutes [2].
Each Lambda handles 30 users in 90 seconds [13], and the 3 seconds per user is generation time. Loading the roughly 2GB quantized Llama 3.2 3B out of the container image [6] happens once per invocation, so the batch divides it 30 ways [6]. Run one user per invocation and you pay that load a million times.
The walkthrough is candid about what the fan-out breaks. Distributed Map "requests up to 10,000 concurrent AWS Lambda executions instantly", the post says [5]. Ten thousand Lambdas writing a million rows into a standard Postgres database would exhaust the connection pool, so the output streams into Kinesis Data Firehose, and Glue converts it to Parquet before SES sends [9][10]. Anyone who has watched a fan-out job take down the database before it reached the interesting part will recognise the sequence. The CPU-local inference argument has the same shape: no HTTP call to a vendor means no tokens-per-minute ceiling to negotiate [8].
What $48 buys is 50 tokens from a quantized 3B model running on a CPU [12][6]. What $10,000 buys is a million managed-API prompt and response pairs at a cent each [3]. The post does not compare the two outputs, and on the numbers given the free tier gets $0.000048 of compute per user [4].
What to watch
- Whether an account's regional Lambda concurrency quota actually supports 10,000 simultaneous executions, or whether the run serialises into more waves.
- Whether SES will accept a million messages inside the ten-minute window at the account's sending rate.
- A published per-invocation timing for loading the roughly 2GB model, which is the input the cost per user currently omits.