Cloudflare's K2 trades a second of produce latency for an event log kept in R2

Object stores cannot append, and a log is a sequence of appends ^[2]. K2 works around this by holding incoming writes in memory on an edge service. It waits briefly for more to arrive, then writes the batch to R2 as one segment file ^[2]. That wait is where the second of p99 produce latency comes from ^[1][2].

The length of the wait is also a billing setting. Commenter sensodine, who disclosed that he works on the competing service s2.dev, wrote: "One of the tensions of course is how long to linger before flushing to object storage - you have to trade off directly between latency and cost of your API ops for PUTs." ^[3]

The design has few moving parts. R2 provides eleven nines of durability behind strongly consistent APIs ^[8]. Its atomic operations give K2 ordering and incrementing offsets, so no coordination service runs at all ^[9]. Replication and consensus live in the storage layer, and compute scales separately from storage ^[8]. I think this is the right call for the problem Cloudflare had. K2 began as a place for Pipelines to park events before its pull-based engine reads and transforms them ^[6]. Pipelines runs on small slices of ephemeral machines in more than 335 cities, over the public internet, and InfoQ reports that Kafka assumes none of that ^[7].

The one-second figure covers produce ^[1]. For it to describe what a consumer sees, the read path has to keep pace. In the beta it has not yet, according to one user. Posting as e1g, that user reported writes under a second but end-to-end delivery at p95 2.5 seconds and p99 7.5 seconds ^[10]. That tail is about 7.5 times the produce figure ^[19]. A commenter answering for the team said it was higher than they would expect, that the Consume API needs performance work, and that read latency improvements should appear within about a week ^[11].

Pricing sets the second condition. K2 charges $0.04 per GB produced and the same again per GB consumed ^[14]. Commenter nnx wrote: "This means actual usage is $0.08/GB in the simplest case (one consumer) but fan-out consumer strategies get very expensive very fast." ^[15] Each added consumer costs another $0.04 per GB. Five consumers bring a stream to $0.24 per GB, triple the one-consumer price ^[20]. Cloudflare positions K2 for fan-out ^[16]. Retention costs $0.02 per GB per month, and nothing is billed during the beta, so for now fan-out is free ^[17].

nnx still made the cost case. He said K2 beats Google Pub/Sub on cost, especially for long retention, and runs much cheaper than self-hosted Kafka or hosted versions such as Amazon MSK and Confluent, with no clusters to manage ^[22]. He put both produce and end-to-end latency at around one second p99 ^[4].

Several systems now use the same pattern. Thread participants asked how K2 differs from AutoMQ and WarpStream, and noted that Kafka now supports diskless topics ^[18]. Another commenter, psanford, wrote: "Object store is quickly becoming the new core data substrate. Lets build kafka, but on s3. Lets build github, but on s3." ^[24] The post's author, necubi, who said he is K2's tech lead, agreed that every data system without a sub-100ms requirement is moving to object storage ^[23].

The s2.dev engineer argued the pattern can go lower. He said s2.dev's stateful backends keep flushing multi-tenant objects holding records from many streams, reaching roughly 50ms p99 acknowledgment latency within a region ^[12]. His claimed figure is one-twentieth of K2's stated produce p99 ^[21]. He pointed to S3 Express and GCS rapid buckets as faster tiers, though both are single-zone and still need quorum writes for regional durability ^[13].