Build1 publisher2 min readPublished
A stateless proxy bounds ZippyDB's million-host client mesh into two managed hops
Meta says routing requests through its new ZGateway tier costs about 6% extra compute on an average use case. What it buys is admission control, quota accounting and hot-read caching that a million client binaries could not deliver.
The Engineer · Build desk
What happened
- Meta's ZGateway, a stateless proxy tier between ZippyDB clients and the ZServer fleet, bounds a many-to-many connection mesh into two managed hops and batches, coalesces and admission-controls what passes through.
- Under direct access a typical client held tens of thousands of outbound connections and a typical database host accepted tens of thousands of inbound ones, most idle but still holding memory, CPU and file descriptors.
- Meta traced host crashes from file-descriptor exhaustion and out-of-memory conditions to that pattern, including a routing bug that opened a connection per shard and pushed the fleet into a reboot loop.
- The tier runs in regional tiers discovered through ServiceRouter and ships in two forms on one pipeline, a pure proxy and a read-through cache.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost The platform team pays about 6% extra compute on an average use case, on behalf of teams whose client binaries it cannot patch on any schedule it controls.
- constraint What limits scaling here is coordination. Once a client library lives in more binaries than one team can update, tuning that library stops being a change you can ship.
- capability One place can now enforce per-tenant admission control and record per-use-case quota usage, traces and metrics. Policy spread across a million client processes could not be enforced.
- decision Teams weighing a proxy tier against another round of client tuning have a published cost and coverage figure to argue from, and a defensible split of what the tier is allowed to own.
More than 1 billion operations a second through the tier, at about 40% of ZippyDB traffic, puts the store above 2.5 billion operations a second overall [1]. If the 6% overhead scales with coverage, the tier is adding roughly 2.4% to ZippyDB's total operation cost now, and about 3.6% at the projected 60% [2]. That assumes the use cases that migrate next look like the ones already on it.
Some of the extra hop is recovered inside the request path. A client holds a sticky connection to a regional gateway host, which terminates TLS, authorizes the request against the use case's ACLs, applies per-tenant admission control, validation and shaping, resolves the shard, checks a local cache where the read-through form is deployed, and batches or coalesces work headed for the same shard [10]. Two reads for one shard arriving in the same window can leave as one request. For the 6% figure to transfer to another fleet, that fleet needs enough same-shard concurrency for coalescing to pay and enough read skew for the cache to hit; without either, you pay the hop and collect nothing back.
The boundary is drawn conservatively, and I'd copy that part. TLS termination stays in the Thrift/ServiceRouter stack, and the shard locator keeps key-to-shard mapping [11]. Replica selection and hedging stay in the embedded client, which is Meta's thick C++ client with one internal client per use case [9]. The tier owns traffic management and does not become a second, subtly divergent implementation of the database client [12].
The client library was the wrong place to ship a fix. ZippyDB's clients can be more than a million hosts owned by hundreds of teams, and they cannot all be changed quickly or on one common schedule [5]. Meta's account puts the boundary this way: a direct path is sensible early in a system's life and becomes a liability once the client population grows faster than the team can coordinate client behaviour, because the library sits in too many binaries owned by too many groups [14]. Netics, writing on dev.to, describes the tier as a way to move policy, failure containment and operational visibility out of a sprawling client fleet and into a tier the platform team can control [13]. The post reports the 6% overhead but not the size of the gateway tier or any latency numbers [15].
What to watch
- Whether coverage actually passes 60% and whether the roughly 6% average holds as heavier use cases migrate onto the tier.
- Latency figures or gateway host counts from Meta, the two numbers an outside team needs to price the extra hop against direct access.
- Whether the read-through cache form spreads beyond hot-read use cases, and what hit rates it reports when it does.