Meta Introduces ZGateway: A Stateless Proxy Tier That Unifies ZippyDB Traffic and Handles Over 1 Billion Operations Per Second

Details

ZippyDB is Meta's most widely deployed key-value store, backing product metadata, counters, and configuration at billions of operations per second. Without a proxy, each client directly connected to every shard it needed—sometimes tens of thousands of TLS connections per host—causing reconnection storms, OOM crashes, and one incident where a routing bug triggered a fleet-wide reboot loop from file descriptor exhaustion. ZGateway is a stateless proxy tier carrying over 1 billion operations per second, currently at 40% of ZippyDB traffic (projected to reach 60%), at roughly 6% computational overhead. The engine is Meta's existing C++ ZippyDB client run as a managed service. Requests arrive on sticky connections, pass through ACL authorization and per-tenant admission control, get batched with other in-flight work for the same shard, then forward to the correct replicas. The math is striking: with 1 million clients, 500,000 database hosts, and 50,000 shards per client, ZGateway collapses per-host connection counts by 97-98% and total persistent connections by about 19x. Direct-access fan-in scales linearly with clients; proxy fan-in scales with regions × shard density, independent of fleet size—a qualitatively different scaling curve. ZGateway also added batching, a read-through cache variant, admission control, load shedding, and safe migration flags. The stateless design means individual proxy hosts can be replaced without coordination, a major operational advantage at Meta's scale.