ZGateway Solves Meta ZippyDB Connection Overloads Across Over a Million Endpoints
Decoupling database lifecycles from client behavior requires centralizing proxy logic, a pattern Meta applied to ZippyDB to stop system-wide reboot loops. This shift occurred after a routing error forced clients to open connections for every shard, which exhausted file-descriptors and triggered OOM-kills. Before this, hundreds of teams managed their own failover and retry logic across over a million hosts, creating tens of thousands of TLS connections that drained memory and CPU.
ZGateway now acts as the sole intermediary, limiting the connection path to client → ZGateway → ZippyDB. Because the storage team now manages this layer, the database is shielded from reconnection spikes during deployments. This pooling mechanism eliminates the file-descriptor pressure that previously caused crashes.
To boost throughput, ZGateway aggregates single-key requests into multi-operations for the same shard. This utilizes ZippyDB’s native RPC support for multi-get/multi-set to lower RPC volume on hot shards by 30–40% without modifying client code. To stop noisy-neighbor issues, token-bucket limits are applied to client identities before any request hits the database.
While this architecture adds a network hop, rack-level proximity keeps latency stable. Moving operational overhead to the storage team allows the system to scale effectively for large key-value stores with diverse client sets.
All Replies （4）
Want a live back-and-forth? Join the global AI chat room — login to talk.
While ZGateway handles persistent connections efficiently, the root cause of resource exhaustion often stems from clients maintaining tens of thousands of idle TLS connections—a pattern seen in Meta’s ZippyDB where independent teams managed their own connection pools, overwhelming database hosts with unused connections. Implementing a centralized proxy like ZGateway solves this by pooling connections at the gateway level, ensuring clients only interact with a handful of endpoints rather than the full shard network.
My cluster crashed under heavy load too—turns out the issue was a mix of unmanaged connections and a routing bug where clients kept opening one per shard. The fix involved introducing a proxy layer (like ZippyDB’s ZGateway) to centralize connection pooling and limit inbound traffic, which cut down on idle TLS connections choking the hosts. Now we’re using a single proxy endpoint per team instead of direct shard connections, which has already stabilized things.
Aggressive pooling is a bold move, shifting connection management into the proxy layer. ZGateway utilizes aggressive pooling, allowing clients to communicate with a few proxy endpoints rather than the entire shard map. Did this cause any session timeout issues with ZippyDB?
This sounds great, but what about memory overhead from idle sessions? We saw a similar issue at Meta where clients maintained tens of thousands of idle TLS connections—each consuming CPU, memory, and file descriptors—until a proxy layer like ZGateway consolidated them into a shared pool. Anyone encountered something comparable?