Greenfield: Design a Real-Time Chat System

Gateway-1 receives the POST /messages request, writes to Postgres, then PUBLISHes to Redis on the channel's topic. Gateway-2 is SUBSCRIBEd to that topic (because it has a user connected who is a member of that channel). Redis delivers the published message to gateway-2, which forwards it over the recipient's WebSocket. No gateway-to-gateway communication needed — Redis is the message bus.

Long-polling: each poll is a new HTTP request. At 100k concurrent users × avg 1 poll/s = 100k HTTP requests/s. WebSocket: 100k persistent connections, ~3,333 frames/s for presence heartbeats. WebSockets have ~30x lower connection overhead at this scale. Server-Sent Events are unidirectional (server to client only) — you still need HTTP for the send path. WebSocket is the right primitive for bidirectional real-time at this scale.

(1) Validate auth token. (2) Load user's channel memberships from Redis (SMEMBERS for each channel, or a user->channels index). (3) SUBSCRIBE to each channel's pub/sub topic. (4) SET presence:{user_id} 1 EX 60. This means a gateway restart triggers a burst of subscriptions as clients reconnect — design the reconnect flow to rate-limit handshakes.

Two messages sent in the same millisecond have the same timestamp. NTP drift means different app servers may disagree on "now" by up to ~100ms. Two messages sent from different app servers within 100ms of each other could have inverted timestamps. A per-channel sequence_number from Redis INCR is monotonically increasing and has no clock dependency — it gives a total order within a channel with zero ambiguity.

INCR channel:seq:{channel_id} in Redis returns the next integer atomically. This runs before the Postgres INSERT. The sequence_number is then included in the INSERT. If the INSERT fails (e.g., idempotency_key conflict), the sequence_number is "burned" (the increment already happened). This is acceptable — clients will see a gap in the sequence (e.g., 42, 43, 45) and must tolerate gaps. Gaps mean a retry, not a missing message.

At-least-once means clients may receive the same message twice (e.g., if the gateway crashes after delivery but before the client acks). Clients must deduplicate by sequence_number: if you receive a message with a sequence_number you have already processed, discard it. This is the idempotent client pattern — it moves the dedup burden from the server to the client, which is the right trade-off for throughput.

5,000 messages/s × 1,000 members = 5M WebSocket writes/s total across all gateways. If 10 gateway servers each hold 10,000 connections (100k total / 10 gateways), and members are uniformly distributed, each gateway handles 5M/10 = 500k writes/s. At 1KB per message frame, that's 500MB/s of WebSocket throughput per gateway — budget accordingly. In practice, channels are sparse — not all 1,000 members of every channel are connected simultaneously.

Redis pub/sub delivers each message ONCE to each subscribed gateway, regardless of how many clients are connected to that gateway. Redis sees 5,000 PUBLISH/s — well within its 1M/s capacity. The gateway sees 500k WebSocket writes/s — this is the likely bottleneck. Monitor gateway CPU and send buffer backpressure, not Redis throughput.

The gateway maintains an in-memory map: channel_id → [WebSocket connections subscribed to that channel]. When a pub/sub message arrives for channel X, the gateway looks up channel X in its local map and writes the frame to each connection in the list. This is a pure in-memory fan-out — O(N) where N is the number of local connections subscribed to that channel. No Redis lookups needed for the fan-out itself.

Disconnect events are unreliable: network partitions, client crashes, and gateway crashes all result in the server not knowing the client is gone until a timeout. A TCP connection can appear alive to the server for minutes after the client goes offline. TTL-based presence (EXPIRE 60s, heartbeat every 30s) handles all failure modes uniformly: if the heartbeat stops for any reason, the key expires and the user appears offline after at most 60 seconds. This is more reliable than disconnect events.

Two steps: (1) SMEMBERS channel:members:{channel_id} to get the member list, (2) MGET presence:{u1} presence:{u2} ... for all member IDs in a single round trip. Non-nil results = online. This is O(N) for an N-member channel. For large channels (1,000 members), MGET for 1,000 keys returns in < 5ms. Online presence is computed on-demand when a user opens a channel, not pushed proactively — this avoids presence update storms.

100k users × 1 heartbeat/30s = 3,333 heartbeats/s. Each heartbeat is a POST /presence that executes SET key 1 EX 60. At 3,333/s this is trivial for Redis. The concern is not Redis throughput — it is HTTP overhead. Consider using WebSocket frames for heartbeats (client sends a ping frame every 30s) to avoid an HTTP round trip per user.

They are immediately terminated. The OS closes the sockets. Clients on mobile or behind NAT may not receive a TCP RST and must detect the crash via a heartbeat timeout (e.g., no pong response within 10s). Client reconnect logic: exponential backoff with jitter (100ms base, up to 30s), reconnect to any available gateway (via DNS load balancing or a sticky-session-free LB). After reconnect, request history replay.

The client stores the last sequence_number it received per channel. After reconnect, it calls GET /channels/{channel_id}/messages?before_sequence=... wait, it needs messages AFTER its last seen sequence. The correct API: GET /channels/{id}/messages?after_sequence= {last_seen}&limit=100. The client replays messages in sequence_number order and deduplicates any it already processed. This is the "replay on reconnect" pattern — it requires the client to persist its last-seen sequence_number across reconnects (local storage on web, SQLite on mobile).

Not lost — it is in Postgres. The write order is: Postgres first, then Redis PUBLISH. If the gateway crashes after receiving the Redis publish but before forwarding, the message is in Postgres. When the client reconnects to another gateway, the history replay fetches it. The Redis publish is a best-effort delivery mechanism; Postgres is the durable fallback. This is why "Postgres before Redis" is the critical ordering rule.