Architecture Stack: Real-Time WebSocket High Availability

This is a conversation between Alex (Engineering Manager) and Jordan (Senior Architect) exploring how to design a high-availability, real-time WebSocket architecture for distributed systems running on Kubernetes.

Part 1: The Stateful Dilemma in a Stateless World

Alex: Jordan, our product team wants live telemetry updates on the dashboard. When electric vehicles plug into charging stations or update their battery state, operators expect to see telemetry stream in instantly. We picked WebSockets, but the ops team is raising red flags about deploying this on our AWS EKS cluster. What's the catch?

Jordan: The catch is the fundamental tension between modern cloud infrastructure and persistent socket connections. Kubernetes is built around stateless workloads. Pods scale up, scale down, crash, and restart behind a load balancer that usually distributes requests via round-robin.

Alex: And WebSockets don't like round-robin?

Jordan: WebSockets hate round-robin. An HTTP request is transactional: you send a request, get a response, and close the channel. A WebSocket is an enduring, stateful TCP connection. Once a client browser connects to Pod A, that connection lives exclusively in the memory of Pod A.

Alex: So what happens when a background data event arrives on Pod B?

Jordan: Exactly that. Our Event Data Processor (EDP) consumes vehicle telemetry from Kafka. Kafka distributes partitions across all EDP pods. If Pod B consumes a battery update for Vehicle #104, but the user monitoring Vehicle #104 has their WebSocket open on Pod A, Pod B has no direct way to deliver that payload to the user.

Alex: So Pod B is holding data meant for a connection anchored to Pod A?

Jordan: Exactly. Without a distributed state and routing mechanism, Pod B either broadcasts blindly to everyone or drops the message. That is why we need a dedicated WebSocket High Availability (HA) architecture.

Part 2: The Core Architectural Blueprint

Alex: Walk me through the building blocks. How do we solve this without turning our pods into stateful pets?

Jordan: We keep the application pods stateless by offloading connection stickiness to the ingress layer, state tracking to a shared cache, and cross-pod routing to a message broker.

                               ┌─────────────────────────────────────────┐
                               │           AWS Load Balancer             │
                               │          (Sticky Sessions)              │
                               └───────┬─────────────────────────┬───────┘
                                       │                         │
                          SockJS / WS  │            SockJS / WS  │
                          Handshake    │            Handshake    │
                                       ▼                         ▼
                            ┌─────────────────────┐   ┌─────────────────────┐
                            │    EKS Pod A        │   │    EKS Pod B        │
                            │   (EDP Worker)      │   │   (EDP Worker)      │
                            │  Holds Client WS    │   │  Consumes Kafka CDC │
                            └─────┬──────────▲────┘   └────┬──────────▲─────┘
                                  │          │             │          │
         Session & Asset Mappings │          │ STOMP Relay │          │
         (e.g., sessionsByVehicle)│          │ Dispatch    │          │
                                  ▼          │             ▼          │
                            ┌─────────────────────┐   ┌─────────────────────┐
                            │     Redis Cache     │   │   ActiveMQ STOMP    │
                            │  (Session Registry) │   │   (Broker Relay)    │
                            └─────────────────────┘   └──────────▲──────────┘
                                                                 │
                                                       Publishes │ Cross-Pod
                                                       Payload   │ Messages
                                                                 │
                                                      ┌──────────┴──────────┐
                                                      │  Kafka (Debezium)   │
                                                      │   Telemetry Stream  │
                                                      └─────────────────────┘

Alex: Break down each component's role for me.

Jordan: We have four primary architectural components:

1. AWS Load Balancer (ALB): Manages ingress traffic and enforces sticky sessions so multi-step connection handshakes don't break.

2. Event Data Processor (EDP) on AWS EKS: Our horizontally scalable backend pods. They terminate the WebSocket connections, process incoming Kafka events, and push messages to connected clients.

3. Redis: A centralized, in-memory registry holding user-to-asset mapping tables (such as sessionsByVehicle and charger subscriptions).

4. ActiveMQ STOMP: A broker relay running the STOMP protocol. It acts as the cross-cluster bridge, routing targeted user messages to the exact pod holding that user's socket.

Part 3: Ingress and the SockJS Handshake

Alex: Let's start at the edge. Why do we need sticky sessions at the load balancer if our backend pods are supposedly stateless?

Jordan: Because of how SockJS and WebSockets establish connections. Clients don't open a raw WebSocket instantly. First, SockJS sends an initial HTTP GET /info request to negotiate transport protocols, check capabilities, and obtain a session ID. Then it initiates an HTTP upgrade handshake or falls back to long-polling if WebSockets are blocked by corporate proxies.

Alex: And if those requests land on different pods?

Jordan: Handshake failure. If the /info request hits Pod A, Pod A initializes internal handshake state. If the subsequent WebSocket upgrade request routes to Pod B because the load balancer is round-robining, Pod B responds with an HTTP 404 or connection reset because it has no record of that handshake.

Alex: So sticky sessions pin the handshake?

Jordan: Yes. We configure duration-based sticky session cookies on the AWS Application Load Balancer. Once the client establishes the connection, the ALB guarantees that every frame from that browser session hits the same pod.

Alex: Does sticky sessions mean if Pod A dies, the user can't reconnect?

Jordan: No. Sticky sessions only bind an active session. If Pod A crashes, the ALB detects the failed target health check, removes Pod A from the target group, and routes the client's reconnection attempt to Pod B. Pod B performs a fresh handshake, and the client resumes.

Part 4: Centralized Session Tracking in Redis

Alex: Once the client connects to Pod A, what happens?

Jordan: The client sends a STOMP SUBSCRIBE frame specifying the assets they care about. For example: "I am viewing Vehicle #104 and Charging Station #12."

Alex: Does Pod A just store that in local memory?

Jordan: Pod A tracks its own local socket handle, but it cannot keep the subscription intent to itself. Pod A writes a mapping directly to Redis.

{
  "key": "sessionsByVehicle:104",
  "members": ["session-user-alpha-pod-a", "session-user-beta-pod-c"]
}

Alex: Why Redis? Why not query our relational database?

Jordan: Velocity and concurrency. When telemetry changes, we are querying these mappings thousands of times per second. Redis gives us sub-millisecond set lookups. If 50 operators across the company are watching Vehicle #104, SMEMBERS sessionsByVehicle:104 returns all 50 active session IDs in a fraction of a millisecond.

Part 5: Cross-Cluster Routing via ActiveMQ STOMP

Alex: Now the big question: an IoT event arrives from Kafka. How does it get from whichever pod consumed it to the user's screen?

Jordan: Let's trace the path:

1. A vehicle's state changes in the field. Debezium captures the change data capture (CDC) event from the database or ingestion pipeline and streams it to a Kafka topic.
2. An EDP worker on Pod B consumes this event from Kafka.
3. Pod B checks Redis: "Who is currently subscribed to Vehicle #104?"
4. Redis returns the active session IDs, including session-user-alpha.
5. Pod B doesn't know—and doesn't need to know—which pod holds session-user-alpha. Instead, Pod B publishes the telemetry payload to ActiveMQ using a user destination queue.

Alex: How does ActiveMQ know where session-user-alpha is?

Jordan: Through the Spring STOMP user registry protocol. Every EDP pod in our EKS cluster maintains a persistent STOMP broker connection to ActiveMQ. When pods spin up and users connect, the pods register active destinations via topic/user-registry.

Alex: So ActiveMQ maintains the map of which pod holds which user?

Jordan: Exactly. ActiveMQ resolves the destination through topic/unresolved-users. It inspects the destination, identifies that session-user-alpha is physically connected to Pod A, and relays the payload directly to Pod A.

Alex: And then Pod A pushes it to the browser?

Jordan: Yes. Pod A receives the message from ActiveMQ, maps it to its local WebSocket channel, and pushes the JSON payload straight down the TCP socket to the browser. The user sees their dashboard update with zero polling delay.

Part 6: Handling Silent Disconnects and Zombie Sessions

Alex: What happens when a user shuts their laptop lid or drives into a tunnel? There is no clean TCP disconnect, right?

Jordan: That is one of the classic failure modes of WebSocket systems: the silent drop. In TCP, if a client disappears without sending a FIN packet, the server-side socket can remain open in a half-open state indefinitely unless monitored.

Alex: If that happens, our Redis cache would fill up with dead sessions.

Jordan: We call those zombie sessions. Pod B would keep querying Redis, finding dead session IDs, and publishing messages to ActiveMQ for clients that are long gone. It wastes CPU, memory, and broker bandwidth.

Alex: How do we prevent that?

Jordan: Heartbeats. The STOMP protocol includes bidirectional heartbeat negotiation—for example, pinging every 10 seconds. If a client misses consecutive heartbeats, the server-side socket triggers a timeout.

Alex: What does the application do when that timeout triggers?

Jordan: Spring triggers a SessionDisconnectEvent. We have a dedicated disconnect listener inside the EDP service. The moment that event fires, the listener queries Redis and purges that session ID from all asset sets—sessionsByVehicle, sessionsByCharger, and the user session index.

Alex: So cleanup is completely automated?

Jordan: Automated and immediate. No orphan sessions lingering in Redis.

Part 7: Architectural Trade-Offs

Alex: Why use ActiveMQ STOMP as the relay? Why couldn't we just use Redis Pub/Sub for everything? We're already using Redis for session tracking.

Jordan: Good question. Many teams try Redis Pub/Sub first because it feels simpler. But Redis Pub/Sub is a pure broadcast model.

Alex: What does that mean in practice?

Jordan: In Redis Pub/Sub, when Pod B publishes a message for a user, Redis broadcasts that message to every single EDP pod in the cluster. If you have 30 pods running on EKS, 29 pods receive the message, deserialize it, check if they own the connection, realize they don't, and discard it.

Alex: That's massive CPU and network waste at scale.

Jordan: Exactly. It turns an $O(1)$ point-to-point delivery into an $O(N)$ cluster-wide broadcast storm. ActiveMQ STOMP, on the other hand, understands user destinations. It delivers the payload only to the specific broker channel connected to the pod that owns the session.

Alex: What about message persistence?

Jordan: ActiveMQ supports transactional queuing and backpressure. If a pod is momentarily saturated, ActiveMQ queues the messages rather than dropping them. Redis Pub/Sub is fire-and-forget; if the subscriber isn't ready, the message vanishes into thin air.

Part 8: Key Health Metrics and Observability

Alex: When this runs in production, what metrics should our on-call team watch on their dashboards?

Jordan: There are three critical signals:

Alex: Explain the "Active Connections per Pod" metric. Why does that drive our Horizontal Pod Autoscaler instead of standard CPU?

Jordan: WebSocket pods rarely max out CPU. They consume memory for socket buffers and file descriptors for open TCP connections. If you scale on CPU, your pods might hold 20,000 idle connections each and run out of Linux file descriptors before CPU reaches 50%. We scale on connection count.

Part 9: Operational Q&A

Alex: Let's run through the quick scenarios our team will ask during the architecture review.

How is WebSocket high availability ensured?

Jordan: Traffic is distributed across multiple EKS instances behind an AWS Load Balancer. Sticky sessions prevent multi-step handshake failures, while Kubernetes health probes automatically cycle unhealthy pods without bringing down the fleet.

How is statefulness managed across the cluster?

Jordan: Redis acts as the centralized source of truth for active subscriptions, while ActiveMQ STOMP bridges the pods. A Kafka event processed on Pod A can be routed and delivered to a user connected to Pod B with minimal latency.

How are silent connection drops handled?

Jordan: Continuous STOMP heartbeats detect severed connections. A missed heartbeat triggers a SessionDisconnectEvent, which immediately executes a cleanup routine in Redis to eliminate orphaned sessions.

Part 10: The Bottom Line

Alex: If I have to summarize this architecture to leadership in two sentences?

Jordan: We decouple WebSocket state from our container instances by combining AWS ALB sticky handshakes, Redis centralized session tracking, and ActiveMQ point-to-point routing. This gives our operators real-time telemetry streaming with zero polling overhead, while keeping our EKS pods fully scalable and resilient.

Alex: That is clean, scalable, and production-ready.

Key Takeaways