Elevated WebSocket reconnects in eu-west-1
A subset of realtime connections in eu-west-1 are reconnecting more often than usual. Messages are delivered, but with added latency.
opened · resolved
Post an update
pyxle-authWatching is open to everyone. Posting an official update needs a responder sign-in.
Timeline
- ResolvedMarcus · SRE ·
Reconnect rates have held at baseline since the broker node was drained. It is back in rotation with a raised connection-pool ceiling, and realtime traffic in eu-west-1 is fully recovered.
- MonitoringMarcus · SRE ·
Mitigation is in place and reconnect rates are trending back to baseline. We're monitoring before calling it resolved.
- IdentifiedPriya · On-call ·
Root cause identified: one broker node in eu-west-1 hit its connection-pool ceiling after a traffic spike. We're draining it and shifting subscribers to healthy nodes.
- InvestigatingPriya · On-call ·
We're investigating elevated reconnect rates on the realtime edge in eu-west-1. Connections recover automatically; you may see brief gaps in live updates.