The Architecture Behind Our Real-Time Dashboard

The Requirements

Our client needed a dashboard that could:

• Display live metrics from 200+ IoT sensors
• Handle 50,000 concurrent viewers
• Deliver updates with sub-100ms latency
• Work on unreliable mobile networks
• Scale horizontally without downtime

Traditional polling was out. SSE was tempting but didn't meet the bidirectional requirement. We went with WebSockets backed by Redis pub/sub.

System Architecture

The system has four layers:

//Ingestion Layer

• IoT sensors push data via MQTT
• A bridge service translates MQTT → Redis Streams
• Each message is validated, timestamped, and tagged

//Processing Layer

• Redis Streams consumers aggregate raw data
• Rolling averages, min/max, and anomaly detection happen here
• Processed events publish to Redis pub/sub channels

//Distribution Layer

• WebSocket servers subscribe to relevant Redis channels
• Each WS server handles ~5,000 connections
• Horizontal scaling via consistent hashing on channel names

//Presentation Layer

• React dashboard with custom WebSocket hook
• Automatic reconnection with exponential backoff
• Local state reconciliation on reconnect

The WebSocket Hook

function useLiveMetrics(sensorIds: string[]) {
  const [metrics, setMetrics] = useState<Map<string, Metric>>(new Map())
  const wsRef = useRef<WebSocket | null>(null)
  const retryCount = useRef(0)

  useEffect(() => {
    function connect() {
      const ws = new WebSocket(WS_URL)
      wsRef.current = ws

      ws.onopen = () => {
        retryCount.current = 0
        ws.send(JSON.stringify({
          type: 'subscribe',
          channels: sensorIds
        }))
      }

      ws.onmessage = (event) => {
        const data = JSON.parse(event.data)
        setMetrics(prev => {
          const next = new Map(prev)
          next.set(data.sensorId, data)
          return next
        })
      }

      ws.onclose = () => {
        const delay = Math.min(1000 * 2 ** retryCount.current, 30000)
        retryCount.current++
        setTimeout(connect, delay)
      }
    }

    connect()
    return () => wsRef.current?.close()
  }, [sensorIds])

  return metrics
}

Performance Results

After 3 months in production:

Lessons Learned

• Redis pub/sub is fire-and-forget. If a subscriber is down, messages are lost. Use Redis Streams for durability, pub/sub for speed.
• WebSocket compression matters. Enabling permessage-deflate reduced bandwidth by 60%.
• Don't trust the client clock. Server-side timestamps saved us from hours of debugging.
• Monitor connection counts per server. Uneven distribution indicates a hashing problem.

"The fastest architecture is the one that does less. Every layer we removed made the system faster and more reliable."

What's Next

We're exploring:

• WebTransport as a WebSocket replacement (HTTP/3 based)
• CRDT-based state sync for offline-first mobile clients
• Edge computing to pre-process sensor data closer to source

If you're building real-time systems and want to avoid the pitfalls we hit, get in touch. We've learned the hard way so you don't have to.