Batch reporting tells you what happened yesterday. Real-time analytics dashboards tell you what is happening right now, enabling your team to respond to problems and opportunities within minutes rather than days. Whether it is detecting a sudden drop in conversion rate during a flash sale, spotting a traffic surge from a viral social post, or monitoring server health during peak load, real-time visibility transforms how eCommerce teams operate. But building dashboards that are genuinely real-time — rather than just auto-refreshing every five minutes — requires careful architectural decisions about data ingestion, processing, storage, and visualisation.
Event Streaming Architecture
Real-time dashboards depend on a streaming data pipeline that ingests, processes, and delivers events with minimal latency. The core components are an event source, a message broker, a stream processor, and a serving layer.
- Event collection: Client-side and server-side events are captured via a tracking SDK or server-side tagging and published to a message broker. Google Tag Manager server-side containers or custom event collectors built on Node.js are common ingestion points for eCommerce data.
- Message brokers (Kafka, Pub/Sub): Apache Kafka or Google Cloud Pub/Sub acts as the durable, ordered buffer between event producers and consumers. Kafka handles millions of events per second with configurable retention, ensuring no data is lost even if downstream processors lag.
- Stream processing (Flink, Dataflow): Apache Flink or Google Cloud Dataflow consumes events from the broker, applies transformations — sessionisation, aggregation, enrichment — and writes results to a serving database. Windowed aggregations let you compute metrics like "orders in the last 5 minutes" continuously.
- Serving layer (ClickHouse, Druid): Column-oriented OLAP databases like ClickHouse or Apache Druid are purpose-built for sub-second analytical queries on streaming data. They support the fast aggregation and filtering that dashboard interactions demand.
Choosing the Right Visualisation Tools
The visualisation layer must connect to your real-time serving layer and support automatic refresh or push-based updates. Grafana excels at operational dashboards with sub-second refresh rates and native support for time-series data sources. Looker Studio (formerly Google Data Studio) integrates tightly with BigQuery for teams already invested in the Google Cloud ecosystem, though its refresh capabilities are limited to one-minute intervals at best. For custom, highly interactive dashboards, frameworks like Apache Superset or bespoke React applications using libraries such as Recharts or Tremor provide maximum flexibility. The choice depends on your team's technical capabilities and how custom the dashboard interactions need to be.
Dashboard Design Principles for Real-Time Data
Real-time dashboards introduce unique design challenges. When data refreshes every few seconds, users need visual cues that help them distinguish signal from noise.
- Anomaly highlighting: Use colour-coded thresholds to flag metrics that deviate from expected ranges. A conversion rate widget should turn amber when it drops below one standard deviation from the trailing average and red when it drops below two.
- Sparklines with context: Show the current value alongside a sparkline of the last 60 minutes or 24 hours. This helps users assess whether the current reading is normal, trending, or spiking without needing to drill into a separate chart.
- Progressive disclosure: Keep the primary view focused on five to eight key metrics. Let users click through to detailed views for investigation. Overloaded dashboards cause alert fatigue and reduce the speed of response.
- Alerting integration: The most valuable real-time dashboards do not require someone to watch them constantly. Configure alert rules that push notifications to Slack, PagerDuty, or email when thresholds are breached, so the dashboard becomes a diagnostic tool rather than a monitoring obligation.
Scaling and Cost Considerations
Real-time pipelines are more expensive than batch processing. Every event is processed individually rather than in efficient bulk operations, and serving databases must be provisioned for peak query load. Manage costs by tiering your data: keep only the last 24 to 72 hours in your real-time serving layer and offload historical data to a cheaper warehouse like BigQuery for trend analysis. Use sampling for high-volume, low-criticality events — you do not need to process every page view in real time if your primary concern is order conversion. Reserve full-fidelity real-time processing for the events that actually trigger operational decisions.
Born Digital designs and builds real-time analytics infrastructure for eCommerce and digital businesses. From event streaming pipelines and OLAP database configuration to custom dashboard development, we help teams gain the operational visibility they need to act decisively in the moment.