Console UI Observability Runbook

Audience: SRE and operators standing up dashboards, alerts, and health checks for the Stella Ops Console.

This runbook is the operational companion to the Console observability guide: it lists the concrete Grafana dashboard, metric names, alert thresholds, and offline verification steps. Use the guide for the “what and why”; use this runbook for the “exactly which knobs.”

Dashboards (offline import)

• Grafana JSON: dashboards/console-ui-observability.json (import locally; no external data sources assumed).
• Panels to include: API latency (p95/p99), error rate, WebSocket/SSE connection count, asset load time, bundle size budget, Core Web Vitals (LCP/FID/CLS), and triage view render time.

Key metrics

• console_ui_http_request_duration_seconds_bucket{route} — API call latency.
• console_ui_http_requests_total{status} — error rate tracking.
• console_ui_websocket_connections — active live session count.
• console_ui_bundle_bytes{chunk} — bundle size by chunk (ensures offline kit budget).
• console_ui_core_web_vitals{metric} — LCP/FID/CLS gauges.
• console_ui_export_duration_seconds_bucket — export trigger to download completion.

Logs & traces

• Correlate by correlationId (propagated from API) and tenant. Include feature (triage, findings, policy) and route fields.
• Traces disabled by default for air-gap; enable by pointing OTLP endpoint to on-prem collector and setting Telemetry:ExportEnabled=true.

Health/diagnostics

• /health/liveness and /health/readiness (UI backend) must return 200; readiness checks asset storage + API gateway reachability.
• /status exposes build version, commit, feature flags; ensure it matches the offline bundle manifest when shipping sealed kits.
• Frontend self-check: open /health/ui to verify core bundles are reachable and integrity hashes match manifest.

Alert hints

• p99 API latency > 1s for /api/findings or /api/policy.
• SSE/WS disconnect rate > 2% over 5m window.
• Bundle size > 3.5 MB for main chunk after gzip (offline kit budget breach).
• Core Web Vitals: LCP > 2.5s, CLS > 0.1 on internal demo dataset.

Offline verification steps

1. Import Grafana JSON locally; point to Prometheus scrape labeled console-ui.
2. Run npm run build -- --configuration=production (or offline kit build) and verify bundle hashes against manifest used by /health/ui.
3. Fetch /status and compare commit/version to the static asset manifest embedded in the offline kit.

Related documents

• Module front doors: README, architecture, implementation plan.
• Console observability guide (concepts): observability-guide.md.
• Grafana dashboard: dashboards/console-ui-observability.json.