Runbook Coverage Tracking
Audience: operations leads and on-call engineers tracking which Stella Ops modules have operational runbooks for their critical failure modes.
This document tracks operational runbook coverage across Stella Ops modules. It is an inventory and gap tracker — the authoritative entry point for responding to an incident is the Incident Response Runbook (see also the Incident Mode pointer).
Target: runbooks for the critical failure modes of every operationally significant module.
Maintenance note: keep the Coverage Summary and Available Runbooks sections in sync with the files in this directory. When you add a runbook, add it to both, and update the module’s status. New runbooks should follow the structure of an existing, well-formed runbook such as attestor-hsm-connection.md.
Coverage Summary
Status reflects whether a module has dedicated runbooks for its primary failure modes. “Runbooks” counts the files currently in this directory for that module.
| Module | Runbooks | Status |
|---|---|---|
| Scanner | 5 | ✅ Covered |
| Policy Engine | 6 | ✅ Covered |
| Release Orchestrator | 5 | ✅ Covered |
| Attestor | 5 | ✅ Covered |
| Feed Connectors | 4 | ✅ Covered |
| Database (Postgres) | 1 | ✅ Covered |
| Crypto Subsystem | 1 | ✅ Covered |
| Evidence Locker | 1 | ✅ Covered |
| Findings Ledger | 1 | 🟡 Partial |
| Backup/Restore | 1 | ✅ Covered |
| Vulnerability / VEX | 2 | ✅ Covered |
| Reachability | 1 | ✅ Covered |
| Air-Gap / HLC Sync | 2 | ✅ Covered |
| Replay | 1 | ✅ Covered |
| Assistant | 1 | ✅ Covered |
| Authority (OAuth/OIDC) | 0 | 🔴 Gap |
Available Runbooks
Scanner
- scanner-worker-stuck.md — worker stuck / not progressing
- scanner-oom.md — out-of-memory on large images
- scanner-timeout.md — scan timeouts
- scanner-registry-auth.md — registry authentication failures
- scanner-sbom-generation-failed.md — SBOM generation failures
Policy Engine
- policy-incident.md — policy-related incident response
- policy-evaluation-slow.md — slow policy evaluation
- policy-opa-crash.md — OPA crashes
- policy-compilation-failed.md — policy compilation failures
- policy-version-mismatch.md — policy version mismatch
- policy-storage-unavailable.md — policy storage unavailable
Release Orchestrator
- orchestrator-promotion-stuck.md — stuck promotions
- orchestrator-gate-timeout.md — gate timeouts
- orchestrator-evidence-missing.md — missing evidence
- orchestrator-quota-exceeded.md — quota exceeded
- orchestrator-rollback-failed.md — rollback failures
Attestor
- attestor-signing-failed.md — signing failures
- attestor-verification-failed.md — verification failures
- attestor-key-expired.md — key expiration
- attestor-rekor-unavailable.md — Rekor unavailability
- attestor-hsm-connection.md — HSM connection issues
Feed Connectors
- connector-nvd.md — NVD connector
- connector-osv.md — OSV connector
- connector-ghsa.md — GHSA connector
- connector-vendor-specific.md — vendor-specific connectors
Database Operations
- postgres-ops.md — PostgreSQL database operations
Crypto Subsystem
- crypto-ops.md — regional crypto operations (FIPS, eIDAS, GOST, SM)
Evidence Locker
- evidence-locker-ops.md — Evidence Locker operations
Findings Ledger
- findings-graph-asset-registry-source-unavailable.md — fail-closed recovery when the required Graph asset-registry event source is absent or inaccessible
Backup / Restore
- backup-restore-ops.md — backup and restore procedures
Vulnerability & VEX Operations
- vuln-ops.md — vulnerability management operations
- vex-ops.md — VEX statement operations
Reachability
- reachability-runtime.md — runtime reachability operations
Air-Gap / HLC Sync
- hlc-troubleshooting.md — Hybrid Logical Clock sync troubleshooting
- concelier-airgap-bundle-deploy.md — Concelier air-gap bundle deployment
Replay
- replay_ops.md — deterministic replay operations
Assistant
- assistant-ops.md — Advisory AI assistant operations
Incident Index
- incidents.md — pointer to the incident-response index and determinism checklist
Gap Analysis
Remaining gaps
- Authority (OAuth/OIDC) — no dedicated runbook in this directory yet. Candidate failure modes: token validation failures, OIDC provider unavailability, and signing-key issues. Related emergency-access guidance lives in the Break-Glass Account Runbook, but a focused Authority operations runbook is still missing.
- Findings Ledger — the Graph asset-registry source contract is covered; other Findings failure modes still need dedicated response procedures.
Doctor Check Integration
Runbooks should be linked from Doctor check remediation output so an operator who hits a failing check is routed straight to the fix. Tracking integration status:
| Module | Doctor Checks | Linked to Runbook |
|---|---|---|
| Postgres | 4 | 0 |
| Crypto | 8 | 0 |
| Storage | 3 | 0 |
| Evidence | 4 | 0 |
Next step: update Doctor check implementations to include runbook links in their remediation output.
Last updated: 2026-08-24 (UTC). Added Findings coverage for the Graph asset-registry source admission failure; per-check counts in the Doctor table are carried over from the prior revision and were not re-verified — see flagged follow-up.