Stella Ops Graph

The Graph module — Graph Indexer plus Graph API — builds the tenant-scoped knowledge graph that powers blast-radius analysis, provenance timelines, and saved-query automation across Stella Ops. It is the home for everything graph-related: ingestion, storage, overlays, and query surfaces. (Cartographer was retired on 2025-10-30; see docs-archive/implplan/updates/2025-10-30-devops-governance.md.)

Runtime cutover status

• Non-testing Graph API hosts now require Graph PostgreSQL configuration; only Testing may boot with the deterministic empty in-memory repository.
• Saved views are durable in non-testing hosts via graph.saved_views; the in-memory saved-view store is harness-only.
• Live Graph query/search/path/diff surfaces remain repository-backed, but live overlays, export jobs, and edge-metadata explanations no longer fabricate in-process data. Non-testing hosts return truthful 501 for those routes until durable backends land.
• Live audit logging is log-only and no longer keeps in-process request history.

Scope & responsibilities

• Ingest SBOM snapshots, advisory/VEX events, policy overlays, and runtime signals to maintain a first-party graph representation with deterministic node/edge identities.
• Serve APIs and saved-query tooling for impact analysis, dependency traversal, diffing, and policy/VEX overlays with explainable provenance.
• Supply Graph Explorer UI/CLI experiences, plus Offline Kit exports (nodes.jsonl, edges.jsonl, overlays/) with DSSE manifests for air-gapped replay. Analytics overlays are emitted as NDJSON (overlays/clusters.ndjson, overlays/centrality.ndjson) with deterministic ordering; PostgreSQL-backed providers support production wiring.
• Maintain the Graph Index Canonical Schema and coordinate query/overlay lifecycle with Scheduler, Policy Engine, Vulnerability Explorer, and Export Center.
• Own the Graph Asset Registry as the shared NIS2 inventory, DORA RoI projection, and TLPT scoping substrate.

Architecture snapshot (Sprint 30 groundwork)

• Graph Indexer service — consumes SBOM (sbom_snapshot), advisory, and VEX events; normalises identifiers; persists into graph_nodes, graph_edges, graph_snapshots, and overlay caches with tenant partitions.
• Graph API service — exposes NDJSON POST /graph/search|query|paths|diff, POST /graph/lineage, POST/GET /graph/export[/{jobId}], the /graph/edges/* metadata surface, the /graph/assets* registry, the Angular-explorer compatibility facade (/graphs*, /search, /paths, /nodes/{id}/adjacency, …), and the Cartographer build/overlay surface (/api/graphs/builds, /api/graphs/overlays). RBAC is enforced via the Graph.ReadOrQuery/Graph.Query/Graph.Export policies. (The earlier-drafted GET /graph/nodes, /graph/impact/{advisory}, and /graph/query/saved routes were never implemented — see the “Legacy” note in architecture.md §3.)
• Overlay & diff workers — materialise impact lists, saved-query caches, and signed diff manifests; feed Scheduler GraphBuildJob/GraphOverlayJob contracts (docs-archive/implplan/updates/2025-10-26-scheduler-graph-jobs.md).
• Console & CLI integrations — planned modules deliver WebGL explorer, timeline viz, and CLI stella sbom graph ... commands aligned with implementation plan phases.
• Storage abstraction — supports document + adjacency (PostgreSQL) or pluggable graph engine; both paths enforce deterministic ordering and export manifests.

Current workstreams (Q4 2025)

• GRAPH-SVC-30-00x (see src/Graph/StellaOps.Graph.Indexer/TASKS.md) — stand up Graph Indexer pipeline, identity registry, snapshot exports.
• Active sprint: docs/implplan/SPRINT_0141_0001_0001_graph_indexer.md (Runtime & Signals 140.A) — clustering/centrality jobs, incremental/backfill pipeline, determinism tests, packaging.
• GRAPH-API-30-00x — draft API planner/cost guard, streaming responses, and Authority scope integration.
• DOCS-GRAPH-24-003 & related backlog — author overview/API/query language docs; update this README again once those deliverables land.
• Deployment/DevOps follow-ups (DEVOPS-VEX-30-001, DEPLOY-VEX-30-001) coordinate dashboards, load tests, and Helm/Compose overlays for the graph stack.

Integrations & dependencies

• SBOM Service (Scanner WebService + Worker) produce sbom_snapshot events consumed by Graph Indexer.
• Concelier/Excititor contribute advisory + VEX edges; VEX Lens consensus overlays attach to graph nodes as attributes.
• Policy Engine & Scheduler trigger recompute jobs and consume overlays for risk/impact automation.
• Vulnerability Explorer & Console surface graph queries, saved views, and diff visualisations.
• Authority defines the graph scopes in the canonical catalog (StellaOpsScopes): graph:read and graph:export are enforced (GraphPolicies.ReadOrQueryScopes/ExportScopes), while graph:write, graph:simulate, and graph:admin are reserved-but-unenforced. The graph:query scope used by the Graph.Query/Graph.ReadOrQuery policies is not in the Authority catalog — it is a Graph.Api-local constant (GraphPolicies.GraphQueryScope). Client registrations and secrets are managed via existing platform patterns. See architecture.md §3.5.

Data, observability & offline

• Tables (schema graph): graph.graph_nodes, graph.graph_edges, graph.pending_snapshots, graph.cluster_assignments, graph.centrality_scores, graph.idempotency_tokens, graph.saved_views, graph.asset_registry_events, plus the legacy un-qualified graph_nodes/graph_edges/graph_snapshots/graph_analytics tables. There is no graph_overlays_cache / graph_saved_queries table — saved views live in graph.saved_views; overlays are not persisted. See architecture.md §4.
• Metrics: see the “Key metrics (new)” list below and architecture.md §6 for the authoritative names. (The legacy graph_ingest_lag_seconds / graph_nodes_total / graph_query_latency_seconds{queryId} names are not emitted.)
• Logs/traces: structured ETL logs, query planner traces, WebGL interaction telemetry (once UI lands).
• Offline bundles: deterministic nodes.jsonl, edges.jsonl, overlay manifests + DSSE signatures, consumable by Export Center and CLI mirroring.

Operations & runbook (Sprint 030)

• Dashboards: import Observability/graph-api-grafana.json (panels for latency, budget denials, overlay cache ratio, export latency). Apply tenant filter in every panel.
• Health checks: /healthz should be 200; search/query/paths/diff/export endpoints require tenant context, Authorization, and graph scopes (graph:read/query/export).
• Tenant context resolution:
  • Canonical header: X-StellaOps-TenantId (StellaOpsHttpHeaderNames.Tenant).
  • Compatibility headers: X-Stella-Tenant, X-Tenant-Id (migration-only). See GraphRequestContextResolver/GraphHeaderAuthenticationHandler.
  • The resolved tenant claim (stellaops:tenant/tid/tenant_id) wins when present; a conflicting tenant value across headers or between header and claim is rejected deterministically with 400 GRAPH_VALIDATION_FAILED.
• Scope enforcement:
  • Graph endpoints authorize against claim-based policies (Graph.ReadOrQuery, Graph.Query, Graph.Export).
  • Header scope compatibility (X-StellaOps-Scopes, X-Stella-Scopes) is bridged once at authentication and then evaluated only through policies.
• Key metrics (new):
  • graph_tile_latency_seconds histogram (label route); alert when p95 > 1.5s for 5m.
  • graph_query_budget_denied_total counter (label reason); investigate spikes (>50 in 5m).
  • graph_overlay_cache_hits_total / graph_overlay_cache_misses_total; watch miss ratio > 0.4 for 10m.
  • graph_export_latency_seconds histogram (label format); alert when p95 > 2s for ndjson/graphml.
• Triage playbook:
  • Budget denials: lower default edges/nodes budget or guide callers to request smaller scopes; verify overlay includes are truly required.
  • Overlay cache misses: ensure cache TTL is ≥5m; check overlay service connectivity to Policy Engine; warm cache by replaying recent hot nodes.
  • Export slowness: reduce export Limit, offload PNG/SVG to worker, and confirm disk I/O headroom.
  • If alerts fire, capture tenant, route, cursor/budget values, and recent deploy SHA in incident note.

Key docs & updates

• architecture.md— inputs, pipelines, APIs, storage choices, observability, offline handling.
• implementation_plan.md— phased delivery roadmap, work breakdown, risks, test strategy.
• schema.md— canonical node/edge schema and attribute dictionary (keep in sync with indexer code).
• asset-registry.md- asset registry v1 schema, persistence, replay, and downstream compliance consumers.
• api-reference.md— Graph API endpoint and DTO reference.
• API surface (spec): docs/api/graph-gateway-spec-draft.yaml (NDJSON tiles for /graph/search|query|paths|diff|export, budgets, overlays).
• Updates (archived): docs-archive/implplan/updates/2025-10-26-scheduler-graph-jobs.md, 2025-10-26-authority-graph-scopes.md, 2025-10-30-devops-governance.md for the decisions/dependencies that shaped this module.
• Index: architecture-index.md— data model, ingestion pipeline, overlays/caches, events, and API/observability pointers.

Epic alignment

• Epic 5 – SBOM Graph Explorer: Graph Indexer, Graph API, saved queries, overlays, Console/CLI experiences, Offline Kit parity.
• Cross-epic ties: Policy reasoning (explain overlays), Scheduler recompute, Notify/Task Runner integration for graph incidents.

Implementation Status

Delivery Phases

• Phase 1 – Graph Indexer foundations: Stand up Graph Indexer service, node/edge schemas, ingestion from SBOM/Concelier/Excititor events, identity stability, snapshot materialisation
• Phase 2 – Graph API service: Expose search, query, path, impact, diff, and overlay endpoints with RBAC, cost controls, streaming responses
• Phase 3 – Console & CLI experiences: Ship Graph Explorer UI (WebGL canvas, filters, diff mode, overlays) and CLI for automation pipelines
• Phase 4 – Advanced analytics: Implement clustering, centrality, saved queries, overlay caching, Policy Engine explain integration
• Phase 5 – Exports & offline: Deliver GraphML/CSV/NDJSON exports, Offline Kit bundles with deterministic manifests
• Phase 6 – Observability & hardening: Complete dashboards, alerts, runbooks, load/perf testing, a11y review

Acceptance Criteria

• Graph Indexer ingests SBOM/advisory/VEX events deterministically with tenant isolation and append-only provenance
• Graph API serves endpoints within budgeted latency and enforces cost limits + RBAC
• Console explorer visualises topology, overlays, diffs; CLI commands mirror functionality for automation
• Exports and Offline Kit bundles reproduce snapshots and overlays with signed manifests
• Observability dashboards/alerts detect ingest lag, query failures, cache churn, memory pressure; runbooks guide remediation
• Policy/VEX overlays align with Policy Engine explain traces and VEX suppressions

Key Risks & Mitigations

• Graph scale/complexity: Adopt adjacency compression, cached overlays, streaming pagination, enforced query budgets
• Tenant bleed: Strict tenant filters, fuzz tests, data masking, compliance reviews
• Runaway queries/visualization: Cost planner, query timeout, UI hints, safe mode renders
• Cache poisoning: Input validation, schema versioning, eviction policies
• Offline parity gaps: Deterministic export pipeline, integration tests for Offline Kit import

Current Active Sprint

• Runtime & Signals 140.A: Clustering/centrality jobs, incremental/backfill pipeline, determinism tests, packaging