Graph Architecture

Derived from Epic 5 – SBOM Graph Explorer.

The Graph module is the generic SBOM dependency/service graph store for the Stella Ops release control plane. It ingests SBOMs, advisories, VEX statements, and asset observations into a content-addressed node/edge model, then serves query, path, diff, lineage, overlay, and export surfaces to the Console, CLI, and SDK. This document captures the core model, ingestion pipelines, API surface, storage, and observability as implemented.

Scope note. This module (src/Graph/) owns the generic dependency/service graph store — nodes, edges, overlays, lineage, and the asset registry. It is distinct from src/ReachGraph/ (the reachability subgraph), which consumes Graph data rather than owning it. The graph:simulate scope and AI-assisted recommendations referenced below are roadmap items not yet enforced by this service.

0) Components

Shipped as a single container (stellaops-graph-api, slot 20). Cartographer (formerly slot 21) has been merged in; cartographer.stella-ops.local is now a network alias on this container. The module is composed of:

• StellaOps.Graph.Api— the ASP.NET host. Minimal-API endpoints (search/query/paths/diff/lineage/export, edge metadata, asset registry, Angular-explorer compatibility facade, Cartographer build/overlay surface), auth/tenant resolution, rate limiting, audit logging, overlay/export services, saved-view store. Entry point Program.cs.
• StellaOps.Graph.Indexer— ingestion + analytics + incremental processing pipelines (SBOM ingest, Inspector/advisory/policy/VEX transformers, Louvain-style clustering + degree/betweenness analytics, change-stream processor, asset indexing). Hosted in-process by the API.
• StellaOps.Graph.Indexer.Persistence— PostgreSQL persistence: EF Core context, document/snapshot/analytics/idempotency/asset-registry repositories, and the embedded startup migrations under Migrations/.
• StellaOps.Graph.Core— the cve_observation_nodes store (CVE observation nodes with signal state for the determinization pipeline); ships its own migrations/003_cve_observation_nodes.sql.

  NOT HOSTED TODAY (verified 2026-07-12). This library is not wired into graph-api— no production project references it (StellaOps.Graph.Api.csproj references Indexer + Indexer.Persistence, not Core; the only .csproj referencing StellaOps.Graph.Core is its own __Tests/StellaOps.Graph.Core.Tests). Its migration is not an embedded resource and no host calls AddStartupMigrations for it, so cve_observation_nodes is neither served nor auto-created by the running service. Treat it as a library awaiting a host, not a shipped store. §2.7 trap for whoever wires it: the migration must be added as an <EmbeddedResource> and registered via AddStartupMigrations first, or the table will not converge on a fresh database.

1) Core model

• Nodes (the kind string the indexer emits is shown in code):
  • artifact (application/image digest) with metadata (tenant, environment, labels).
  • sbom (sbom digest, format, version/sequence, chain id).
  • component (package/version, purl, ecosystem).
  • advisory and vex_statement nodes linking to Concelier/Excititor records via digests.
  • policy_version nodes representing signed policy packs.
  • asset nodes for NIS2/DORA/TLPT inventory. Asset v1 types are exactly host, container, image, service, integration, and plugin; credentials and people are opaque refs only.
  • file nodes (per-component source/binary files) are emitted today by SbomIngestTransformer.CreateFileNode (kind file; carries normalized_path, content_sha256, language_hint, size_bytes, scope), linked to their declaring component via DECLARED_IN edges. There is no separate Path node kind — the file path is the normalized_path attribute on the file node.
  • license nodes (kind license; SPDX id, name, classification, notice URI) are emitted today by SbomIngestTransformer.CreateLicenseNode, deduplicated per (license_spdx, source_digest).
  • Separately, StellaOps.Graph.Core owns cve_observation_nodes (content-addressed CVE observation nodes with signal state) intended for the determinization pipeline — distinct from the main node store, and not hosted by graph-api today (see the §0 caveat). The live node store is graph.graph_nodes/graph_edges only.
• Edges: directed, timestamped relationships. Emitted today: DEPENDS_ON, CONTAINS, BUILT_FROM, DECLARED_IN, AFFECTED_BY, VEX_EXEMPTS, GOVERNS_WITH, OBSERVED_RUNTIME, PROVIDES, SBOM_VERSION_OF, and SBOM_LINEAGE_* (e.g., SBOM_LINEAGE_PARENT). Each edge carries provenance (SRM hash, SBOM digest, policy run ID). Asset lifecycle edges use ASSET_VERSION and are backed by append-only graph.asset_registry_events rows for replay. Edge kind is normalised to upper-case in the canonical edge id; node kind to lower-case.
• Overlays: projections over reachability, blast radius, policy verdict, VEX status, and AOC status. The materialised overlay index tables (e.g., graph_overlay/vuln/{tenant}/{advisoryKey}) are a roadmap target — no overlay table is shipped today. When a Testing host serves overlays, three kinds are emitted inline per node: policy (policy.overlay.v1), vex (openvex.v1), and aoc (aoc.overlay.v1), each with a deterministic overlay ID (sha256(tenant|nodeId|overlayKind)) and a sampled explainTrace on the policy overlay. Outside Testing the overlay path returns 501 GRAPH_FEATURE_UNAVAILABLE until a durable overlay backend exists (see §3.1).

2) Pipelines

1. Ingestion: SBOM Service emits SBOM snapshots (sbom_snapshot events) captured by the Graph Indexer (now hosted inside graph-api; Cartographer merged). Ledger lineage references become SBOM_VERSION_OF + SBOM_LINEAGE_* edges. Advisories/VEX from Concelier/Excititor generate edge updates, policy runs attach overlay metadata. Asset source observations from SBOM scans, integration profiles, runtime services, and plugin catalogs normalize through the Graph-owned asset indexer pipeline into asset nodes, append-only asset events, and ASSET_VERSION lineage edges. Production asset source subscriptions poll normalized *.asset-source-batch.json files from Graph:AssetSourceSubscriptions:SpoolDirectory on a default and capped 30-minute interval, then process them through AssetIndexProcessor so source services do not write Graph tables directly.
2. ETL: Normalises nodes/edges into canonical IDs (GraphIdentity.ComputeNodeId/ComputeEdgeId — tenant-scoped, kind-prefixed, Base32-Crockford SHA-256 of the sorted identity tuple; gn:/ge: prefixes), deduplicates, enforces tenant partitions, and writes document-JSON adjacency rows to PostgreSQL (graph.graph_nodes/graph.graph_edges). A graph-DB backend is a roadmap option, not shipped (see §4).
3. Overlay computation: Roadmap. The intended design has batch workers build materialised views for frequently used queries (impact lists, saved queries, policy overlays) and store them as immutable blobs for Offline Kit exports. Today only inline, per-request overlays exist (Testing only — see §1 and §3.1); there is no durable overlay materialisation worker yet.
4. Diffing: graph_diff jobs compare two snapshots (e.g., pre/post deploy) and generate signed diff manifests for UI/CLI consumption.
5. Analytics (Runtime & Signals 140.A): background workers run Louvain-style clustering + degree/betweenness approximations on ingested graphs, emitting overlays per tenant/snapshot and writing cluster ids back to nodes when enabled.

3) APIs

• POST /graph/search — NDJSON node tiles with cursor paging, tenant + scope guards.
• POST /graph/query — NDJSON nodes/edges/stats/cursor with budgets (tiles/nodes/edges) and optional inline overlays (includeOverlays=true) emitting policy.overlay.v1 and openvex.v1 payloads; overlay IDs are sha256(tenant|nodeId|overlayKind); policy overlay may include a sampled explainTrace.
• POST /graph/paths — bounded BFS (depth ≤6) returning path nodes/edges/stats; honours budgets and overlays.
• POST /graph/diff — compares snapshotA vs snapshotB, streaming node/edge added/removed/changed tiles plus stats; budget enforcement mirrors /graph/query.
• POST /graph/export — returns a deterministic manifest (jobId, sha256, size, format, downloadUrl) for one of ndjson/csv/graphml/png/svg; download via GET /graph/export/{jobId} (sets X-Content-SHA256). ndjson/csv/graphml are real serialisers; png/svg are placeholders today. Only available in Testing; production returns 501 GRAPH_FEATURE_UNAVAILABLE (see §3.1).
• POST /graph/lineage — returns SBOM lineage nodes/edges anchored by artifactDigest or sbomDigest, with optional relationship filters and depth limits.
• GET /graph/assets, POST /graph/assets/query, GET /graph/assets/{assetId}, and POST /graph/assets/export expose the EU Asset Registry v1 runtime surface. Responses use deterministic ordering, preserve credential/person references as opaque refs, and never expand secret material or human PII.
• Runtime repository selection is config-driven at service resolution time: when Postgres:Graph is configured, the live /graph/query, /graph/diff, /graph/assets*, and shipped /graphs* compatibility surfaces materialize persisted rows from graph.graph_nodes, graph.graph_edges, and graph.pending_snapshots; when it is not configured, only the Testing environment may use the empty in-memory repository harness.
• Naming caveat: the search/query/paths/diff/lineage service classes are named InMemoryGraph*Service but are the production implementations — they read through IGraphRuntimeRepository, which resolves to PostgresGraphRuntimeRepository in production. Only overlays, export, and edge metadata have dedicated Unsupported* (501) production variants; search/query/paths/diff/lineage serve real Postgres data.
• The production Graph API PostgreSQL repository is fail-fast: DI construction must throw on invalid/missing PostgreSQL options instead of resolving null and silently falling back to in-memory behavior.
• Compatibility facade for the shipped Angular explorer:
  • GET /graphs, GET /graphs/{graphId}, GET /graphs/{graphId}/tiles
  • GET /search, GET /paths
  • GET /graphs/{graphId}/export, GET /assets/{assetId}/snapshot, GET /nodes/{nodeId}/adjacency
  • GET/POST/DELETE /graphs/{graphId}/saved-views
  • The compatibility tile surface emits only policy, vex, and aoc overlays on the shipped web path.
  • Saved views are persisted in PostgreSQL table graph.saved_views when Postgres:Graph is configured; the API falls back to an in-memory store only in the Testing environment.
• Cartographer-compatible build/overlay surface (consumed by the Scheduler Worker’s HttpCartographerBuildClient/HttpCartographerOverlayClient; see Endpoints/CartographerEndpoints.cs):
  • POST /api/graphs/builds, GET /api/graphs/builds/{jobId} — graph build jobs keyed by a deterministic job id derived from the Idempotency-Key header (or, absent it, the canonical request body) so retries collapse. Audited as graph_build.
  • POST /api/graphs/overlays, GET /api/graphs/overlays/{jobId} — overlay jobs with the same deterministic-id contract. Audited as graph_overlay.
  • These endpoints currently track jobs in an in-memory dictionary (synchronous completed responses); persistent job storage lands when the full pipeline is wired. They are not tenant-resolved through GraphRequestContextResolver (tenant arrives in the request body).
• Edge Metadata API (added 2025-01):
  • POST /graph/edges/metadata — batch query for edge explanations; request contains EdgeIds[], response includes EdgeTileWithMetadata[] with full provenance.
  • GET /graph/edges/{edgeId}/metadata — single edge metadata with explanation, via, provenance, and evidence references.
  • GET /graph/edges/path/{sourceNodeId}/{targetNodeId} — returns all edges on the shortest path between two nodes, each with metadata.
• GET /graph/edges/by-reason/{reason} — query edges by EdgeReason enum (e.g., SbomDependency, AdvisoryAffects, VexStatement, RuntimeTrace).
• GET /graph/edges/by-evidence?evidenceType=&evidenceRef= — query edges by evidence reference.
• Legacy: GET /graph/nodes/{id}, POST /graph/query/saved, GET /graph/impact/{advisoryKey}, POST /graph/overlay/policy remain in spec but should align to the NDJSON surfaces above as they are brought forward.

3.1) Current runtime posture (Sprint 20260416_003)

• Non-testing Graph hosts require Postgres:Graph; startup fails fast when the canonical Graph PostgreSQL connection string is absent.
• Testing is the only supported environment for the empty in-memory Graph runtime harness.
• Graph Indexer change-stream no-op sources are limited to Development and Testing; production hosts must register real IGraphChangeEventSource and IGraphBackfillSource implementations or startup fails closed.
• Production change-stream writes use the IAtomicGraphChangeWriter path: PostgresGraphDocumentWriter claims the graph.idempotency_tokens.sequence_token and upserts graph_nodes/graph_edges inside the same PostgreSQL transaction, so concurrent/replayed processors cannot apply the same sequence token twice.
• Non-testing POST /graph/query and POST /graph/paths no longer fabricate overlays. Requests that ask for overlays return truthful 501 GRAPH_FEATURE_UNAVAILABLE responses until a durable overlay backend exists.
• Non-testing POST /graph/export and GET /graph/export/{jobId} now return truthful 501 GRAPH_FEATURE_UNAVAILABLE responses instead of synthesizing in-memory export jobs.
• Non-testing edge-metadata surfaces (/graph/edges/metadata, /graph/edges/{edgeId}/metadata, /graph/edges/path/..., /graph/edges/by-reason/..., /graph/edges/by-evidence) now return truthful 501 GRAPH_FEATURE_UNAVAILABLE responses until a durable backend exists.
• The shipped /graphs/{graphId}/tiles compatibility surface no longer emits fabricated overlays outside Testing.
• Live audit logging is log-only; request history is no longer retained in-process as pretend durable state.

3.2) Tenant and auth resolution contract (Sprint 20260222.058)

• Graph uses a single tenant resolver path (GraphRequestContextResolver) across search/query/paths/diff/lineage/export and edge-metadata endpoints.
• The tenant comes from the validated claim only — there is no header fallback (CANON-1 Phase 2; Security/GraphRequestContextResolver.cs:36-68). Corrected 2026-07-12: this section previously read as a claim-then-header precedence list, which invited an operator to believe X-StellaOps-TenantId selects the tenant. It does not, and never should.
  • Source (the only one): the claim stellaops:tenant (with bounded aliases tid, tenant_id). Absent ⇒ tenant_missing; a raw header is never promoted to the tenant (it is forgeable on a bypass network).
  • Headers are a defence-in-depth conflict check, not a source. If any of the three recognised tenant headers — X-StellaOps-TenantId (canonical), X-Stella-Tenant, X-Tenant-Id — is present and disagrees with the claim, the request is rejected tenant_conflict rather than silently ignored, so cross-tenant probing is observable.
• Deterministic failures:
  • missing tenant: 400 GRAPH_VALIDATION_FAILED
  • conflicting tenant claim/header values: 400 GRAPH_VALIDATION_FAILED
  • missing auth: 401 GRAPH_UNAUTHORIZED
  • missing scope: 403 GRAPH_FORBIDDEN
• Scope checks are policy-driven (Graph.ReadOrQuery, Graph.Query, Graph.Export) and no endpoint directly trusts raw scope headers.
• Rate limiting and audit logging use the resolved tenant context; authenticated flows no longer collapse to ambiguous "unknown" tenant keys.

3.3) Edge Metadata Contracts

The edge metadata system provides explainability for graph relationships:

• EdgeReason enum: Unknown, SbomDependency, StaticSymbol, RuntimeTrace, PackageManifest, Lockfile, BuildArtifact, ImageLayer, AdvisoryAffects, VexStatement, PolicyOverlay, AttestationRef, OperatorAnnotation, TransitiveInference, Provenance.
• EdgeVia record: Describes how the edge was discovered (method, version, timestamp, confidence in basis points, evidence reference).
• EdgeExplanationPayload record: Full explanation including reason, via, human-readable summary, evidence list, provenance reference, and tags.
• EdgeProvenanceRef record: Source system, collection timestamp, SBOM digest, scan digest, attestation ID, event offset.
• EdgeTileWithMetadata record: Extends EdgeTile with Explanation property containing the full metadata.

3.4) Localization runtime contract (Sprint 20260224_002)

• Graph API now initializes localization via AddStellaOpsLocalization(...), AddTranslationBundle(...), AddRemoteTranslationBundles(), UseStellaOpsLocalization(), and LoadTranslationsAsync().
• Locale resolution order for API messages is deterministic: X-Locale header -> Accept-Language header -> default locale (en-US).
• Translation layering is deterministic: shared embedded common bundle -> Graph embedded bundle (Translations/*.graph.json) -> Platform runtime override bundle.
• Remote Platform override fetches are bounded and loaded concurrently per provider locale so scratch bootstrap cannot hold the Graph API offline while optional translation overrides load.
• This rollout localizes selected error paths (for example, edge/export not found, invalid reason, and tenant/auth validation text) for en-US and de-DE.

3.5) Authorization scopes

The three authorization policies (Graph.ReadOrQuery, Graph.Query, Graph.Export) map to OAuth scopes via Security/GraphPolicies.cs; a request is admitted if it carries any scope in the set:

• Scope constants graph:read and graph:export come from StellaOps.Auth.Abstractions/StellaOpsScopes.cs (GraphRead, GraphExport). graph:query is a Graph-local literal (GraphPolicies.GraphQueryScope) and is not (yet) a named constant in the shared scope catalog.
• The shared scope catalog also defines graph:write, graph:simulate, and graph:admin, but the Graph API does not enforce them today — they are reserved for the Cartographer build surface and future what-if/admin operations.

4) Storage considerations

• Shipped backend: PostgreSQL only. There is no graph-DB (Neo4j/Cosmos Gremlin) implementation — the relational store is canonical. The IGraphRuntimeRepository abstraction (Services/IGraphRuntimeRepository.cs) exists so a graph-DB backend could be added later, but only the Postgres and (Testing-only) in-memory implementations exist today.
• Persistence is split across two migration sets, both applied via AddStartupMigrations on startup (embedded SQL resources):
  • StellaOps.Graph.Indexer.Persistence/Migrations/ (schema graph): graph.graph_nodes, graph.graph_edges (document-JSON adjacency rows written by PostgresGraphDocumentWriter), graph.pending_snapshots (the canonical change feed read by PostgresGraphSnapshotProvider), graph.cluster_assignments and graph.centrality_scores (analytics), graph.idempotency_tokens (sequence-token claim table used in the same transaction as document writes), plus the legacy un-qualified graph_nodes/graph_edges/graph_snapshots/graph_analytics/graph_idempotency tables from migration 001. There is no graph_overlays table — overlays are computed/served inline, not persisted in a dedicated overlay table.
  • StellaOps.Graph.Api/Migrations/003_saved_views.sql (the saved_views table) and the graph.asset_registry_events table (see below).
• Tenant partitioning is enforced by a tenant_id/tenant column on every table (not row-level security). The one exception is the cve_observation_nodes table in StellaOps.Graph.Core, which enables RLS via app.tenant_id — but that table is not created or served by the running graph-api(see the §0 caveat), so RLS is not part of the shipped tenancy story today. Deterministic ordering and streaming NDJSON exports back the Offline Kit story.
• The shipped compatibility saved-view surface now owns a small relational persistence slice via startup migration 003_saved_views.sql, which auto-creates graph.saved_views and keeps saved views durable across host restarts.
• Asset Registry v1 owns graph.asset_registry_events via Graph Indexer Persistence startup migrations. Events are append-only, deletions are tombstones, and each event carries payload, Merkle leaf, and event hashes for downstream Findings Ledger projection. Findings Ledger mirrors those rows read-only into findings.asset_registry_events; Graph remains the event source of truth.

5) Offline & export

• Each snapshot packages nodes.jsonl, edges.jsonl, overlays/ plus manifest with hash, counts, and provenance. Export Center consumes these artefacts for graph-specific bundles.
• Saved queries and overlays include deterministic IDs so Offline Kit consumers can import and replay results.
• Runtime hosts register the SBOM ingest pipeline via services.AddSbomIngestPipeline(...). Snapshot exports default to ./artifacts/graph-snapshots but can be redirected with STELLAOPS_GRAPH_SNAPSHOT_DIR or the SbomIngestOptions.SnapshotRootDirectory callback.
• Runtime hosts register the production asset indexing bridge through services.AddGraphIndexerPersistence(...), which binds Graph:AssetSourceSubscriptions:*, registers the file-spool source subscription bridge, and exposes the canonical IAssetSourceBatchPublisher for Scanner, Integrations, Signals, and plugin host emitters. Tests and custom hosts can still register the Graph-only pipeline directly via services.AddAssetIndexerPipeline(...).
• Analytics overlays are exported as NDJSON (overlays/clusters.ndjson, overlays/centrality.ndjson) ordered by node id; overlays/manifest.json mirrors snapshot id and counts for offline parity.

6) Observability

• Graph API metrics (meter StellaOps.Graph.Api, see Services/GraphMetrics.cs): graph_query_budget_denied_total (budget enforcement), graph_tile_latency_seconds (query/tile latency histogram), graph_overlay_cache_hits_total / graph_overlay_cache_misses_total (overlay cache), and graph_export_latency_seconds (export latency histogram). (An earlier draft listed graph_ingest_lag_seconds, per-saved-query latency, and overlay-generation duration; those names are not emitted — they remain roadmap targets.)
• Analytics metrics (meter StellaOps.Graph.Indexer, see Analytics/GraphAnalyticsMetrics.cs): graph_analytics_runs_total, graph_analytics_failures_total, graph_analytics_duration_seconds, graph_analytics_clusters_total, graph_analytics_centrality_total.
• Change-stream/backfill metrics (same StellaOps.Graph.Indexer meter, see Incremental/GraphBackfillMetrics.cs): graph_changes_total, graph_backfill_total, graph_change_failures_total, and the graph_change_lag_seconds histogram. All carry tenant, backfill, and success tags.
• Logs: structured per-request audit events (LoggerAuditLogger: tenant, route, method, actor, status, duration, scopes) plus ETL/ingestion stage logs. Audit logging is log-only — request history is not retained in-process (see §3.1).
• Traces: ETL pipeline spans, query engine spans.

7) Rollout notes

• Phase 1: ingest SBOM + advisories, deliver impact queries.
• Phase 2: add VEX overlays, policy overlays, diff tooling.
• Phase 3: expose runtime/Zastava edges and AI-assisted recommendations (future).

Local testing note

Set STELLAOPS_TEST_POSTGRES_CONNECTION to a reachable PostgreSQL instance before running tests/Graph/StellaOps.Graph.Indexer.Tests. The test harness falls back to Host=127.0.0.1;Port=5432;Database=stellaops_test, then Testcontainers for PostgreSQL, but the CI workflow requires the environment variable to be present to ensure upsert coverage runs against a managed database. Use STELLAOPS_GRAPH_SNAPSHOT_DIR (or the AddSbomIngestPipeline options callback) to control where graph snapshot artefacts land during local runs.

Refer to the module README and implementation plan for immediate context, and update this document once component boundaries and data flows are finalised.