ADR-024: Internal OCI metadata registry, opt-in upstream attachment, and attachment-state

Status: Accepted — implemented. Worker-path internal-first store + dual-write + OCI read API + attachment-state shipped (sprints 20260604_013–018); CLI/CI ingest, retention sweep, and gate internal-first-then-external sourcing shipped 2026-06-22 (sprint 20260622_001). See Implementation status below. Date: 2026-06-04 Sprint: SPRINT_20260604_014_Scanner_internal_oci_metadata_registry.md (with 013, 015–018) Related ADRs: ADR-004 (forward-only migrations), ADR-006 (EPF), ADR-009 (per-env registry override), ADR-013 (.NET callgraph extractor)

Read this before changing how SBOM/reachability artifacts are attached to images, adding a registry connector, or touching the SBOM readiness gate.

Context

Stella generates SBOMs (static analysis of image layers) and computes call-reachability, and it can attach an SBOM to a container image as an OCI referrer (SbomOciPublisher → OciArtifactPusher, subject = image digest). In practice the path is silently lossy, and the loss is invisible:

  1. The registry “connection test” is read-only. TestConnectionAsync/CheckHealthAsync probe GET /v2/ (+ token handshake) and at most GET /v2/_catalog. They never verify the credential can pull the target repo, let alone push. “Connection test passed” therefore says nothing about attach capability. (src/Integrations/__Plugins/StellaOps.Integrations.Plugin.DockerRegistry/DockerRegistryConnectorPlugin.cs)
  2. OCI publish failure is swallowed. On 401/403/network, SbomOciPublishStageExecutor logs a warning and returns; the scan still completes. (src/Scanner/StellaOps.Scanner.Worker/Processing/SbomOciPublishStageExecutor.cs)
  3. Readiness conflates “generated” with “attached.” SbomReadinessService returns ready the moment an internal ArtifactBomRow exists — it cannot distinguish “SBOM exists internally” from “SBOM attached to the registry image,” so the ReleaseOrchestrator gate passes even when nothing was attached.
  4. No retry, no alert, no UI signal, no drift detection. Integration health is one-shot at create; revoked write access is never noticed.

This is exactly how the 2026-06 customer predeploy reported green while attaching nothing: registry.example.com returned 401, the slice fell back to partner-registry, and a push (which needs write) was never validated or surfaced.

The platform is offline / air-gap-first. Customer registries are frequently pull-only mirrors; provenance must not depend on the ability to write back to them.

Decision

  1. The internal OCI metadata registry is the system of record. Every SBOM/reachability referrer is written to a Stella-owned, content-addressed store (scanner.oci_referrers + oci_blobs over the existing IArtifactObjectStore), keyed by (tenant, registry host, repo path, subject image digest). It is written first and always; metadata is never lost. Stella serves it through an OCI-Referrers-compatible read API (GET /v2/{repo}/referrers/{digest}, plus manifest/blob GET and a native by-image lookup).
  2. Upstream attachment to an external registry is per-registry opt-in and fails hard. A registry integration carries publishOciMetadata (default false). When on, Stella verifies write capability at add/test time (non-destructive POST blobs/uploads/ + DELETE) and, at publish time, any persistent push failure is a surfaced hard error (the scan stage fails) — never a silent skip. When off, Stella never pushes externally; internal-only is the normal, healthy state.
  3. Attachment is a first-class state distinct from SBOM generation: not_attempted | attached_internal | attached_external | publish_failed, propagated to the readiness probe (additive field) and optionally to the RO gate (RequireExternalAttachment, default false).
  4. Failures are durable and visible. Failed external pushes retry with backoff, dead-letter on exhaustion, drain back to attached_external when write access returns, and raise Notify events (sbom.publish.failed, registry.write-access-lost, registry.write-access-restored) plus UI chips/banners.
  5. Co-produce at build time. When Stella builds and publishes the image itself (EPF build.docker), it has the source and push rights — so SBOM and reachability are co-produced and attached as bundled referrers on the same image subject (hybrid/phased; reachability from the live build context yields the genuine non-stub graph).
  6. Signing is opt-in and local. Referrers may carry a DSSE envelope signed via Attestor FileKeyProvider (local ES256/PS256 keys, no cloud KMS); off by default so no key provisioning is forced.

Consequences

Implementation status

Worker path (sprints 20260604_013–018, shipped): internal store scanner.oci_referrers + oci_blobs over IArtifactObjectStore; LocalOciReferrerSink (always-first) + RemoteOciReferrerSink + DualWriteOciReferrerCoordinator (fail-hard external); OCI-Distribution read API OciReferrersReadEndpoints (GET /v2/{repo}/referrers|manifests|blobs/{digest}); attachment-state machine + SbomReadinessService; retry/dead-letter (OciReferrerRetryJob); per-registry publishOciMetadata opt-in + write-probe; reachability referrers + opt-in DSSE.

CLI/CI + retention + gate sourcing (sprint 20260622_001, shipped 2026-06-22): this closed the gaps where evidence produced outside the worker (the CLI in CI) could not feed the internal store and the gate could not see metadata that lived only on the deployment registry.

Still design-only: build-time co-production bundling (sprint 20260604_017) and optional SBOM-referrer DSSE signing (reuses the reachability signer). Layer/config blob GC in the retention sweep is deferred (only orphan manifest blobs are reclaimed).