SBOM version publication stream

Contract and ownership

SbomService defines two owner seams for a downstream SBOM projection without foreign database access:

  1. the ordered P6 stream sbom.versions plus a bounded live-version snapshot; and
  2. the existing tenant-scoped exact document route GET /api/v1/sbom/subject/{subjectRef}/document?version={versionId}.

src/SbomService/__Libraries/StellaOps.SbomService.Contracts freezes the v1 payload and snapshot records. It is a P19-classified producer SDK with zero project/package references and only BCL types. It intentionally does not define a second event envelope: the standard P6 envelope from StellaOps.Eventing.Reliability supplies eventId, stream, streamEpoch, seq, type, v, and occurredAt at the runtime API boundary.

The stream, snapshot, and consumer-floor routes enumerate or mutate installation-wide cross-tenant state and therefore require Sbom.Internal / sbom:operate. A tenant-scoped sbom:read principal cannot call them. The future Findings service-client grant is part of the consumer slice; this producer task does not widen an operator or tenant token.

Activation gate

The source is deliberately dormant by default. SbomService:VersionStream:Enabled defaults to false; on a disabled durable boot the host does not register the stream owner or outbox pruner, does not map any stream/snapshot/consumer route, and does not append/fence sbom.versions rows. Ordinary ledger writes remain compatible: they still allocate from the durable tenant/artifact cursor and populate exact-document identity whenever the first-writer sidecar exists, so later bootstrap can include eligible pre-stream rows. Truly sidecar-less legacy rows remain withheld. The forward-only SBOM 008 and shared Eventing migrations still converge dormant tables, and the standard reliability primitives such as IOutboxWriter remain registered against the owner database; neither fact activates an append, owner API, consumer-floor mutation, or retention worker.

Activation is fail-closed behind one complete setup-owned contract. Setting only SbomService:VersionStream:Enabled=true stops startup; the host registers no partial producer. The setup must explicitly provide every value below (there are intentionally no deployment numbers in source):

All sizes/counts/durations must be positive; the pause reserve must be below the database limit, one version-row reservation must fit the WAL-per-write limit, and the remote lease must be shorter than the retention window. The setup derives these values from its capacity/replay objectives and a representative upload/retirement rehearsal. Enabling retention alone cannot expose a feed.

Deployment keys and measuring the WAL reservation

devops/compose/docker-compose.sbomservice.yml maps every value above from the operator’s git-ignored devops/compose/.env (SBOMSERVICE_VERSION_STREAM_* and SBOMSERVICE_OUTBOX_RETENTION_*; the template rows are in devops/compose/env/stellaops.env.example). Every mapping defaults to empty or false, which a disabled boot binds to nullable options and ignores. Durations are .NET TimeSpan strings (d.hh:mm:ss); sizes are bytes.

WalBytesPerVersionRowReservation is measured, never inferred from payload bytes, because the WAL an upload produces is not proportional to the document: the sidecar text, the components_json JSONB, the ledger/audit/outbox rows, the stream fence and index maintenance all contribute, and TOAST compression varies with content. Procedure (the one used for the first activation on 2026-08-26, on PostgreSQL 18.1):

  1. Start a throwaway PostgreSQL of the same major version, provision the database/role with tools/scripts/deploy/postgres/provision-service-database.sh … --container <scratch>, and run the target image against it with SbomService:VersionStream:Enabled=true, a provisional 1-byte reservation, and SbomService:Ledger:MaxVersionsPerArtifact=2 so a prune retires rows. Keep the router bridge and integrations off so the rehearsal never joins the estate’s HELLO or Authority-client surfaces.
  2. CHECKPOINT, then around each POST /api/v1/sbom/upload and the final POST /internal/sbom/retention/prune read pg_current_wal_lsn() before and after and take pg_wal_lsn_diff(after, before). Repeat five times per document size; the first write after a checkpoint carries full-page images and is the per-size worst case.
  3. Use your largest admitted SBOM as the representative size. Measured on that date: a 441-byte body cost 6–13 KB; a 398 KB body 212–247 KB; a 1.06 MB body 2.9–3.5 MB (the worst ratio, 3.25x); a 7.05 MB body (the 10.9 MB javascript-ghost fixture, 4,417 components) 11.5–12.4 MB. Retirement cost 1.8 KB–859 KB per row across the same sizes.
  4. Set the reservation to the largest per-row figure plus margin (16 MiB was chosen for a 12.4 MB maximum), MaxWalBytesPerWrite to at least reservation × MaxVersionRowsPerWrite, and size the pause reserve so an under-reservation up to the request-body cap is absorbed before the database limit.

Re-measure after a PostgreSQL major upgrade, a change to the ledger/sidecar schema, or a change in the admitted document size. The separate literal P19 acceptance remains blocked by the pinned sbomservice|integrations build edge owned by SPRINT_20260722_024 S7/M2.

Write admission and pause

Every enabled upload and retirement calls the same gate inside its owner-database transaction, before a ledger cursor/version, stream envelope, fence, tombstone, delete, or audit write. One upload charges one version row; retirement charges the exact tenant/artifact candidates locked by that transaction. The gate refuses a write when its version-row batch exceeds the configured bound, its setup-measured per-row WAL reservation exceeds the WAL-per-write bound, or current pg_database_size(current_database()) plus the reservation reaches the configured DatabaseBytesLimit - DatabasePauseReserveBytes threshold. Refusal rolls the transaction back, increments sbom_version_stream_writer_paused_total{operation,reason}, logs the bounded reason, and returns 503 sbom_version_stream_write_paused from upload or ledger-retention endpoints.

Writer-pause recovery

Treat a pause as a capacity incident, not a retryable validation failure. Use the response reason (version_row_batch_limit, wal_batch_limit, or database_capacity) and metric/log detail to:

  1. stop automatic retries for that write path;
  2. inspect SbomService database size, outbox consumer progress/lease, and the configured outbox retention pass; then drain or provision capacity through the normal setup/change window; and
  3. re-measure the same bounded upload/retirement shape, update the explicit setup-owned limits if approved, restart to revalidate configuration, and retry the original operation.

Never bypass the gate, delete sbom.ledger_chains, prune unacknowledged envelopes, or split the upload/event and retire/tombstone transactions to recover capacity.

Events

Both events use stream sbom.versions, version 1:

EventPayload fields
sbom.uploadedtenantId, versionId, chainId, sequenceNumber, artifactRef, documentDigest, format, formatVersion
sbom.version.retiredtenantId, versionId, chainId, sequenceNumber, artifactRef, documentDigest, reason

documentDigest is always lowercase sha256:<64 hex> over the exact UTF-8 bytes returned by the version-document route. It is not the pre-existing logical/minified JSON digest stored in ledger_versions.digest. Whitespace-equivalent uploads share that logical key, so the immutable first-writer sidecar bytes — and their exact-byte digest — remain authoritative for every duplicate version, snapshot row, upload event, and retirement tombstone.

The authenticated tenant claim must be non-empty and already trimmed. It is an opaque string: a non-GUID claim such as customer/eu-west is valid and is preserved exactly. A null, empty, or untrimmed legacy tenant is never rewritten to a default/global owner.

Upload sequence allocation is serialized only for the exact (tenantId, artifactRef) chain and comes from durable sbom.ledger_chains high-water state. Retention never deletes that cursor: retiring sequence 2 or every live version cannot recycle the chain id or a sequence. The ledger upgrade/defensive convergence never adopts an empty UUID or a nonpositive sequence as cursor identity, but it advances above every observed positive sequence so malformed history cannot be reused. If no nonempty legacy chain exists, the first valid upload supplies the candidate chain and starts above that same positive high-water. An unchanged convergence pass neither rewrites the cursor tuple nor rebuilds the bootstrap index; only a strictly higher imported sequence raises the cursor while preserving its established chain id. The ledger insert, P6 append, version-row epoch/sequence fence, and create audit commit in one transaction, after the immutable raw document sidecar has been stored. If the append fails, neither the ledger row nor event is visible. Retention locks the actual candidate rows, appends ordered tombstones, deletes those rows, and records their audits in one transaction. Repeating retirement for a missing version returns zero and emits nothing. Candidate enumeration, locking, deletion, and audit are all scoped to the same exact tenant/artifact key, so two tenants sharing one artifact cannot prune one another.

The pre-existing sbom.version_events table and /internal/sbom/events surface remain the projection-read-triggered backlog. They carry no ledger/document identity and are not read or written by sbom.versions.

Ordered catch-up

GET /api/v1/sbom/version-stream/events?afterSeq={n}&limit={1..5000}&streamEpoch={optional} returns:

The response head is captured before the envelope query. Any append that interleaves after that capture is excluded from the current response even if PostgreSQL READ COMMITTED makes it visible to the query; it appears on the next catch-up request instead. Therefore every returned envelope has seq <= headSeq.

A supplied epoch mismatch, or an afterSeq whose next event is below the retained horizon, requires bootstrap. Envelopes from a mismatched epoch are not returned as if their sequences were comparable.

Bounded bootstrap snapshot

The first request is GET /api/v1/sbom/version-stream/snapshot?limit={1..1000}. It creates/locks the P6 stream-state row, captures the current epoch/head, and returns a UUID-keyset page. A continuation supplies all three values from the preceding page:

afterVersionId={nextAfterVersionId}&streamEpoch={streamEpoch}&headSeq={headSeq}

Every page uses the original fence. Versions published later in that epoch are excluded and arrive through catch-up after the recorded head. A producer epoch change returns 409 and the consumer discards the partial snapshot. withheldLegacyCount reports rows missing an exact document digest or carrying a null, empty, or non-trimmed closed-wire field (tenantId, artifactRef, format, or formatVersion). SQL uses the same .NET whitespace set as the contract, including tab, newline, non-breaking, and Unicode separator characters; those rows are withheld rather than failing the whole snapshot. Valid non-GUID tenants, including opaque non-whitespace Unicode, do appear. Rows with an empty versionId/chainId UUID or a nonpositive sequenceNumber are also counted and withheld by that same SQL predicate, so none can reach closed-contract validation and fail the page.

After importing all pages, the consumer reads catch-up strictly after the snapshot headSeq. An upload or retirement racing the page walk is therefore either reflected in the live snapshot or appears after the fence; applying the ordered tail reconstructs current live state without a gap.

Remote retention-floor registration

The producer exposes:

Reports are accepted only for the current epoch and at or below the committed head. Within an epoch, duplicate/lower reports do not move progress backwards. Once the source gate is enabled, the host declares the stream RemotelyConsumed, uses the explicitly configured producer-owned lease, and binds explicit setup-owned window/cadence settings from Eventing:OutboxRetention; the retention default remains disabled.

Exact document fetch and failure posture

For every upload/snapshot item, fetch the immutable bytes by authenticated tenantId, exact artifactRef, and exact versionId through the existing document route, then validate documentDigest. Tenant mismatch, a retired/missing version, or a legacy row without a sidecar is 404; a consumer treats that as document_unavailable, never an empty SBOM. No endpoint exposes cross-database SQL or assigns tenantless history to a consumer.