StellaOps.Vulnerabilities — module dossier

Currency. Reconciled 2026-09-11 against src/Vulnerabilities/, the completed VULN-B1 source retirement and the BIN-4 implementation. The hub has served production traffic since 2026-08-04. The old vuln, vex, vexhub, and concelier schemas are dropped, their services are stopped, and the VexHub/VexLens/Concelier source trees are retired. BIN-4 source validation does not imply activation of binary-reference derivation in an estate. Target design and the full build record live in vulnerability-data-plane-v2-design.md.

1. What this service is

The vulnerability knowledge hub: one microservice owning one database (stellaops_vuln) that holds raw + normalized vulnerability, VEX and distro knowledge for the whole platform. It replaces Concelier (incl. Feedser + Excititor) and VexHub, which shared a database with the rest of the platform and had grown to 91 GB, ~98% of which was this domain.

Two properties define it, and almost every design decision follows from one of them:

It is tenant-free. There are no tenant columns, no row-level security, and no per-tenant gate decisions anywhere in stellaops_vuln. What the world says about a vulnerability is not a per-customer fact. Everything tenant-scoped — a tenant’s own VEX decisions, source-trust overrides, gate verdicts — lives in the control plane (Policy, Findings), which is why policy.vuln_gate_current exists and vuln.issue_gate_decisions does not.

It is rebuildable from its sources. Every derived table can be dropped and reconstructed from ingest.raw_document + a re-ingest. That is not an aspiration: it is the documented disaster recovery path (T12), and it is why the v2 build was a fresh rebuild rather than a migration — the v1 corpus was already pre-flattened past the point where the original claims could be recovered.

2. Source layout

Installation posture for online ingestion

The worker reads installation posture from its own local catalog replica before each online HTTP attempt. This covers NVD, CVE, OSV, KEV, Red Hat, SUSE, Alpine, Debian, binary corpus indexes, EPSS and package payload retrieval. A declared seal strengthens the existing local egress policy; a declared unseal cannot weaken a local seal or its destination rules. Missing or unreadable replica state refuses the attempt. A witnessed absence uses local bootstrap configuration. Local mirror-file import is unaffected.

Each retry and redirect passes through the posture check after acquiring any throttle slot. The transport disables automatic redirects; the connector’s bounded GET/HEAD redirect handler rechecks each destination, refuses HTTPS downgrades and strips credentials when changing origin. Local policy refusal propagates once as AirGapEgressBlockedException; it is not a publisher HTTP response or a transient transport failure.

Enable the drain through Catalog:Replication:EnvironmentState with authenticated client-credentials mode and an explicit AuthTenant. The worker reuses its own database and requires an existing service identity granted catalog:replicate. When no Auth.Client identity is already composed, configure Vulnerabilities:Authority:Authority, ClientId, ClientSecret and an absolute TokenCacheDirectory; TokenEndpoint is optional. Provision the credential through runtime secret configuration and mount a durable private writable cache directory. There is no anonymous feed, automatic identity enrollment or implicit memory cache. A new worker with an unconverged replica refuses online ingestion, so feed authentication and convergence belong to the deployment preflight.

ProjectRole
StellaOps.Vulnerabilities.ContractsClosed contracts SDK (DC-25). Zero ProjectReferences by contract; purity is conformance-verified, not assumed.
StellaOps.Vulnerabilities.ConsensusTHE consensus evaluator (DC-38) — one pure function shared by the hub (global consensus) and Policy (tenant-effective consensus). Also a classified client SDK, for the same reason: it is pure.
StellaOps.VulnMatch.CoreDC-30 artifact-consumer SDK. The one deterministic, offline-capable SBOM-to-advisory matcher used by online and offline consumers. The assembly/namespace contract is preserved; its closed graph reaches only neutral version comparison plus the pinned Semver package and contains no database provider, HTTP, DI, host, clock, or configuration binding.
StellaOps.Vulnerabilities.RawModels / .Normalization / .CanonicalIngestion substrate carried over from the v1 tree where the transfer was purely mechanical: raw document shapes, version-range normalization, canonical domain primitives.
StellaOps.Vulnerabilities.PersistenceThe fresh schema baseline + startup migration wiring. Auto-migrates the service’s OWN database; fails closed without STELLAOPS_POSTGRES_VULN_CONNECTION.
StellaOps.Vulnerabilities.IngestionConnector SPI, the ingest pipeline, the capacity guard, and first-party issuer submission.
StellaOps.Vulnerabilities.FactsThe fact store, projectors, EPSS, mirror import/export, the corpus artifact, and the serving queries.
StellaOps.Vulnerabilities.BinaryArtifactsClosed offline SDK for native-code normalization, signature authoring/matching and predicate formats. Shared by the CLI and hub; no reference admission, persistence or issuer workflow.
StellaOps.Vulnerabilities.BinaryAnalysis / .BinariesDigest-bound reference generation, bounded package analysis, exact binary subjects and owner persistence.
StellaOps.Vulnerabilities.Fingerprints / .DeltaSigHub-owned feature generators, feature storage, evidence policy and Symbols contract adaptation.
StellaOps.Vulnerabilities.WebService / .WorkerThe two deployables (vulnerabilities-web, vulnerabilities-worker).

The family solution (StellaOps.Vulnerabilities.slnx) deliberately lists only family-owned projects. dotnet slnx auto-add pulls the whole transitive closure — trim it, or the solution stops being evidence of anything.

3. Data model — seven schemas in one database

One owner means internal schemas are fine; they are organizational, not ownership boundaries.

The identity rules worth knowing before you touch anything

A fact’s identity includes its applicability, and a revision hash is a value, not an identity. Two statements about the same (vulnerability, product) with disjoint version ranges are two distinct facts and must never collapse into one — that collapse is what flattened the v1 corpus. When a vendor republishes the same statement with a changed status, the fact keeps its identity and gains a revision; the old revision stays in history.

Provenance is many-to-one, and only the last origin’s removal tombstones a fact. The same statement arriving via a connector claim and via a derived distribution is ONE fact with TWO provenance rows. Removing one origin leaves the fact alone. Removing the last one tombstones it and rebuilds its linkset in the same transaction — there is no retention sweep in v2, and adding one would reintroduce the deletion-by-schedule behaviour the v2 design removed.

An issue linkset’s subject identity is (vulnerabilityId, productKey); its content hash is an observation revision. This is the consensus.issue_linkset primary key, with no tenant dimension. The values are ordinal and case-sensitive — especially productKey, where mixed-case Maven group IDs are ordinary — and consumers must preserve the exact producer-returned case. contentHash is SHA-256 over the canonical projection including live fact revision hashes, so a value change keeps the subject and changes its observation revision. Scanner’s worked evidence contract applies that distinction in advisory-evidence-identity.md; it never derives a Guid. SCN-ID-1 added the zero-reference producer response DTO plus OpenAPI parity. SCN-ID-3 now maps the product-issues endpoint through that closed DTO and advertises its typed 200 response. Scanner’s disabled-by-default adapter consumes only this zero-reference producer contract, preserves exact case/content hash, and emits no evidence. SCN-ID-4’s canonical v2 codec is DONE. SCN-ID-5’s authoritative producer has emitted genuine v2, but live acceptance remains blocked until its all-products-available bootstrap and forcing/rollback/soak pass; SCN-ID-6 retirement follows. Product-key path routes recover the final segment from Kestrel’s raw request target and decode HTTP transport exactly once. This preserves canonical PURL escapes such as OCI-name %2F and digest %3A; applying UnescapeDataString again to the mixed bound route value changes the producer-owned identity and is forbidden.

4. Reconciliation protocol

Every projection in this service follows the same four rules, and they are not independently negotiable:

  1. One source transaction commits the fact revision/tombstone and its ordered outbox row. Absence-tombstones are authorized only by a completed generation — a connector that died halfway through must not be able to delete what it simply failed to fetch.
  2. One leased writer per stream, with a fencing token checked INSIDE the destination transaction. A second instance blocks on the lease; an expired holder’s commit is rejected by its stale token rather than by hoping it noticed in time.
  3. One destination transaction per event: inbox admit → rebuild exactly the affected rows → advance the checkpoint. A failure rolls back all three. A checkpoint that could advance past a failed derivation would silently drop the event forever.
  4. Replay is physically idempotent. Content-guarded upserts mean a replay writes zero hot-row versions — asserted directly by summing xmin before and after a full re-consumption.

Trap, learned the hard way (2026-08-02): the producer-epoch check must run before any empty-batch early return. A rebuilt producer restarts its sequence at 1, so a stale checkpoint of (old epoch, 7) filters seq > 7 against the new epoch and legitimately returns zero envelopes. An early return there leaves the consumer permanently frozen on a stream that is producing normally. An empty batch is not evidence that your checkpoint is still valid.

5. What the hub serves

Under /api/vulnerabilities/v1/ (gateway route group added by SPRINT_20260722_006):

Canonical OpenAPI baseline: src/Api/StellaOps.Api.OpenApi/vulnerabilities/openapi.yaml. The retained-publication routes are source-complete and specified below, but their generated API catalogue synchronization is separately owned outside SPRINT 010’s cross-module allowance; release must close sprint risk F-R0821-X18-2 before advertising them through generated API artifacts.

Measured binary reference derivation

The binary-plane Doctor census expects 11 binaries tables and five symbols tables after startup migration 009_symbol_management.sql adds symbols.catalog_manifest. Missing or additional tables produce an unhealthy census with measured and expected counts; folded-plane marker checks also reject a different database that happens to use the same schema names.

Source verification: 0e5792ebea77d6fff0f58ca6ee04d204737360c3. The BIN-4 acceptance receipt records the executed suites, recheck commands and estate boundary.

The worker can compare package code with operator-admitted vulnerable/patched reference pairs. Each pair declares a reference ID, CVE, component, ABI, target symbols and exact payload URLs and digests. Native payloads remain transient. Both binaries must contain every requested symbol, share their measured architecture/format and produce at least one different normalized code signature. Relocation alone does not establish a patch discriminator.

Complete exact comparisons can emit affected or fixed. Missing symbols, unknown code, conflicting states or an unknown/mismatched configured distro ABI produce no statement. The worker records the reference digests, recipe, measured symbol hashes and binary digest in the signed evidence. It does not infer not_affected from absence. Per-package observation limits are logged and mean remaining binaries received no derivation.

Enable through Vulnerabilities:Binaries:References; the shipped example configuration has an empty Pairs array and is inactive. At most 16 pairs are admitted, with 1–256 named target symbols per pair. DistroAbis is explicit source configuration, not measured ABI. The default observation limit is 64 binaries per package. Signed analyzer admission uses the existing BinaryIndex:RuntimePlugins configuration; the worker has no compiled B2R2 implementation.

Publication configures the Signer and hub base URLs, actual worker image ProducerDigest, service AuthTenant and an absolute mounted ProofOfEntitlementFile. Existing Vulnerabilities:Authority credentials request signer:sign and vuln:submit. Signer must trust that exact digest under TrustedProducerDigests["vulnerabilities-worker"], and its configured KMS key must match the public key enrolled for the hub issuer. The worker requests signing through Signer, checks the returned predicate and binary subject, then submits the DSSE proof to the normal issuer endpoint. A signing/submission failure holds the corpus checkpoint for retry. No private key enters the worker or hub.

Exact OpenVEX subjects use urn:stellaops:binary:<scheme>:<escaped-variant>:<escaped-value>. Build-id variants are gnu-build-id, pe-cv and macho-uuid. Fingerprints use an empty variant and preserve the opaque ID’s case. These identities survive consensus and full mirror transport; version-range matcher-rows and the compact vulnerability matcher exclude them.

6. The corpus artifact

The exploit-evidence consumer contract distinguishes curated exploitation, exploit availability and probability in Findings, Policy and offline exports.

Consumers that need to match against the corpus (Policy’s gate path, the CLI’s compact export, offline estates) do not query the hub. They consume a generation-stamped, sectioned artifact: matcher-rows, consensus-inputs, exploit-evidence, reachability-sinks, and advisory-metadata. Each consumer declares the sections it uses and fetches only those, which caps per-host disk without a second artifact format.

Compact exports additionally require three optional publication sections: compact-metadata (v2 metadata rows scoped to one source record, without long prose), compact-osv-documents (v1 OSV-shaped identifiers, aliases, withdrawal status and symbol-bearing affected entries), and compact-kev-documents (v1 current raw CISA entries, including ransomware status). They use the same publication snapshot and retention limits. The five mandatory v2 descriptors remain unchanged for other readers. The additive sourceStatementId provenance property in consensus-inputs identifies an upstream record for compact license routing; existing consumers may ignore it, while the compact exporter refuses a publication that omits it. Fact identity and consensus semantics are unchanged.

6.1 Advisory metadata and generation publication wire

The producer-owned closed SDK freezes the first X18 slice in CorpusAdvisoryMetadata.cs and CorpusGenerationPublication.cs:

FND-X18-2 completed the retained producer seam in source on 2026-08-21:

The legacy in-memory v1 producer and unstamped /corpus/export route remain rollout compatibility surfaces. They still omit advisory-metadata and are not valid retained-generation/bootstrap authorities. The compose wiring was applied to the running estate on 2026-08-26 (FND-9 residual window): both hub roles run post-X18-2 images, migration 002 converged at boot, and the worker holds the RW publication volume. Two bring-up defects measured there are recorded with their fixes in vuln-ops.md: the volume root must be owned by the runtime uid, and stored fact identity strings must equal the trimmed identity VulnFact.ComputeFactId() hashes (migration 003_fact_identity_representation_convergence converges rows written before that guard).

The metadata projection is written for NEW documents only, so an estate that normalized before migration 002 publishes an all-null advisory-metadata section indefinitely (measured 2026-08-26: 0 candidates against 669,406 live vulnerabilities on a corpus whose connectors report “N seen, 0 new” per cycle). The owner-side backfill (sprint 003 VULN-B7) closes that: the worker-hosted CorpusAdvisoryMetadataBackfill walks each metadata-authority source’s retained ingest.raw_document payloads in the pump’s own (fetched_at, digest) order, at-or-before the source’s normalization checkpoint, only for documents whose digest still backs a live fact (the selector’s reachability predicate), and feeds them through the SAME normalizer and writer the pump uses — the result is a pure function of the retained document set, never of batch size or resumption. Its per-source checkpoint lives in facts.advisory_metadata_backfill (migration 004); the pass is default-off (Vulnerabilities:CorpusPublication:MetadataBackfill:Enabled), commits one batch per transaction, parks its checkpoint on the normalization head on completion, and a re-run changes nothing. Vulnerabilities known only through a non-authority source (alpine-secdb, debian-security-tracker) stay all-null by contract. Procedure: vuln-ops.md.

That pass has RUN on this estate (2026-08-26 15:08Z -> 2026-08-27 06:23:28Z; verified 08:30Z at 0388024ee9). facts.advisory_metadata_input holds 1,247,490 rows over 668,228 distinct vulnerabilities against 669,579 live vulnerabilities (99.798%), 0 of them unreachable by the selector’s live-provenance predicate, and the uncovered remainder is exactly the debian-security-tracker / alpine-secdb-only set that has no metadata authority — so coverage of the derivable population is 100%. The first publication after it carried a 979,698,214-byte advisory-metadata section where the last pre-backfill one carried 221,164,791 B of all-null rows. Re-verify with the coverage and reachability queries in vuln-ops.md step 3; the backfill is one-time and the pump is the standing mechanism from here, with FactWriter provenance now following the latest asserting document so a re-published record does not strand the selector on a stale digest.

Two bounds a consumer of that section has to respect, both measured 2026-08-27. Retention is a count, not a window: RetainedCorpusPublicationStore keeps the newest RetainedPublicationCount publications (live value 2, valid range 2-4) and prunes the rest inside the publishing transaction, with the CAS objects deleted immediately after commit. There is no age floor and no consumer-position floor. Under the pre-2026-08-27 cadence — one publication per completed source generation across 8 sources — the shortest measured artifact lifetime on this estate was 35 minutes. VULN-B8 (a) cut that to one publication per cycle, which multiplies the depth-2 grace by the number of enabled sources without changing a single retention value; (b) below is still open, so retention remains a blind count that consults no consumer. VULN-B8 © (2026-09-02) gave the pruned handle a typed signal: the prune is now a soft reclaim — the catalogue row survives as a time-bounded tombstone (reclaimed_at, migration 006; default window 14 days, Vulnerabilities:CorpusPublication:TombstoneRetention) while the bulk files stay GC-eligible (the CAS reference set excludes tombstones). The artifact routes answer HTTP 410 corpus_publication_reclaimed with reclaimedAt and the catch-up leg (GET /api/vulnerabilities/v1/corpus/publications/current) for a reclaimed handle, so it is no longer byte-identical to a handle that never existed (those still 404 corpus_publication_not_found / corpus_publication_section_not_found). GET /corpus/publications/current stays 200; the recovery path is unchanged, just now authoritatively signposted. Sharpest form of the problem: a full generation import measures 36 m 26 s while the shortest measured artifact survival under the old cadence was 35 m 25 s, so at RetainedPublicationCount = 2 no depth guaranteed a consumer could finish before its source was reclaimed — and HubEventMetrics.CorpusCatchUpLeg names publications/current as the declared recovery path for a consumer below the EVENT retention horizon, i.e. the protected plane’s recovery route points at the unprotected one. The once-per-cycle cadence makes the ordinary case comfortable; VULN-B8 (b) — owner-ruled to the HORIZON branch 2026-09-02 — completes the contract: GET /corpus/publications/current publishes an artifactRetention block (retainedCount, tombstoneRetentionSeconds, oldestRetainedSequence, retainedArtifactRefs[]). Membership of retainedArtifactRefs is the horizon test a consumer runs against the refs it holds; a held ref absent from the window answers 410 corpus_publication_reclaimed (while its tombstone lives) or 404 if it never existed. Producer deletion never depends on consumer state — deliberately, per the ruling: no lease cliff and no byte-ceiling/consumer-floor conflict can arise. The ranged-read requirement above remains the one open bound, recorded in sprint 003’s Decisions & Risks.

The publication catalog’s P13 class is cache-bounded: it holds generated corpus manifests and bounded reclaim tombstones, separate from the authoritative advisory/fact store and immutable release evidence. Forward migration 007_corpus_publication_retention_class.sql corrects the older bounded-current-previous comment without changing publication count/byte limits, tombstone windows or reclamation behavior. This source change does not imply a live migration has run.

7. Boundaries — who may reference what

src/ is the source of truth here and the rule is enforced, not documented: architecture conformance rejects every new cross-service source edge.

8. Connectors

Tiering, the removal list, and the plugin contract are in connectors-architecture.md. The short version: Tier 0 is the credential-free default set that every estate gets; Tier 1 is opt-in (including the regional feeds); a removed connector can return as a Tier-1 plugin without a hub migration, because exploited-in-the-wild and friends are already modelled as fact attributes.

distro-binary-corpus is the DC-33 addition and rides the same SPI with no special casing. It streams distro repository index documents (deb-packages/rpm-repomd/apk-index) as binary-corpus raw documents; unpacking packages and extracting build-ids/fingerprints is the hub-side analysis stage, and package bytes are transient working data that never become ingest.raw_document rows. That split is a capacity invariant, not a preference — see design §9.1. The source is opt-in per estate, ships no default repository host, and fails closed when unconfigured; bring-up (mirror-first guidance, the capacity lever, and how to read the footprint check) is in the vulnerability operations runbook.