Vulnerability mirror seeds

This page accompanies VULN-B1’s source cutover. Deployment and final acceptance are recorded in the consolidation sprint; source implementation alone is not a rollout receipt.

Mirror delivery, custody and storage descriptions are source-verified against 727dcc45fb9faa361fb6480f082855fcd5fe3b58 (2026-10-23). The source anchors and re-verification commands identify the reviewed paths. The earlier SIF-2 source receipt qualifies the first current-record and advisory-origin slice; it is historical progression. The 2026-10-23 initial paired full-seed receipt records full semantic equivalence and an initial candidate M9 of 4.817880213750253; all 80 gates passed. The separate complete delta and post-delta receipt records replacement, replay, custody and a final whole-database M9 of 4.817880213750253 under the same five-times limit. Fresh-machine release installation and ordinary configured OSV refresh were not exercised by this isolated run; their acceptance remains pending. Source tests and this isolated proof do not establish shared-estate deployment.

Seed-version delivery bindings are implemented at 7779d371c37b2dc067410091e5639624918b827f. Re-verify MirrorRoundTripTests with tools/scripts/test-targeted-xunit.ps1 and -BuildProjectReferences; the completion lane also checks the adjacent raw-retention cases. The accepted initial pair and complete delta/post-delta receipts establish this isolated storage measurement; source tests and that proof remain separate from deployment.

The CLI builds a seed from one retained corpus publication. It resolves the publication handle first, downloads its consensus-input and advisory-metadata sections with pinned byte ranges, and verifies their lengths, strong ETags and SHA-256 digests. A rotating current pointer cannot mix two publications in one seed.

Export accepts the contract’s supported consensus-inputs window (CorpusFormatVersion.IsSupported: the current version and the one before it, 3 and 4 since the NVD endpoint fix raised the current version to 4) and advisory-metadata version 2 only. Unsupported section versions are refused before projecting records.

stella mirror seed export --output-directory ./seed
stella mirror seed validate --seed-directory ./seed
stella mirror seed import --seed-directory ./seed

The configured CLI profile supplies the hub URL and credentials. Export requests the token scopes advisory:read vex:read aoc:verify; Authority refuses advisory or VEX read scopes without aoc:verify. Import requests advisory:ingest vex:ingest. Advisory and VEX scopes also require a tenant on the token request. The CLI sends STELLAOPS_AUTHORITY_TENANT (StellaOps:Authority:Tenant) when set, otherwise the global --tenant, STELLAOPS_TENANT or the active profile tenant. A configured operator login (STELLAOPS_AUTHORITY_USERNAME and STELLAOPS_AUTHORITY_PASSWORD) uses the password grant; without one the CLI uses client credentials. From outside the estate network, set STELLAOPS_AUTHORITY_TOKEN_ENDPOINT to the gateway’s /connect/token; token requests then skip discovery.

export STELLAOPS_AUTHORITY_TOKEN_ENDPOINT=https://stella-ops.example/connect/token
export STELLAOPS_AUTHORITY_TENANT=default
stella mirror seed export --output-directory ./seed

Local validation requires neither a database nor a network connection. The hub’s mirror data is tenant-free, so seed commands take no database connection and no seed-level tenant option; the tenant above only scopes the access token. Each seed command exits 0 on success. A seed that fails a check, such as a digest, length or advisory-count mismatch, exits 2 and prints the reason on standard error. An import the hub refuses also exits 2 and names the hub’s reason, for example HTTP 409 mirror_seed_baseline_mismatch. HTTP 409 mirror_seed_ingest_in_progress means another operation holds the mirror source, such as the worker publishing the previous import or retaining raw documents after it. The refusal is transient: rerun the same command after that publication completes.

Each seed directory contains manifest.json, advisories.jsonl and advisory-index.jsonl. The manifest (formatVersion stellaops-mirror-seed-v3) identifies the source publication and generation, scope, body digest/length/advisory count, and the digest of the complete advisory state index. Hashes use the existing Stella Ops cryptography library. The seed records mirrored claims; it does not assert that the upstream issuer signed the copied document. The body contains projected advisory claims and selected metadata, not the producer’s upstream raw documents or historical fact revisions. Seeds with an older formatVersion are refused, including as delta baselines; export a new full seed.

Installation, refresh and recovery

When a release supplies a full seed, the installer bootstraps advisory data through the owner import route. The body is included in the bundle when it is at most 256 MiB. Larger bodies are published separately as mirror-seed.v1.ndjson; keep that file beside the bundle archive before extraction. The installer verifies the body against the shipped SHA-256 before importing it. A missing separate body records awaiting-download; the installer records imported only after a completed HTTP 200 response. The installer sends the bindings that stella mirror seed import sends for a full seed: the body digest, and the state-index and scope digests from the bundle’s mirror-seed.manifest.json. The hub records the scope with the mirror state, so a later stella mirror seed import --delta-apply matches the installed baseline. When that manifest exists but its scope cannot be read, the installer records failed and does not import. A hub installed by an earlier installer, which sent no scope digest, refuses every later CLI import of the scoped seed and its deltas with 409 mirror_seed_baseline_mismatch. Reapply the full seed with stella mirror seed import --replace-existing before applying a delta to it. See the release bundle instructions.

Connected refresh is a separate Worker path. It requires enabled ingestion scheduling, an enabled osv source and configured Vulnerabilities:Ingestion:Sources:Osv:EcosystemEndpoints, under the installation’s connectivity posture. An unconfigured OSV source fails rather than completing an empty generation. Importing the seed does not establish that an online refresh job ran.

For an air-gapped installation, transfer and verify the full seed and then apply only matching deltas under the scope and state checks below. This restores current mirrored data; it does not restore the producer’s historical database. Historical database recovery requires its retained backup and the owner database recovery procedure. This source review does not establish a completed fresh installation or scheduled OSV refresh.

Record format

advisories.jsonl is UTF-8 JSON Lines sorted by advisoryKey. Each line is one advisory record of at most 16 Mi characters. An advisory whose claims do not fit one record is split across consecutive records with the same advisoryKey:

• every record except the last carries "continued": true;
• every record repeats the advisory metadata and carries a non-empty affectedPackages subset;
• the claims of all records, in order, are the advisory’s complete claim set;
• every record carries its own origins, whose claim counts cover that record’s affectedPackages, and repeats the prose origins (see “Record origins and the OSV upstream rule”); the manifest’s upstreams counts the advisory once.

Readers refuse a continued record that is not followed by the same advisory, a repeated advisoryKey without the marker, a marker whose value is not true, an empty split record, and a seed that ends inside a continued advisory. The advisory count in the manifest and the import receipt counts advisories, not records. The state index holds one entry per advisory; its digest is SHA-256 over the advisory’s record lines joined by a newline, so an unsplit advisory’s digest is the digest of its single line. A record is hashed exactly as written: its UTF-8 JSON text without the line break, never re-serialised, so escapes and literal characters such as +, < and non-ASCII keep their written spelling. A delta replays every record of a changed advisory as written, with ,"replaceAdvisory":true in place of its closing brace; stella mirror seed validate removes exactly that suffix before hashing, so a delta record hashes like the full record it replaces, and refuses a replacement record spelled any other way.

The exporter projects a full publication without a per-advisory size limit. If a single claim cannot fit one record with its advisory metadata, the export fails and names the advisory, the claim’s source and fact, and the record size. --keep-staging keeps the staging directory (next to the output directory) with the downloaded publication inputs when an export fails; without it the staging directory is removed.

Scope and deltas

stella mirror seed export --source osv --ecosystem npm --output-directory ./npm-seed
stella mirror seed export --delta-baseline ./seed/manifest.json --output-directory ./delta
stella mirror seed import --seed-directory ./delta --delta-apply

Source and ecosystem selectors accept repeated or comma-separated values. A source selector matches the publication’s issuer/source key; an ecosystem matches the package type, such as npm, deb or rpm. Metadata aggregated from excluded sources is omitted and counted in the manifest rather than silently redistributed under a narrower scope.

A delta retains its baseline’s scope. Its full state index lets the next delta use it as a baseline without loading a corpus-sized map into memory. The delta body carries added, changed and removed advisory records in identity order. The importer verifies the base state-index digest and scope before any generation or raw-document write. Delta baselines identify the complete resulting state, independently of the delta body’s digest; two empty deltas cannot be confused merely because their bodies have identical bytes. A matching replay is safe.

Installing a different full seed over an existing mirror dataset requires explicit --replace-existing. Replacement removes only obsolete mirror claims; independently ingested provider claims remain. --overwrite on export or copy applies only to a seed directory’s three owned files and refuses unrelated content.

Record origins and the OSV upstream rule

Seed delivery adds a fact-provenance origin of the form seed:sha256:<advisory-index-state-digest>/<advisoryKey>. The state digest identifies the complete resulting advisory index, including for a delta; it is distinct from the delivered body’s digest. The CLI verifies the manifest, index and body before import. The owner validates the body and records the authenticated request’s declared state digest; the index itself is not uploaded to the owner endpoint.

A delivered advisory’s facts use that delivery version. An advisory omitted from a delta keeps its existing origin. Delivering identical record bytes in a different full seed updates the record’s seed binding and replaces the former delivery origin, without changing its raw digest, fact identity or revision value. Maintenance uses the current binding. An independently held older raw record remains available as custody and cannot restore an older delivery’s claims or provenance through maintenance.

The owner stores each exact (origin_kind, origin_ref) once in facts.provenance_origin. facts.fact_provenance retains each fact association and its document digest, verification, signing fingerprint and recorded time. The ordinary facts.fact_provenance_read view reconstructs the original logical fields; evidence APIs return the complete literal reference. The internal numeric key changes neither the origin string nor fact identity. See provenance storage.

Every seed record carries an origins array. Each entry names one upstream the record’s claims or prose came from:

Entries are ordered by sourceKey, then upstreamDb. Their claims add up to the record’s package count. The importer and stella mirror seed validate refuse a record whose origins are malformed or miscounted, and accept records without the field, which seeds exported before 2026-09-26 lack. The hub keeps each imported record, origins included, as its retained mirror document.

The hub stores every OSV record under the one key osv, whatever database OSV aggregated it from. The exporter resolves the upstream of each osv claim from the OSV record id it was normalized from (the OSV schema’s database prefix) and the record’s package:

Prose origins resolve the same way from the metadata row’s source record id. The id prefix decides because it identifies the publishing database. An npm record published by another database, such as MAL-, is therefore unknown, not ghsa. The counsel ruling also maps the RubyGems and Packagist ecosystems. A record in those ecosystems with a GHSA- id resolves to ghsa; one with another id stays unknown until its database prefix is recorded. The terms and attributions of each admissible upstream are in third-party-licenses/DATA-NOTICES.md, section “OSV (Open Source Vulnerabilities)”.

--osv-upstream applies an allowlist. The exporter then drops every osv claim whose upstream is not listed. An advisory whose prose cites an excluded or facts-only osv upstream ships facts only, and the manifest counts it under omittedMetadataAdvisories. The allowlist accepts only the admissible upstreams in the table. A distro upstream also needs its own key in --source.

stella mirror seed export \
  --source alpine-secdb,cve,debian-security-tracker,kev,nvd,osv,redhat-csaf \
  --osv-upstream alpine-secdb,debian-security-tracker,ghsa,govuln,pypa,rustsec \
  --output-directory ./release-seed

The manifest records the allowlist as scope.osvUpstreams, so a delta must keep it. The manifest’s upstreams block states what the seed contains:

"upstreams": {
  "rule": "osv-upstream/v1",
  "osvRuleApplied": true,
  "admitted": [
    { "sourceKey": "osv", "upstreamDb": "pypa", "licenseId": "CC-BY-4.0",
      "claims": 1, "metadataRows": 1, "advisories": 1 }
  ],
  "excluded": [
    { "sourceKey": "osv", "upstreamDb": "ubuntu", "licenseId": "CC-BY-SA-4.0",
      "claims": 2, "metadataRows": 1, "advisories": 2 }
  ]
}

admitted counts the claims, prose rows and advisories of each upstream the seed carries. excluded counts what the allowlist dropped. metadataRows there counts advisories whose prose was omitted because of that upstream. The counts cover the complete projected state, including for a delta. Without --osv-upstream, osvRuleApplied is false and nothing is dropped. Local validation refuses a manifest whose flag, allowlist and counts disagree.

Import and publication

Configure Vulnerabilities:MirrorSeed:StagingRoot as storage writable by the hub web role. Vulnerabilities:MirrorSeed:MaxBundleBytes defaults to 32 GiB. Individual advisory records are bounded to 16 Mi characters; a larger advisory arrives as continued records. Invalid UTF-8, malformed records or continuations, digest mismatches, out-of-order identities and oversized records are refused. Temporary uploads are removed after the operation.

The owner validates the complete upload before writing, then uses the ordinary ingestion and normalization paths under a source fence. Each current record is one raw document. A split advisory’s claim-set replacement is applied once, at its last record, with the claims of all of its records. Reusing an older raw-document digest does not suppress an A-to-B-to-A value restoration. Fact writes remain content-guarded.

The owner first drains previously pending mirror work through its existing checkpoint path. It then applies each currently delivered record in bundle order, inside the same transaction as raw admission, facts, provenance, metadata, revisions, events and generation accounting. Observation timestamps remain exact; a backward or equal clock cannot reorder first application. The normalization checkpoint advances only when its retained timestamp/digest tuple increases. An interrupted run can retain a committed normalized prefix, but cannot authorize completed replacement or cleanup. Existing pending or emitted history is not reordered.

Replacement and full-seed absence use the advisory origin key, including when a GHSA advisory’s facts resolve to a CVE alias. Removing one origin keeps a fact that another origin still supports. A tombstone event names the fact’s resolved vulnerability, so downstream projection removes the correct claim.

After applying the complete seed and its cursor, the owner prunes superseded mirror records for each delivered advisory. Cleanup preserves every delivered continuation record. The existing retention store rechecks normalization, supersession and every declared custody reference under its custody lock; a referenced older document remains. Obsolete mirror metadata that no live exact-digest fact backs no longer pins a withdrawn record. Provider records and their retention policy are unchanged. Each deletion batch releases its custody lock before the next batch. Cleanup does not shorten the event or revision windows.

The current delivery binding is the raw record’s mirrorSeedStateDigest metadata. Re-delivering identical bytes in another full state updates that binding without changing the raw digest. Normalization uses the binding for the fact origin; metadata candidates still require exact-digest, live fact backing. Maintenance of an older held delivery cannot restore its claims, origin or metadata over the current binding. An explicit new delivery can restore A-to-B-to-A values.

The worker services a durable publication request using the existing corpus publisher, including when online ingestion is disabled or the estate is sealed. The web role does not write the publication root. Repeating an identical seed does not repeatedly publish the same corpus. The import receipt describes data application; publication is a separate worker completion boundary.

The owner endpoint is POST /api/vulnerabilities/v1/mirror/seeds/import, with content type application/vnd.stellaops.mirror-seed+ndjson, required X-StellaOps-Content-Digest and X-StellaOps-Mirror-State-Digest, optional delta base/scope headers, and the validateOnly and replaceExisting query controls. A baseline mismatch or active source operation returns 409; a busy custody writer during replacement cleanup also returns 409. Cleanup follows the complete seed/cursor application, so resolve contention and reapply the same seed with its matching baseline to finish removing obsolete records. Malformed input returns 400; another content type returns 415; an unconfigured staging root returns 503. While the hub’s capacity writer gate is closed (its newest capacity receipt shows stellaops_vuln at its stop line, or is stale or missing), an import returns 503 problem details titled capacity_writer_paused, with the gate’s reason as detail and Retry-After: 900, after the header and argument checks (malformed input still returns 400) and before the upload is staged; nothing is written. Once a newer receipt is below the line the same request is admitted, but a 409 (baseline mismatch or active source operation) can still answer it, because those checks need the source fence and the stored cursor. validateOnly=true requests are not refused. Export requires a current publication with the independent fixedVersion field; an older publication is refused rather than silently losing that value.

Storage and recovery boundary

A current seed restores current mirrored claims. It does not restore the producer’s historical fact revisions, replace a database backup or guarantee reproduction of an arbitrary past corpus generation. The owner-approved customer replay substrate is the digest-verified seed file set kept by the operator.

The current implementation writes each mirror record as an owner raw document, then derives current facts, provenance, local revisions, consensus linksets and events (distribution statements are computed on read since sprint 063 HSR-1; the VRD-4 baseline source predates that retirement). The worker publishes the local corpus once the imported source generation is complete and its quiet period has passed (MirrorSeedPublicationService); it does not wait for the consensus projection. Transfer size is therefore not the database size. Customer sizing requires per-table measurements after a from-empty import and completed projections.

Sprint 061’s VRD-4 measured this path from empty on 2026-10-08. A 4,905,152,865-byte seed produced 44,908,713,663 bytes in the Vulnerabilities owner database, a ratio of 9.16 (per-table receipt). The largest layers were:

• event delivery tables: 15.1 GB, the post-import peak;
• current facts, revisions, provenance and metadata: 17.8 GB;
• stored distribution statements: 7.3 GB;
• consensus linksets: 2.7 GB;
• raw seed records: 1.9 GB.

The measured source predates HSR-1, so the figure includes the distribution table that current source no longer stores. It is a dated baseline, not a current-source measurement. SPRINT_20261007_065_Vulnerabilities_customer_seed_footprint.md SIF-1 consumes it and sets the target. The selected SIF-2 storage option keeps the operator’s verified seed files as replay input and retains current mirror records under the existing custody rules. Import cleanup protects the delivered continuation record set; independently referenced older records remain. Local fact revisions retain their existing policy. This source change introduces no new revision window. Sprint 063 HSR-1 has retired the stored distribution layer; the historical measurement does not measure these source changes.

The original measurement seed is no longer available, and its source publication’s required data sections have been reclaimed. The owner approved a replacement acceptance input on 2026-10-20: a newly pinned full seed with the same seven sources and OSV allowlist, measured with both the unchanged and modified importers. The pinned replacement body is 5,175,715,788 bytes; its whole-database ceiling is 25,878,578,940 bytes (M9 at most 5.0). The historical 4,905,152,865-byte seed and its 24,525,764,325-byte target remain SIF-1 evidence. The new input requires full semantic coverage and paired per-table measurement; a smaller sample is insufficient.

The 2026-10-23 initial comparison measured 24,935,978,687 candidate database bytes against 42,498,266,815 control bytes on that same complete input. The initial receipt binds all 80 gates, per-table accounting and complete canonical/event evidence. The complete delta/post-delta proof also passed with 24,935,978,687 database bytes, 942,600,253 bytes below that same maximum. It preserves the original event prefixes and full coverage.

The raw-document lifecycle retains latest and referenced records, including split-record references. Normalized, superseded, unreferenced mirror records are eligible for the same bounded sweep as provider inputs. The owner seed import additionally reclaims obsolete records for its delivered advisories after the complete data/cursor application. The delivered continuation keep-set is protected during that cleanup; the general raw sweep still applies its normalized, superseded, unreferenced predicate. Applying deltas does not require retaining every obsolete mirror payload indefinitely. See raw-document lifecycle for the retention rule and measurements.

Repeated origin text has one exact dictionary home, with every fact association and proof field retained. Current facts and consensus linksets also reference one exact owner-local product literal. Public product values, fact hashes, event payloads and literal-order cursors retain their meaning; internal product IDs provide storage references only. See product storage. Vulnerabilities stores every complete logical outbox envelope in bounded gzip batches. Event IDs, order, occurrence times and PostgreSQL-normalized payload strings remain intact. Inbox storage, consumer checkpoints, event APIs and retention windows are unchanged. Other owners retain their existing row storage. See event storage.

Both Vulnerabilities images must contain the matching forward migrations and conversion code. Stop both old owner roles before the upgrade; old and new binaries must not serve concurrently. Startup completes owner migrations, awaited provenance and product conversion, shared eventing migrations and awaited outbox conversion before serving or retention. Bounded moves preserve values and remove their source rows atomically; interrupted conversion resumes. Temporary conversion homes are removed at convergence. A restart of an older image cannot perform this upgrade. The storage notes give the exact prerequisites.

Acceptance requires both importers to consume identical full input, complete import/projection/publication, and preserve full semantic coverage, custody, required indexes and the retained event burst. The paired full-seed, complete delta/custody and post-delta five-times proof passed on 2026-10-23; see the accepted receipts above. Count the complete owner database, including indexes and TOAST; report seed/corpus files, transient upgrade storage and WAL separately.

Source verification

This storage-specific review does not replace the historical VRD qualification or establish a shared-estate rollout. The following anchors belong to the source commit named at the top of this page.

Re-verify the source boundaries without modifying a runtime:

rg -n 'MetadataKey|OriginRef|latestState|SeedStateDigest' src/Vulnerabilities/__Libraries/StellaOps.Vulnerabilities.Facts/Normalizers/MirrorSeedProvenance.cs src/Vulnerabilities/__Libraries/StellaOps.Vulnerabilities.Facts/Store/NormalizationPump.cs
rg -n 'AddProvenanceOriginStorage|AddProductStorage|UseCompressedOutboxBatches' src/Vulnerabilities/__Libraries/StellaOps.Vulnerabilities.Persistence/Extensions/VulnerabilitiesPersistenceExtensions.cs
rg -n 'DEFAULT_MAX_INLINE_BYTES|awaiting-download|record_seed_state "imported"' devops/release/bundle/tools/stage-vulnerability-seed.py devops/release/bundle/install.sh
rg -n 'AddHostedService<IngestSchedulerWorker>|settings.Enabled|EcosystemEndpoints|GetByteArrayAsync' src/Vulnerabilities/StellaOps.Vulnerabilities.Worker/Program.cs src/Vulnerabilities/__Libraries/StellaOps.Vulnerabilities.Ingestion/Connectors/OsvSourceConnector.cs

Behavioral re-verification uses MirrorSeedCommandGroupTests in CLI and MirrorRoundTripTests in Vulnerabilities Ingestion, plus ProvenanceOriginStorageTests, the compressed outbox tests and their adjacent controls listed in the storage notes. Use tools/scripts/test-targeted-xunit.ps1 with -BuildProjectReferences and record actual ran-versus-discovered counts. Source tests do not replace the completed full-seed measurement and delta receipt.

Release seeds and data notices

A seed that Stella Ops ships in a release bundle or an offline kit is third-party advisory data under its sources’ terms, not BUSL-1.1 software. Release staging (devops/release/bundle/tools/stage-vulnerability-seed.py) accepts only a seed exported with an explicit --source list whose every key the shipped third-party-licenses/DATA-NOTICES.md clears (“Hub mirror seeds”). suse-csaf is refused by name until the conditions in docs/legal/decisions/mirror-seed-data-notices-counsel-thread-reply.md are met. osv is cleared per record only. The seed must be exported with --osv-upstream, and its manifest must show upstreams.osvRuleApplied. The allowlist and every carried OSV upstream must lie inside the register’s osvUpstreams (devops/release/bundle/tools/vulnerability-seed-cleared-sources.json), each mapped to its notice row. A seed record carries the upstream title, summary and description verbatim. Its origins list and the manifest’s scope.sources and upstreams are the source statement. The exporter itself applies no licence gate: an operator may mirror any source hub-to-hub inside its own estate.

The assembled release manifest signs an explicit vulnerabilitySeed state. For a seeded release it binds the producer’s digest, byte length and advisory count to both data/vulnerabilities/mirror-seed.v1.ndjson.sha256 and mirror-seed.manifest.json in the bundle checksum list. A body above the 256 MiB inline limit is published as mirror-seed.v1.ndjson beside the release archive; the installers discover that sibling after extraction and verify it before import. An explicitly seedless release carries none of those files and makes no import claim.