Vulnerability provenance storage

SIF-2-P normalizes repeated origin references inside the Vulnerabilities owner database. The physical facts.fact_provenance table keeps one association per fact and exact origin. The immutable facts.provenance_origin dictionary stores each (origin_kind, origin_ref) once. Its generated numeric ID is an internal reference; it does not change fact identity, revision hashes, source generations, event IDs or stream epochs.

facts.fact_provenance_read is the canonical seven-column read projection. It is an ordinary join view with no persisted copy. Existing evidence APIs return the complete literal reference, including seed:sha256:<state>/<advisoryKey>. Origin text uses deterministic C comparison and is not trimmed, case-folded or replaced with a checksum. A GHSA advisory origin remains distinct from the CVE to which its facts resolve.

Document digest, verification result, signing fingerprint and recorded time remain on each association. Two facts sharing one origin can retain different proofs and times. A newer allowed assertion updates only its association under the existing raw-revision ordering guard. An unchanged replay rewrites neither association nor dictionary row. A missing incoming digest never erases one already held.

Startup and interruption

Forward migration 030_provenance_origin_dictionary.sql leaves applied migrations unchanged. Fresh and empty databases converge during the migration. A nonempty upgrade temporarily renames the original rows into a startup conversion queue. The awaited ProvenanceOriginStartupConversion moves at most 1,000 rows per transaction, preserving every logical field and the original timestamp, checking exact reconstruction before deleting those source keys. Its cursor/count commit with the move. A failed transaction keeps its source rows; a restarted host continues the committed cursor and checks for any source rows outside it before completing. Migrations create structure only. The awaited converter initializes its bounded operational receipt; an empty provenance cutover preserves an existing corpus resume witness. The singleton receipt is durable owner control state, like the corpus-resume and ingest-cycle cursors. Its old-key strings are cleared when conversion completes.

The owner first registers its own and eventing startup migrations, then the awaited complete-outbox conversion, provenance conversion and product conversion. The narrow eventing startup registration does not start retention; the later full eventing registration reuses its migration/converter and keeps all retention workers after the storage cutovers. This can retire the complete old event source before growing the provenance targets, but does not promise a lower transient peak: compressed target bytes and WAL still need measured admission. Both old owner roles must be stopped for an upgrade; old and new binaries do not serve concurrently during conversion. A non-pooled session fence serializes starting replicas and is released on every exit. A conflicting target, missing source, invalid required index or foreign key fails startup without discarding source data. During conversion both association homes reject inserts, updates, deletes and truncates from ordinary writers, including prepared statements that use columns common to both schemas. Only the fenced converter admits its transactions. Finalization removes these temporary gates.

Finalization drops only the empty queue, installs the canonical view and physical mutation trigger, and completes the bounded receipt. No old complete table, compatibility writer or parallel rollback copy remains. The disposable corpus-publication resume witness changes once, so an interrupted old publication attempt rebuilds its parts. Completed retained publications and DC-06 event stream epochs remain unchanged.

Custody, reclamation and capacity

Raw retention continues to lock the physical association table and probe its document-digest index. Metadata’s restrictive raw-document foreign key remains. Interning an origin never makes a referenced raw record disposable. The fact-leading primary key and digest index remain; an origin-leading index protects the dictionary foreign key and its bounded orphan walk.

The owner collector walks at most ten windows of 1,000 dictionary rows every fifteen minutes. It deletes only unreferenced rows after locking and rechecking them. Link writers keep resolved origins key-share locked through their transaction. Referenced or busy origins survive. The collector writes no fact, event or proof history, and a later reassertion can allocate a different internal ID while returning the identical literal origin.

Capacity treats links plus dictionary as the existing provenance byte family. The owner measurement augmentor adds both physical relation sizes, including indexes and TOAST, to the facts.fact_provenance family line while preserving logical association rows and whole-database bytes. An incomplete conversion or missing storage home is reported absent. A separate dictionary line overlaps the family line and must not be added to it. The existing 32 GB allowance, growth limit, HSR-22 pin and database ceiling remain; HSR-22 owns live budget settlement.

Storage normalization does not establish M9. SIF-2 measures a completed full pinned seed import, projection and publication with every retained event, index and custody association included. Seed/corpus files, transient upgrade disk and WAL are reported separately.

The 2026-10-23 initial paired full-seed receipt records 24,935,978,687 candidate database bytes and M9 at 4.817880213750253, including both physical provenance homes under the unchanged accounting rules. The complete delta/replay/custody and post-delta proof also passed; the same whole-database total includes all retained associations and required custody. This isolated measurement does not establish fresh-machine installation or ordinary configured OSV refresh acceptance.

Re-verification

Run ProvenanceOriginStorageTests in the owner Persistence test project and the adjacent FactBatchWriteTests, FactProvenanceLatestDigestTests, convergence, provenance repair, raw retention and owner evidence endpoint controls using tools/scripts/test-targeted-xunit.ps1 with -BuildProjectReferences. Confirm actual ran-versus-discovered counts. Run the source/table retention and data-identity gates. Source tests establish storage semantics; the paired full-seed measurement establishes the customer footprint target. No shared-estate deployment is implied.