VEX storage-model decision — vex.claims / vexhub.statements double-store: KEEP

Currency notice (2026-07-11): REOPENED. The KEEP conclusion below is a historical 2026-07-01 decision, not the current recommendation. Its own revisit trigger fired: live totals reached approximately 77 GB for vex.claims and 103 GB for vexhub.statements, and a deterministic sample found 988/1,000 statement rows had an exact claim twin on the product-grain identity. The immediate identity repair and source-preserving cleanup are tracked in SPRINT_20260711_003_Concelier_openvex_identity_repair_and_store_cleanup.md; the normalized membership model is intentionally deferred to the next architecture pivot. Preserve this document as the record of the earlier decision, but do not use it to justify shipping both wide derived tables to mirrors.

Status: DECISION (2026-07-01, SPRINT_20260701_002 R4). Recommendation: KEEP the two stores. Scope. Records the blast-radius map of every consumer of both VEX stores, the load-bearing reason the double-store is deliberate (not accidental duplication), the measured relative footprint, the revisit-triggers that would reopen this decision, and the guard test that pins the projection so the two stores cannot silently diverge.

Explicitly NOT doing (rejected): no migration dropping/renaming columns, no unifying view, no change to the VexLens or matcher read paths, and — most importantly — no collapse of vex.claims into vexhub.statements (or vice-versa). The collapse is rejected on the evidence below.

1) The two stores at a glance

The projection is one-directional: vex.claims → (worker VEX provider run) → vexhub.statements. It is not a copy — it is a lossy, re-keyed, verification-annotated derivation (§3).

2) Blast-radius map — every consumer of both stores (verified against src/)

2.1 vex.claims consumers

• Matcher / /excititor/resolve— src/Concelier/StellaOps.Excititor.WebService/Endpoints/ResolveEndpoint.cs:

  • endpoint POST /excititor/resolve registered at ResolveEndpoint.cs:36 (scope vex.read, tenant-required).
  • reads claims at ResolveEndpoint.cs:120 via IVexClaimStore.FindAsync(vulnerabilityId, productKey, …).
  • advisory-only apply gate at ResolveEndpoint.cs:122–133: claims stamped vex.application.mode=advisory (Phase-1 OCI OpenVEX direct-ingest) are visible in the raw claim/observation reads but are excluded from the apply path (IsAdvisoryOnly predicate at ResolveEndpoint.cs:239–241). VexLens computes finding verdicts from the claims this endpoint returns, so filtering advisory claims here is what guarantees they never suppress/affirm a finding in Phase 1.
  • This consumer needs the full claim set incl. advisory claims (it makes the advisory-vs-enforced decision itself). vexhub.statements cannot serve it — the projection only holds enforced claims (§3).

• VEX retention sweep — src/Concelier/StellaOps.Excititor.Worker/Scheduling/VexRetentionSweepService.cs:

  • vex.claims prune target declared at VexRetentionSweepService.cs:60–61 (grouped (tenant, vulnerability_id, product_key, provider_id), latest-wins → superseded rows pruned to current-state).
  • dedup side-tables (vex.product_dedup at :190, vex.claim_metadata_dedup at :196) are orphan-GC’d against vex.claims via an anti-join.
  • vex.claims is FORCE ROW LEVEL SECURITY (migration 001); the sweep and all reads run on a tenant-scoped connection (VexRetentionSweepService.cs:180–181).

2.2 vexhub.statements consumers

• VexLens consensus — src/VexLens/StellaOps.VexLens.WebService/Services/VexHubStatementProvider.cs:

  • VexHubStatementProvider (:10) reads via IVexStatementRepository.SearchAsync(filter{vuln, product}, …) at VexHubStatementProvider.cs:32 (src/VexHub/__Libraries/StellaOps.VexHub.Persistence/Postgres/Repositories/PostgresVexStatementRepository.cs → FROM vexhub.statements).
  • maps the normalized/verification shape into the VexLens NormalizedStatement (statement.Status at :87) and the signature-verification result from statement.VerificationStatus at VexHubStatementProvider.cs:133 (Verified→valid + signer fingerprint, Pending/Failed/Untrusted→warnings/errors) — a shape vex.claims does not carry (it has no verification_status/content_digest column; verification is computed during the projection, §3).
  • This consumer needs the verification-annotated, source-statement-keyed view; it must NOT see advisory-only claims (they were already filtered out of the projection at write time, §3).

• VexHub distribution API — the same vexhub.statements table backs the CVE/PURL/source query + bulk export endpoints Trivy/Grype consume (PostgresVexStatementRepository.GetByCveAsync/GetByPackageAsync/ SearchAsync). Distribution is a first-class VexHub responsibility (docs/modules/vex-hub/architecture.md §2).

• VEX retention sweep — VexRetentionSweepService.cs:86–87: vexhub.statements is an independent prune target grouped (source_id, vulnerability_id, product_key), latest-wins → also bounded to current-state. It is probed with to_regclass and skipped when absent (peer schema migrated by vexhub-web).

2.3 Mirror export / distribution

• Mirror seed export — src/Concelier/__Libraries/StellaOps.Concelier.Persistence/Postgres/MirrorSeed/PostgresMirrorSeedArchiveService.cs: both stores are seed-table specs — vex.claims (tier DerivedVex, ClassifyTier :2826) and vexhub.statements/provenance/conflicts (tier DerivedVexHub, :2831, isVexHubProjection:true at :260–274). vexhub.statements is re-derivable from retained vex.claims and may be dropped from compact bundles. Raw-only VEX source bundles are not safely re-derivable because product fan-out can recreate the storage incident (§4 / R4-D3).

3) Load-bearing reason to KEEP (why this is deliberate, not duplication)

Two different keys and shapes feed two independent consumers, and the two stores encode a deliberate advisory-vs-enforced asymmetry:

1. Different keys. vex.claims is keyed per provider claim (tenant, vulnerability_id, product_key, provider_id); vexhub.statements is keyed per source statement (source_id, source_statement_id, vulnerability_id, product_key). A single provider claim can carry many source-statement ids; the projection re-keys and de-duplicates on the statement key (PostgresVexHubProjectionSink.cs:234–235 conflict key). Neither key is a superset of the other.

2. Different shapes. vex.claims is the full VexClaim (status, justification, detail, product, signals, provenance metadata, application-mode stamp). vexhub.statements is a verification-annotated distribution statement — it adds verification_status, verified_at, signing_key_fingerprint, and a content_digest that are computed at projection time from the raw document signature (ResolveSignature at PostgresVexHubProjectionSink.cs:616; digest at :569). Those columns do not exist on vex.claims. Collapsing would either bloat every claim row with distribution-only columns or strip the columns the distribution/consensus consumers require.

3. Deliberate advisory-vs-enforced asymmetry (the crux). vex.claims holds ALL claims — including advisory-only (per-case untrusted) claims that the resolve endpoint deliberately keeps visible but excludes from apply (ResolveEndpoint.cs:122–133). vexhub.statements holds ONLY the enforced/worker-projected statements — the projection sink is only invoked on the enforced VEX provider run (DefaultVexProviderRunner.cs:185 / VexWorkerLineageCheckpoint.cs:234 → PostgresVexHubProjectionSink.ProjectAsync at :32). Advisory-only claims never reach the projection. This asymmetry is a feature: the matcher can reason over the whole (advisory + enforced) claim set and decide, while VexLens/consensus and the public distribution API only ever see the enforced set. A single store cannot express “visible-but-not-applied” for one consumer and “only-enforced” for the other without re-introducing a per-consumer filter on every read — which is exactly the split we already have, just materialized once at projection time.

4. Both are already supersede-bounded to current-state (§2.1 / §2.2 retention targets), so neither store grows unbounded relative to the other; the double-store is not a bloat driver (§5).

Conclusion: KEEP. The stores are not redundant copies; they are two materialized views with disjoint keys, disjoint shapes, and a deliberate enforced-vs-advisory scope difference, each read by a different consumer on a hot path. Collapsing them would force a per-consumer filter/reshape on every read and delete the verification annotations the distribution/consensus consumers depend on — strictly worse.

4) Re-derivation on import (R4-D3 determination)

Does import re-derive vexhub.statements? YES, but only from retained claims — via the Excititor Worker’s provider-independent local repair loop, declared as the manifest vexhub-project rebuild stage.

• The pure-SQL, host-agnostic MirrorSeedRebuildService does not run the vexhub-project stage; it only drains the issue projection.
• VexProjectionRepairHostedService runs the actual rebuild inside the Excititor Worker without a provider fetch. It asks PostgresVexProjectionRepairCandidateStore for retained claim documents with missing statements. VexProjectionRepairExecutor reuses the exact imported claims and invokes the production alias writer and PostgresVexHubProjectionSink. It never normalizes raw-only documents. All provider schedules may remain disabled.
• Normal provider ingest remains an additional recovery path: re-seen documents re-drive claim, alias, and VexHub writes because claims are not an atomic completion marker.
• VexHub migration 003_statement_source_document_lookup.sql keeps the repair anti-join bounded by (source_document_id, source_id, vulnerability_id, product_key).

Because claim-backed re-derivation exists, compact mirror seeds default to dropping the statement bytes (R4-D3):

• The SkipVexHubProjection flag already existed on both the export and import requests (MirrorSeedArchiveModels.cs:44 / :156) and drops the whole VexHub surface (vexhub.sources/statements/provenance/conflicts — IsVexHubSeedProjection at PostgresMirrorSeedArchiveService.cs:1019, OptionalScope.VexHub or isVexHubProjection).
• SourceOnly drops both claims and the DerivedVexHub tier. Export therefore refuses a non-empty VEX source-only bundle rather than promising an unsafe raw-to-product-grain rebuild.
• New default (StellaOps.Concelier.MirrorSeedTool): a Compactseed now defaults to SkipVexHubProjection=true (ResolveSkipVexHubProjection in MirrorSeedTool/Program.cs), because the statements are re-derivable from retained claims. Full stays a faithful clone (never auto-skips). --skip-vexhub-projection is accepted only when claims are retained; --keep-vexhub-projection opts a compact seed back in. Forward-only, additive, no destructive change.
• Trade-off (documented): a skipped seed imports with empty VEX consensus until local Worker repair completes. Air-gap sites trade a one-time destination re-projection (local CPU/DB I/O) for a smaller distribution bundle; no upstream network is required.

Proof is deliberately split:

• PostgresMirrorSeedArchiveServiceTests.ExportImport_SkipVexHubProjection_OmitsStatementBytes_AndDeclaresLocalRebuild proves the archive omits projection bytes, retains the claim rebuild source, and declares vexhub-project; export fails closed when no safe rebuild source is retained.
• OpenVexCanonicalIdentityPersistenceE2ETests.RawOnlyZeroState_LocalRepairDoesNotExpandProductGrainClaims proves a raw-only fresh database remains unexpanded.
• OpenVexCanonicalIdentityPersistenceE2ETests.PartialVexHubProjection_LocalRepairPreservesExistingReaderRowAndAddsMissingStatement proves retained claims rebuild a missing statement while preserving an existing reader row.
• OpenVexCanonicalIdentityPersistenceE2ETests.RetainedAdvisoryClaim_RemainsReadableButIsNotProjectedToVexHub proves advisory-only claims remain visible to the claim reader but outside VexHub consensus.

5) Measured footprint — what fraction of vex.claims the projection actually is

Real live-DB pg_total_relation_size measurements recorded in docs/implplan/SPRINT_20260701_001_Mirror_source_only_distribution.md (Execution Log, 2026-07-01, stellaops_platform, read-only):

Fraction: vexhub.statements (~2.5 GB) ≈ 96% of the vex.claims claim surface (~2.6 GB) on the current corpus. This is higher than the “expected << 50%” the sprint hypothesized — see the first revisit-trigger (§6). It is still a small fraction of the total DB (~2.5 GB of a store dominated by the ~68 GB issue projection), and it is fully re-derivable (§4), so it is dropped from distribution bundles. Note the per-relation vex.product_dedup / vex.claim_metadata_dedup sizes are not separately recorded in SPRINT_20260701_001 (they are subsumed in the ~2.6 GB claim figure); the command below returns them explicitly.

Reproducible read-only measurement command (no mutation)

The read-only reader VexStorageStatsReader (src/Concelier/__Libraries/StellaOps.Excititor.Persistence/Postgres/VexStorageStatsReader.cs) issues exactly these queries. To measure the four relations by hand (a tenant-scoped connection — vex.claims is FORCE-RLS so require_current_tenant() must be satisfied; the pg_* functions are global):

-- Read-only; no writes. Run on a tenant-scoped connection (SET app.tenant_id = '<tenant>').
SELECT rel,
       pg_size_pretty(pg_total_relation_size(to_regclass(rel))) AS size,
       pg_total_relation_size(to_regclass(rel))                 AS size_bytes
FROM (VALUES
    ('vex.claims'),
    ('vex.product_dedup'),
    ('vex.claim_metadata_dedup'),
    ('vexhub.statements')
) AS t(rel);
-- statements-as-fraction-of-claims:
SELECT round(100.0 * pg_total_relation_size(to_regclass('vexhub.statements'))
                   / NULLIF(pg_total_relation_size(to_regclass('vex.claims')), 0), 1) AS statements_pct_of_claims;

to_regclass returns NULL for an absent relation and pg_total_relation_size(NULL) is NULL, so a missing peer table is simply skipped (never fatal) — identical to the reader. This is a pure catalog read; it does not touch or restart the running VEX ingest.

6) Revisit-triggers (when to reopen this decision)

Re-evaluate KEEP vs. collapse if any of the following becomes true:

1. vexhub.statements grows to rival or exceed vex.claims. The projection is a subset (enforced only) with a different key; if the measured statements-as-fraction-of-claims stays ≥ ~90% (as it is today, §5) or exceeds 100%, the projection is materializing more than it should — investigate whether advisory claims are leaking into the projection, whether supersede-bounding is failing on the statement side, or whether the fan-out per claim has grown. (Today’s ~96% is already high enough to warrant a watch; it is retained because both stores are supersede-bounded and the absolute size is small.)
2. A new consumer needs a unified read across both the advisory and enforced sets with the same key and shape. If a single consumer legitimately needs both stores joined on every read, a unifying view (or a single store with a mode discriminator column) may beat maintaining two materialized views.
3. The enforced-vs-advisory asymmetry disappears — e.g. if Phase-1 advisory-only gating (ResolveEndpoint.cs:122–133) is removed and all claims become enforced, the advisory-vs-enforced reason (§3.3) no longer holds and the two stores would carry the same claim set (different key/shape only).
4. The projection stops being re-derivable (e.g. the worker vexhub-project path is removed), which would break the distribution disk-win (§4) and change the KEEP cost/benefit.

7) Guard — the projection cannot silently diverge

PostgresVexHubProjectionSinkTests (SPRINT_20260701_002 R4-T2, src/Concelier/__Tests/StellaOps.Excititor.Worker.Tests/VexHub/PostgresVexHubProjectionSinkTests.cs) pins the projection derivation against the claim shape/enum vocabulary using the real write-path derivation (exposed via internal test-only surfaces on the sink and PostgresVexClaimStore — no test-only logic). It asserts:

• content_digest equals an in-test-computed SHA-256 over the documented material order (computed from the inputs, not a hard-coded literal — refactor-safe);
• a stable projected status string for every VexClaimStatus value;
• a verified signature ⇒ verification_status='verified' + the signing-key fingerprint;
• the claim-side (PostgresVexClaimStore.ToStorageValue) and sink-side (PostgresVexHubProjectionSink.FormatStatus) status strings agree for every enum value (iterated over the whole VexClaimStatus enum) — a divergence in either mapping fails the guard.

If a future change re-keys, re-shapes, or re-maps the status/justification/verification vocabulary on one side without the other, this guard fails before the divergence reaches production.

8) Related

• VexHub architecture — cross-linked (§ Storage model).
• Excititor / Concelier architecture — cross-linked (§ VEX storage model).
• Mirror source-only distribution — recorded footprint measurements + rebuild-on-import.
• SPRINT_20260701_002 R4 — this decision’s delivery tracker.