Scanner record and projection retention
Verified against source commit 981420da61b02afd8dabd348f331463353f9fac7 (RFC-1) and re-checked at 942a18727c (2026-09-26, adds the RFC-19 artifact-BOM window). Re-verify with python tools/scripts/validate/check-p13-retention.py --repo-root . --tables and the ScannerObservationRetentionTests / ConsolidatedScannerSchemaTests suites.
RFC-1 reviews the Scanner persistence chain through migration 010. Every active parent table in the P13 source census carries one class: source-of-truth, append-audit, derived-rebuildable or cache-bounded (the per-table inventory is docs/architecture/retention/p13-source-inventory.json; the census command above prints the current totals). Unused tables are retired by migration 006.
The classification inventory proves source contracts. It does not prove that every repository has a running producer, that historical analysis can be replayed after its inputs are discarded, or that these migrations have reached a particular deployment. Re-run the five P13 commands in the retention guide, then inspect the service’s migration ledger and Doctor checks for that deployment.
Records that must be preserved
Migration 007 states disposal boundaries for the existing authoritative records. The hash-bound proof set separates the classification, the startup migration registration, the disposal boundary and the table’s creating identity. A foreign-key cascade describes what deletion would destroy; it does not authorize deleting the parent. In particular, proof bundles, manifests, binary evidence, proof segments and triage decisions must not be treated as disposable cache entries.
Migration 009 corrects earlier rebuildability claims:
| Table | Why the stored record is authoritative |
|---|---|
call_graph_snapshots | Drift comparison consumes this captured graph; a scan manifest digest does not reconstruct it. |
callgraph_ingestions | The ingestion service stores caller-supplied JSON directly and keeps no second CAS copy. |
cve_symbol_mappings | Upserts include supplied and operator fixture assertions, not just a reproducible corpus import. |
reachability_stack_frames | The repository accepts supplied frame evidence; the read endpoint cannot reconstruct missing frames. |
risk_state_snapshots | Captured EPSS, KEV, VEX and policy values are the historical inputs to risk comparisons. |
triage_effective_vex | Collected signed/source assertions cannot be recovered from references alone. |
triage_reachability_result | An input hash and proof references do not contain the historical assessment inputs. |
triage_risk_result | The policy version, input hash and explanation describe a recorded assessment, without a replay writer. |
unknowns | Scanner queries stored ranking and containment observations, but has no reconstruction writer. |
vex_candidates | Human reviewer, action, time and comment cannot be regenerated by candidate detection. Its store and routes were withdrawn by PCD-222 (2026-10-04); existing rows are kept and nothing reads or writes the table. |
These are storage classifications. They do not add missing producers. func_node and func_trace remain mapped children of signed function proofs; this review found no current application write call for those child sets. The triage assessment and unknown-state readers also remain distinct from a demonstrated ingestion workflow. Preserving their record semantics does not claim those workflows are complete.
Recomputable projections
Migration 010 binds the derived tables to concrete computation code:
| Tables | Reconstruction inputs and code |
|---|---|
code_changes, reachability_drift_results, drifted_sinks | Retained base/head call_graph_snapshots; the Worker’s ReachabilityDriftStageExecutor invokes the extractor, detector and repository replacement (SPRINT_20261017_006 PCD-874; the drift GET recomputes without storing). |
material_risk_changes | Retained risk-state pairs; MaterialRiskChangeDetector.Compare, followed by the material-change repository. |
reachability_results | Captured call graph; ReachabilityAnalyzer.Analyze, followed by the result repository. |
reachability_public_verdicts | Evidence-job analysis and verdict composer; ReachabilityEvidenceJobExecutor publishes through the public verdict store. |
scan_evidence_projections | Scan analysis fragments and findings; ScanEvidenceProjectionStageExecutor. |
vuln_surfaces, vuln_surface_sinks | The same vulnerable/fixed package inputs; VulnSurfaceFeedProcessor builds and persists the result. |
vuln_surface_triggers | Package call graph and sink keys; TriggerMethodExtractor, with the repository’s trigger-write primitive. The feed currently retains the trigger count, not an automatic persistence/rebuild workflow for these rows. |
No projection deletion schedule is introduced. Reconstruction requires the relevant input artifacts and algorithm version. Re-running against a changed feed or package is a new result, not proof of byte-identical historical reconstruction.
Running cleanup
Scanner’s daily ScannerAuditRetentionService (src/Scanner/StellaOps.Scanner.WebService/Retention/ScannerAuditRetentionService.cs) calls scanner.prune_scan_audit(), scanner.prune_reach_cache() and scanner.prune_observations_and_responses(); the last one applies:
| State | Disposal rule |
|---|---|
scan_metrics | 90 days after finished_at, adopting the existing Scanner audit window. |
execution_phases | The parent metric’s window, including recent phase rows attached to an expired metric. A retained metric keeps all its phases. |
runtime_observations | 90 days after observed_at. Signed witnesses and verdicts are stored separately. |
idempotency_keys | After the row’s expires_at; this removes the cached response, not its scan or upload. |
The procedure rejects null, zero and negative retention intervals. A failed daily pass is logged and retried on the next pass, so failure causes over-retention. The observation writer uses the current node/function contract; migration 008 makes the earlier optional image/symbol attributes nullable and defaults was_observed to true for new observations. Existing values are preserved.
The real-PostgreSQL keeper test inserts through PostgresRuntimeObservationStore, proves idempotent observation writes, expires samples and responses, preserves recent lineages and scan identities, and checks a second cleanup pass removes nothing.
Artifact-BOM append-audit window (RFC-19)
scanner.artifact_boms is append-audit by shape (monthly RANGE partitions). Since RFC-19 the daily ScannerPartitionRunwayService (src/Scanner/StellaOps.Scanner.WebService/Retention/) runs two independent steps: it keeps the partition runway ahead of the current month, then calls scanner.drop_artifact_boms_partitions_older_than(12, false) with ScannerSchemaTopology.ArtifactBomRetentionMonths (12), which equals migration 004’s backward runway so the keeper never re-creates a month it has just reclaimed. A failing step does not stop the other. doctor.scanner.retention.artifact-boms-window warns when a monthly child is more than one month past the window. Operations: sbom-hot-lookup-operations.md.
Empty-table retirement
Migration 006 retires binary_patch_verifications, evidence_bundles, reach_graph_snapshots, reach_state_flips, reachability_stacks, scan_findings, vex_statements and vuln_surface_trigger_paths, plus the latter’s two unused SQL readers. Repository-wide caller search and a read-only deployment census found no application storage callers and zero rows in all eight tables. The differently named active stores remain in place.
The migration locks and checks every candidate before any drop. Unexpected records stop startup with SQLSTATE 55000; preserve and review those records before attempting retirement. Explicit drop order handles the VEX-to-stack reference. No CASCADE drop conceals new dependencies. Applied migrations 001–005 are unchanged.
Owner capacity monitoring (DIP-17)
Scanner web registers a leased probe using its explicit Doctor owner pool for measurement, receipt storage and lease fencing. Migration 015_capacity_probe_receipt.sql automatically creates scanner.capacity_probe_receipt; each successful insert keeps only the newest 90 receipts. doctor.scanner.capacity.budget rejects missing/stale receipts, absent required measurements and stop-line breaches. It measures the 40 registered entries against the Scanner budget; that does not certify the other 22 identity declarations or generated reviews.
The database envelope and event-rate basis are in the capacity guide. The probe adds no evidence deletion or automatic writer interception. Its pause flag means pause required; actual writer-gate adoption remains an open obligation. The scanner.scans outbox stream’s retention settings do not authorize deleting the same-named scan identity table. Re-verify the host composition with ScannerCapacityAdoptionTests and the shared CapacityProbeTests through the targeted xUnit wrapper with -BuildProjectReferences.