Estate service-consolidation and data-hygiene review

Status: Accepted direction (ADR-039 rev-4 D14, owner-approved 2026-07-22; per-family decisions land in programs SPRINT_20260722_010–016). Evidence: live container list, live PostgreSQL catalog (read-only, 2026-07-22), src/ module layout, §1.4 module map.

Purpose: the vulnerability-plane incident is a symptom of two estate-wide habits — deployables that were split without a real boundary, and data homed with its consumer instead of its domain. This review applies the same correction everywhere before implementation starts, so the foundations survive the rewrite instead of leaking the old shape into it.

1. Consolidation criteria (the durable rule)

Consolidate two deployables into one service when all hold:

1. They share one domain language (same nouns, same lifecycle).
2. They read/write the same tables or one is meaningless without the other.
3. Their scaling profile is not fundamentally different (web/worker split within a service is fine).

Keep deployables separate when any hold:

• Security boundary — key custody, token issuance (Signer, Authority).
• Failure/capacity blast radius — bulk data planes vs. control plane (Vulnerabilities, Scanner).
• Regional/optional shipping — features that some estates must be able to exclude entirely.

Corollary (domain-homing rule, ADR-039 P14): data lives with its domain, not with its consumer. A service may cache another domain’s data only as a declared, rebuildable projection.

1.1 Product context (the immersion baseline every merge program starts from)

Stella Ops is a self-hosted release control plane for non-Kubernetes container estates. The people it serves and the process every service must map onto:

• Personas: the platform operator (runs the stack — connected or fully air-gapped, regional crypto via plugins); the release manager (promotes Dev → Stage → Prod, needs gates that explain themselves); the security engineer (triages findings, authors VEX/exceptions, tunes trust); the auditor (verifiable evidence, attestations, deterministic replay).
• Core process: knowledge in (Vulnerabilities hub) → scan artifacts (Scanner/SBOM) → match + reachability (Findings/Signals/ReachGraph) → policy verdict (Policy) → gated promotion (ReleaseOrchestrator) → deploy by digest (agents) → evidence (Attestor/EvidenceLocker/Timeline).
• Environment truths: everything must work offline (mirror bundles are first-class); every release pins a sha256 digest; multi-tenant with a single-operator default.

A merge is only right if it makes one of these persona/process seams simpler to explain. Each program’s first task is to verify this brief against src/ and the users’ actual flows before any code moves.

2. Target service topology (now → goal)

~44 application deployables today → ~24 deployable groups / ~20 services & databases.

Per owner decision (ADR-039 D14): each merge is its own program — sprints SPRINT_20260722_010–016 — and each starts with an immersion/design phase. Method per D2: rebuild-from-scratch default, mechanical transfer only where purely tool-based and cheaper (agent’s judgment); never port a layer the merge exists to fix. Execution checklist (binding for every program): service-consolidation-recipe.md — stages S0–S10 cover design, decision, new service, persistence, refactor, API/gateway, UI, doctor/conformance, devops, data move, and obsoletization/docs; a stage without its evidence is not done. The full container-level rename map lives in database-ownership-matrix.md §5.

2.1 Family briefs (what it is, who uses it, what the merge develops into)

Findings family — SPRINT_20260722_010. Findings ledger (per-tenant finding lifecycle/triage history), RiskEngine web+worker (risk/EWS scoring), Findings-Security (the digest-scoped security read-models the release gates and Console query), VulnCorrelation (analytics correlation). Users: security engineer (triage), release manager (gate context), dashboards. Process seat: everything after match/VEX/policy — “what do we effectively have, how bad is it, what did we decide.” Target: one StellaOps.Findings (web + scoring worker), stellaops_findings. This program also resolves the X9 seam (release.security_finding_projection writer) and pins the digest-scoping contract (a real past leak) as a test.

Attestation family — SPRINT_20260722_011. Attestor (attestation verification, proof chains, trust verdicts), tile proxy (Rekor tiles for offline verification), TSA (local RFC 3161, ADR-036). Users: auditor (proof), release manager (evidence on decisions), machines (verify-at-gate). Target: one StellaOps.Attestor with tileproxy/TSA as roles, stellaops_attestor (absorbing attestor + proofchain). Signer stays out — key custody is a security boundary. Design note: TSA keeps runtime isolation as its own role so timestamp availability does not couple to verification load.

JobEngine — SPRINT_20260722_012. Scheduler is the estate’s job backbone: schedules, runs, workers, locks, execution logs, failure signatures — plus queues other services feed (policy_jobs/policy_run_jobs, graph_jobs) and doctor_trends. PacksRegistry is the signed Task Pack registry: packs, pack_versions, attestations, lifecycles, mirror_sources (air-gap distribution), parities. One domain: packs define work, the scheduler runs it — a registry of automation with nothing to execute it, or an engine with nothing to define its work, is half a product. Users: operator (schedules/monitoring), pack authors, and every service that submits jobs. Target: one StellaOps.JobEngine (web + worker), stellaops_jobengine. The merge’s payoff is the job-submission API/queue that kills X8 (Policy writing scheduler tables directly) and any future direct-schema submitters. Alternative considered: PacksRegistry as part of a registry/artifact family — rejected: its only consumer is the engine.

AdvisoryAI — SPRINT_20260722_013. Advisory-AI web+worker (drafting/summarization over a RAG store — kb_chunk) + OpsMemory (operational memory). Users: security engineer, operators. Target: one StellaOps.AdvisoryAI, stellaops_advisoryai. Carry-in: P13 index-budget review (kb_chunk holds 130 MB of index over 37 MB of heap) and a retention class for embeddings.

BinaryIndex — SPRINT_20260722_014. Binary fingerprinting/backport detection (ADR-022 delta signatures) + Symbols server. User is mostly the scanner pipeline (machine consumer). Target: one StellaOps.BinaryIndex, stellaops_binaryindex (absorbing binaries, binary_index, symbols).

Notify — SPRINT_20260722_015. Notify-web (channels/templates/routing/inbox) + notifier-worker (delivery). The notifier schema is live-empty — the split is an accident of history, not a boundary. Target: one StellaOps.Notify web+worker, stellaops_notify. Keep the monthly delivery partitioning (it is the estate’s good P13 precedent).

Authority + IssuerDirectory + tenancy/IdP — SPRINT_20260722_016. IssuerDirectory (trusted VEX issuer catalog) is identity-adjacent and already maps to Authority in §1.4; D10 additionally makes Authority the sole owner of tenants + IdP configuration (killing X6/X7). Users: operator (trust and identity config), every service (authN). Design caution: Authority is the bootstrap root — the merge must not widen its unauthenticated surface; issuer-directory endpoints keep their own scopes.

VexLens folds into the Vulnerabilities hub when the hub serves its consensus queries — tracked as a close-out item of the hub program, not a separate merge.

2.2 Owner evaluation questions (2026-07-22) — architect positions with evidence

Owner approved all positions below on 2026-07-22 (“i agree for everything”) except airgap, which got a follow-up investigation — see §2.3. Decided outcomes: Evidence merge → SPRINT_20260722_011 widened; Graph+ReachGraph → 023; registry→Integrations → 024; doctor-web retirement → 009 DOC-5; remediation archive → 020 W3-14; export→hub rejected.

1. Evidence + Signer + Attestor merged? Signer: no, ever — key custody is the one boundary this review never trades away (a signing key next to a parser of untrusted attestations is how supply-chain products die). EvidenceLocker + Attestor(+TSA/tileproxy): lean merge into one StellaOps.Evidence service — same evidence-spine domain, same auditor persona, both append-only, and the locker’s relational footprint is 1.6 MB (bulk artifacts live in object storage), which removes the blast-radius objection. EVL-4, run inside 011’s immersion.
2. Export* into the Vulnerabilities hub? No. ExportCenter is tenant-scoped, multi-domain operator tooling (evidence/report/SBOM exports too); the hub is tenant-free by rule (P12). The vulnerability-specific exports (compact corpus, mirror bundles) are already hub plugins — that is the correct amount of “export in the hub”. EVL-3 verifies the profile inventory.
3. graph* + reachgraph*? Lean merge graph-api + reachgraph-web (one graph domain: index + serve); signals stays separate (runtime fact ingest, eBPF-adjacent, different scaling profile). EVL-2.
4. doctor-web after the Platform registry? Lean retire. Checks execute inside services; the capability catalog lives in Platform; Console reads a thin aggregation endpoint; scheduled estate-wide runs become JobEngine schedules. The Doctor module survives as plugin SDK + CLI. 009 DOC-5.
5. registry* under Integrations? Lean yes — registry credentials/config are integration domain (ADR-033; connector:credentials:* already lives there) and registry-token is a thin issuer. Guardrail: token issuance is on the deploy hot path — availability-isolated role, revisit if Integrations grows webhook-heavy. EVL-5.
6. Remediation used at all? Dormant, evidence-backed: src/Remediation (fix-template repository) has no compose service, no gateway route, no UI consumer; AdvisoryAI’s autofix has its own in-module Remediation/ (delta service + GitHub/GitLab/AzureDevOps PR generators). Disposition: archive or fold into AdvisoryAI. EVL-6; the 020 W3-14 row is disposition-only.
7. airgap* — used? merge into export? Used, actively: Platform is wired to the controller and time anchor (STELLAOPS_AIRGAP_CONTROLLER_URL/_TIME_URL), airgap:seal/ airgap:status:read scopes are live, and Concelier runs an air-gapped EPSS import mode. Superseded by the §2.3 investigation after the owner’s follow-up — the standalone-keep lean was wrong in one respect: the module genuinely mixes posture with exchange machinery.

2.3 AirGap investigation (owner follow-up 2026-07-22): “isn’t that a mixture of evidence/export? how does it work with its own db?”

Evidence:

• Its database barely exists. The airgap schema is a single 32 kB table, airgap.state (per-tenant: sealed, policy_hash, time_anchor, last_transition_at, staleness_budget, drift_baseline_seconds, content_budgets) — pure posture state. Nothing else.
• The module mixes two concerns. src/AirGap/ contains the Controller (seal/status posture API), Time (anchor), Policy (staleness lib) — and StellaOps.AirGap.Importer with Bundle + Sync libraries: bundle import machinery, which is exchange/mirror domain. Meanwhile bundle creation lives in src/Mirror/StellaOps.Mirror.Creator and operator export jobs live in ExportCenter — the offline-exchange capability is scattered across three homes.
• Evidence is a consumer, not a home: sealing transitions and bundle imports should emit evidence/timeline events; that does not make AirGap part of the Evidence service.

So the owner’s instinct is correct: it is a mixture — of posture (security control) and exchange (data movement) — and the one-table DB shows a standalone service+database is over-boundaried. Proposed split (EVL-3, S1 pending owner confirmation):

1. Posture → Platform. Sealed-mode state, seal/status API, staleness budgets, time-anchor validation are estate posture — a platform concern, not domain logic (the platform-stays- domain-agnostic rule holds). The airgap-controller and airgap-time deployables retire; the airgap schema retires into Platform’s database.
2. Exchange → StellaOps.OfflineKit. AirGap Importer/Bundle/Sync + Mirror.Creator + ExportCenter become one offline-exchange service (stellaops_offlinekit) owning bundle envelope/packaging/import gating/export jobs. Domain content stays with the domains: the Vulnerabilities hub keeps its own mirror plugins (D2/D13); OfflineKit owns transport and gating, and consults Platform’s posture before any exchange.
3. Replay stays separate (deterministic replay verification is its own concern) — confirmed in the same EVL-3 decision.

Net effect if confirmed: −3 deployables (airgap-controller, airgap-time, export split re-formed), −1 database, and the offline story gets one owner instead of three.

Owner approved 2026-07-22 (“I agree, we need to unify them — go for it”) → program SPRINT_20260722_025. The program’s dig added: Mirror.Creator already schedules ExportCenter runs (the coupling exists de facto — DC-22); AirGap.Bundle is the envelope seed (DC-21); the CLI already owns the operator surface (airgap/offline/export/mirror-seed command groups).

3. Data mis-homing and bloat findings outside the vulnerability plane (measured)

4. Anti-bloat data-modeling rules (ADR-039 P13 — binding for all new schemas)

1. Every table declares a retention class in its migration header comment: source-of-truth (immutable/content-addressed), derived-rebuildable (must have a documented rebuild path), append-audit (partitioned, with a retention window), or cache-bounded (size/TTL-capped).
2. No second full representation of the same semantic fact. Derivations are lineage/projections, rebuildable by definition.
3. Verbose payloads (explanations, raw JSON) never live in hot rows — separate table, single copy, TOAST-compressed, partitioned if append-only.
4. Every index is justified by a named reader; index bytes may not exceed heap bytes without a recorded reason.
5. Per-database budgets + a doctor-plugin size check (P11) report monthly growth per schema; breaching a budget is an operational alarm, not a cleanup sprint.
6. Physically idempotent writers — no unconditional DO UPDATE; content guards everywhere.

These rules are enforceable in review and partially by conformance tests (migration-header lint, index/heap ratio check in the doctor plugin).