Evidence — architecture

Status. This is the module dossier for the consolidated Evidence service (ADR-039 D14), live on the local estate since the EVD-9 cutover window (2026-09-05). It supersedes attestor/architecture.mdand evidence-locker/architecture.md, which describe predecessor hosts that no longer exist. Signer is not part of this family and never joins it: key custody is a separate service with a separate database.

Verified against commit 4e99c41ea1 (2026-09-05). Re-verify with:

# the two roles and what composes them
rg -n "CreateBuilder" src/Evidence/StellaOps.Evidence.WebService/Program.cs src/Evidence/StellaOps.Evidence.Worker/Program.cs
rg -n "^app\.(Map|Use)" src/Evidence/StellaOps.Evidence.WebService/Program.cs
# the family's own database, and only its own
rg -n "ConnectionEnvironmentVariable|AdditionalOwnedSchemas|refuses to start" \
  src/Evidence/__Libraries/StellaOps.Evidence.Persistence.Consolidated/Extensions/ConsolidatedEvidencePersistenceExtensions.cs
# the removed trusted-network bypass
rg -n "StellaOpsBypassMode|BypassNetworks" src/Evidence/StellaOps.Evidence.WebService/Program.cs
# the published route surface, pinned row by row
wc -l src/Evidence/__Tests/StellaOps.Evidence.WebService.Tests/Fixtures/consolidated-route-inventory.tsv
# the compose definition and its conformance pack
pwsh ./tools/scripts/test-targeted-xunit.ps1 \
  -Project src/__Tests/architecture/StellaOps.Architecture.Contracts.Tests/StellaOps.Architecture.Contracts.Tests.csproj \
  -Method "*EvidenceConsolidationCompose*"

Timestamp Assurance is composed in source. The web role registers installation checks and the tenant API; EvidenceWorkerComposition registers the same sources plus the schedules. Runtime availability still depends on the deployed image and configured provider/trust inputs.

The worker health update was activated on 2026-09-12 from clean source 5dec570ecf (image pin a16beb008d). Both operational endpoints returned HTTP 200 and the original storage/network bindings were preserved. The live audit initially omitted a shared catalog migration; its correction and exact outcome record the additive empty-observations table and zero rows at verification. Readiness proves host startup, not successful queue processing.

1. One family, two roles

src/Evidence/ ships one deployable family with two replica roles. They are not two services: they share one database, one migration set and one source graph, and neither project references the other (P19, stated in both csprojs).

The worker serves the shared liveness and startup-readiness endpoints, with matching Compose healthcheck paths. Its loops are composed by EvidenceWorkerComposition.cs:52 (AddAttestorBackgroundWorkers) and :59 (AddEvidenceLockerInfrastructure(..., EvidenceLockerHostedServiceOptions.ConsolidatedWorker)), and the object store is resolved eagerly after Build() so a misconfigured store fails at startup rather than on the first job (Worker/Program.cs:35-36, pinned by EagerStoreResolutionConformanceTests).

Verified-by: src/Evidence/StellaOps.Evidence.WebService/Program.cs:52,386; src/Evidence/StellaOps.Evidence.Worker/Program.cs:15,35-37; devops/compose/docker-compose.evidence.yml.

What the tree holds

Twenty-three libraries under src/Evidence/__Libraries/, in three groups. The grouping matters because it says which code was written for this family and which was carried into it:

• Born new (5) — StellaOps.Evidence.Persistence.Consolidated (the family’s one DAL) and the four StellaOps.Evidence.TimestampAssurance* projects (composed through the hosting library, see the banner).
• Carried from Attestor (14) — StellaOps.Attestor.Application, .Core, .Infrastructure, .Persistence, .ProofChain, .Verify, .Bundle, .Bundling, .CapsuleProjection, .Spdx3, .StandardPredicates, .TrustRepo, .Watchlist.
• Carried from EvidenceLocker (4) — StellaOps.EvidenceLocker, .Core, .Application, .Infrastructure.

The carried application types use StellaOps.Evidence.Attestation.* and StellaOps.Evidence.Locker.Api.*. The latter includes the carried audit, security and contract types. Assembly names and wire contracts are unchanged. The retained verdict-ledger endpoint library also uses the consolidated attestation namespace; its historical project filename is not a host. EvidenceApplicationNamespaceTests pins the emitted types and source declarations against predecessor namespace reintroduction. Rebuild each consuming role with its matching application libraries; these internal CLR type names have no compatibility aliases.

2. The database boundary

The family owns one physical database, stellaops_evidence, on the shared PostgreSQL installation, with its own role (CoC §8.2, ADR-039). Not a per-service PostgreSQL container: the scanner precedent e01c4a43ce retired exactly that over-realization.

One connection knob, no fallback. The DAL resolves STELLAOPS_POSTGRES_EVIDENCE_CONNECTION, then the equally specific configuration key Postgres:Evidence:ConnectionString, and otherwise throws:

Consolidated Evidence persistence refuses to start without its own database. Set STELLAOPS_POSTGRES_EVIDENCE_CONNECTION (or configuration key Postgres:Evidence:ConnectionString) to the stellaops_evidence connection string. There is no generic shared-connection fallback (CoC §8.2 / ADR-039 database-per-service).

Both roles then assign that one resolved string into the carried compatibility keys (EvidenceLocker:Database:ConnectionString, attestor:postgres:connectionString and the rest) rather than binding them, and set EvidenceLocker:Database:ApplyMigrationsAtStartup = false. That is the mechanism that makes a second way into a different database impossible: a carried key cannot point somewhere else, because nothing reads it from configuration any more.

Forty-six tables, of which twenty-seven carry FORCE ROW LEVEL SECURITY— every evidence_locker table and every evidence table except the migration ledger itself. Measured on the live database 2026-09-05, not read off the migrations:

select n.nspname, count(*) filter (where c.relkind='r') as tables,
       count(*) filter (where c.relkind='r' and c.relforcerowsecurity) as forced
from pg_class c join pg_namespace n on n.oid = c.relnamespace
where n.nspname in ('evidence','attestor','proofchain','evidence_locker')
group by n.nspname order by n.nspname;

That is why the owning role must be neither superuser nor BYPASSRLS: the service connects as the owner, which is precisely the account FORCE exists to subject to the policy. The locker policies call evidence_locker_app.require_current_tenant().

Migrations are forward-only (ADR-004), embedded, and applied on startup by both roles through AddStartupMigrations(schemaName: "evidence", moduleName: "Evidence.Consolidated", ...), with the ledger in evidence.schema_migrations. Four files: the consolidated baseline plus three Timestamp Assurance migrations.

Correction of record — the applied baseline’s DC-26 note is stale and is NOT edited in place. .../Migrations/001_evidence_consolidated_baseline.sql:76-77,223-224 still says that platform-web’s AttestorMigrationModulePlugin (schema proofchain) and EvidenceLockerMigrationModulePlugin (schema evidence_locker) run and must stay “until the EVD-9 cutover”. That cutover executed on 2026-09-05 and SPRINT_20260722_026 CM-2 (85290801d1) deleted both plugins the same day, so the sentence is false at HEAD. The SQL is an APPLIED migration and ADR-004 is forward-only, so it must not be corrected in place by anyone — this paragraph is the correction the note points to. Nothing else in the baseline is affected: the note is a comment, not a statement the migration acts on. The same retirement also makes this assembly the family’s sole migration authority, because the consolidated host composes with RegisterLegacyMigrationAuthority false and therefore does not register attestor-web’s own AddStartupMigrations either.

Verified-by: src/Platform/__Libraries/StellaOps.Platform.Persistence/MigrationModulePlugins.cs:34-41,111-114 (the removal notes stood where the two plugins were, and the file no longer declared either type; 2026-09-14 (SPRINT_20260722_021 PLT-4): that file and the whole IMigrationModulePlugin mechanism are now deleted, so the verification point is the absence of the mechanism itself); src/Evidence/StellaOps.Evidence.WebService/Program.cs:114; src/Evidence/__Libraries/StellaOps.Attestor.Application/AttestorWebServiceComposition.cs:677-683; DatabaseOwnershipConformanceTests.CentralMigratorPluginFile_ShrinksOnly (src/__Tests/architecture/StellaOps.Architecture.Contracts.Tests/DatabaseOwnershipConformanceTests.cs:1127).

Three named Npgsql pools are built from that one string — Evidence.DoctorStandard, Evidence.TenantLifecycle, Evidence.Doctor — so pg_stat_activity attributes a session to the subsystem that opened it.

Verified-by: src/Evidence/__Libraries/StellaOps.Evidence.Persistence.Consolidated/Extensions/ConsolidatedEvidencePersistenceExtensions.cs:31,37,40,49,56-57,89-92,120-141; .../Migrations/001_evidence_consolidated_baseline.sql:229-232,964,1848-1860; src/Evidence/StellaOps.Evidence.WebService/Doctor/EvidenceSchemaTopology.cs:33; src/Evidence/StellaOps.Evidence.WebService/Program.cs:72-73,88-91,217-229,257-264; DatabaseOwnershipConformanceTests, WebServiceStartupMigrationsConformanceTests.

Decision capsule ledger writes

The consolidated baseline defines attestor.verdict_decision. The verdict writer binds the lowercase decision label using the destination column’s declared PostgreSQL type; it does not require attestor on the connection search path. Invalid enum labels are rejected by PostgreSQL.

CapsuleVerdictLedgerPersistenceTests runs the production baseline and writer as a restricted role, verifies all four decisions, tenant-specific chain continuity and foreign-tenant refusal, and confirms invalid labels cannot append a row. Re-run it through tools/scripts/test-targeted-xunit.ps1 in the consolidated Evidence host test project.

3. The canonical HTTP surface

The canonical prefix is /api/evidence/v1. It is served natively — there is no rewrite, no alias and no compatibility route group, because AGENTS.md §2.11 forbids one before the first external install and EVD-HTTP-00 ruled it out explicitly.

The published surface is pinned row by row in src/Evidence/__Tests/StellaOps.Evidence.WebService.Tests/Fixtures/consolidated-route-inventory.tsv (95 rows) by EvidenceRouteInventoryTests.WebHost_PublishedRouteInventoryMatchesTheReviewedFixture, which enumerates the real host’s endpoint data source. Regenerating the fixture (STELLAOPS_EVIDENCE_ROUTE_INVENTORY_UPDATE=1) rewrites it and still fails, so a regeneration can never read as a green run.

Twenty-four rows are deliberately not under the canonical prefix, and that is not drift. Only the minimal-API maps take the routePrefix argument; attribute-routed MVC controllers carry absolute [Route(...)] templates the prefix parameter does not touch. ProofChainController is the one place both worlds meet: it declares [Route("api/v1/proofs")] and [Route("api/evidence/v1/proofs")], which is why the same operations appear under both. A blanket prefix rewrite of any consumer is therefore wrong — EVD-10c measured eleven predecessor literals that are correct exactly as written.

The Attestor half is mapped by app.UseAttestorWebService(..., routePrefix: "/api/evidence/v1", mapHealthEndpoints: false); the legacy health routes are suppressed so the host keeps a single readiness truth. The locker half maps nine native endpoint groups (evidence core, capsules, capsule signing keys, evidence threads, exports, verdicts, audit, regulatory artifact ledger, legacy capsule erasure).

Four retrieval routes that the retired evidence-locker-web mapped inline were carried deliberately and are pinned by EvidenceCarriedPredecessorRoutesTests, which also pins that one withdrawn route does not reappear.

Measured live, 2026-09-05, from inside the running container:

Verified-by: src/Evidence/StellaOps.Evidence.WebService/Program.cs:342,351-355,357-361,366-374,380; src/Evidence/__Libraries/StellaOps.Attestor.Application/Controllers/ProofChainController.cs:17-18; src/Evidence/__Tests/StellaOps.Evidence.WebService.Tests/EvidenceRouteInventoryTests.cs:63-123.

Retired with the predecessors

The fifteen predecessor gateway rows were retired by RAR-7 in SPRINT_20260809_001; the predecessor hostnames attestor.stella-ops.local and evidencelocker.stella-ops.local were deleted with their compose keys and are not claimed by the successor. Callers still addressing them are the six cross-program repoints tracked in caller-repoints.tsv, owned by sprints 007/010/017/018/025/026. Handing them a silently working alias would re-open the tokenless submit path ROA-4 closed.

4. Identity, scopes and the removed bypass

The host registers as a resource server and then clears the global required-scope set: a single required scope would make either the Attestor half or the Locker half unreachable by construction. Authorization is per-endpoint instead, with a fallback policy of “authenticated, and holds evidence:read”, so an unpolicied endpoint is closed rather than open.

Anonymous by design: /health/ready, /openapi/{documentName}.json, and the capsule signing-key JWKS.

There is no trusted-network bypass on this host, by construction. ROA-4 sets options.BypassMode = StellaOpsBypassMode.Disabled and clears BypassNetworks in code, not only by omitting the networks from compose, so a stray BypassNetworks entry cannot re-open the door. The Attestor policies lost their bypass branch in the same change: the scope set is the whole decision on the direct path as well as through the gateway. The measured failure this closes is on record — during the EVD-9 window a token carrying no attest:* scope got HTTP 200 on POST /api/evidence/v1/rekor/verify through the gateway. The 401 measured above on an anonymous in-network request is the live confirmation.

watchlist:admin is deliberately not registered: it guarded zero routes, and AttestorPolicyClaimPublicationTests.WatchlistAdminPolicy_IsNotRegistered_AndGuardsNoRoute keeps it that way.

Verified-by: src/Evidence/StellaOps.Evidence.WebService/Program.cs:157,159,162,165-173,186-195; src/Evidence/__Libraries/StellaOps.Attestor.Application/Security/AttestorPolicies.cs:21-33,75-80,81-130; .../Security/AttestorScopeRequirement.cs:35-42,80; src/Evidence/__Tests/StellaOps.Evidence.WebService.Tests/AttestorPolicyClaimPublicationTests.cs:172.

5. Transparency, timestamping and the verification context

Local transparency log. LocalTransparencyRekorClient is selected only when an NpgsqlDataSource is present and attestor:transparency:local:enabled is true; otherwise the host falls back to HttpRekorClient. It signs the checkpoint (signed tree head) with the key named by attestor:transparency:local:signingKeyId, falling back to the sole enabled entry in attestor:signing:keys. If that resolution is ambiguous or empty the checkpoint is left unsigned and SignCheckpointAsync swallows it — which is why a proof backfill run without those keys looks successful and the entries still verify as tampered.

There is no separate “checkpoint public key” setting: the public half is reached through the signing key registry, and verification uses the same signer.

RFC 3161 timestamping. Capsule sealing POSTs a DER timestamp query to the in-repo tsa service and embeds the returned token as an existence-time anchor, never folded into the deterministic bundle digest (ADR-036 §Determinism). The runtime configuration validator refuses to start outside the local harness if the client resolves to the null implementation, “which seals evidence without an RFC3161 anchor”.

The carried verification context is a compose contract, not a default. The EVD-9 window measured 26 of 26 carried attestations verifying ok:false because the consolidated overlay restated none of the 74 attestor__ keys the predecessor carried, so product defaults applied: mTLS-only writes and no checkpoint key. Twenty-eight keys per role are now restated in docker-compose.evidence.yml and asserted key by key by EvidenceConsolidationComposeConformanceTests.StagedCompose_CarriesTheTransparencyVerificationContextAndTheCarriedWritePosture. Delete any one of them and that test fails naming the key, while docker compose config still renders green — which is exactly how they went missing.

Verified-by: src/Evidence/__Libraries/StellaOps.Attestor.Infrastructure/Rekor/LocalTransparencyRekorClient.cs:18,188-194,391-408,435-452,460-501; src/Evidence/__Libraries/StellaOps.Attestor.Infrastructure/ServiceCollectionExtensions.cs:111-120; src/Evidence/__Libraries/StellaOps.EvidenceLocker.Infrastructure/Signing/Rfc3161TimestampAuthorityClient.cs:22,39-48; .../Hosting/EvidenceLockerRuntimeConfigurationValidator.cs:224-237.

6. Tenancy: the replica, residency and lifecycle

The consolidated host resolves tenant slugs from its own replica of Authority’s tenants catalog, held in stellaops_evidence. It does not read Platform’s shared.tenants, and it refuses to fall back to one:

Consolidated Evidence tenant resolution requires Catalog:Replication:Tenants:Enabled=true. The target owns no shared.tenants fallback and refuses to register the predecessor’s cross-database resolver.

The dual-mode registration’s generic argument is a fail-closed tripwire, not a usable fallback: the type it names throws. CatalogReplicaUnavailableException is deliberately not caught — an unknown or inactive tenant is a measured null, but an undrained replica stays a loud fault.

Residency works the same way: the carried default-region resolver is removed and replaced by one over a setup-owned value that is resolved fail-closed at startup, so the installation owns the default and no developer-selected fallback exists. Requests carrying an explicit region still override it.

Tenant lifecycle (SCR-4) registers EvidenceTenantLifecycleHandler with a legal-hold override keyed on evidence_locker.evidence_holds; the drain is opt-in and report-only until both Catalog:Replication:TenantLifecycle:Enabled and :DryRun say otherwise. Its eleven tests run as a non-superuser role, so row-level security actually evaluates.

Verified-by: src/Evidence/StellaOps.Evidence.WebService/Tenancy/EvidenceTenantCatalog.cs:24,65-132,72-79,122-123,141-153; src/Evidence/StellaOps.Evidence.WebService/Program.cs:59-60,124-126,266-269,331-338; src/Evidence/__Tests/StellaOps.Evidence.WebService.Tests/EvidenceTenantLifecycleHandlerTests.cs:160-325.

Export tenant isolation

Export creation resolves the authenticated tenant through the same tenant catalog used by the owner APIs. That UUID travels through bundle metadata, archive construction, job status and download. Storage uses a tenant-scoped connection under the constrained Evidence role; it never discovers a bundle owner by querying an RLS table without tenant context. A bundle or export job belonging to another tenant is unavailable. Archive SHA-256 is the format-defined digest computed through the shared Stella Ops cryptography library.

Native snapshot export uses the existing package contract: manifest.json, signature.json, bundle.json, checksums.txt and instructions.txt. It preserves the original signed manifest and detached signature. Snapshot materials are signed references; original material bytes absent from the snapshot are not synthesized. An empty artifact-table projection is not a snapshot export.

The package writer honors the immutable locator already sealed into its owning bundle. Both storage implementations validate the exact tenant, bundle and canonical artifact name and require write-once storage. The complete native locator is used verbatim; the configured S3-compatible material prefix continues to apply to generated content-addressed material keys. Actual digest metadata is retained. No sealed row is rewritten. Export jobs copy the cached native package to separate job-owned output directories, so requests for the same bundle cannot overwrite each other’s output.

Regression coverage uses the non-superuser evidence_app role, real filesystem storage and existing Stella Ops purpose-bound signature verification, including payload tampering and cross-tenant refusal. Source validation and a subsequent live export are tracked separately during the rollout.

Capsule public trust before the first seal

The CapsuleSeal JWKS lookup resolves its configured purpose key before the first signing operation. The owner key service delegates profile, algorithm and provider selection to Stella Ops cryptography. A provider-held key takes precedence; when that key is absent, the shared configured-key codec loads only the existing installation material. This read never generates an ephemeral key or creates a snapshot. Missing material or a missing explicitly selected provider returns 503. A subsequent seal uses the same key identity. The existing rotation source continues to publish its public roots and statuses; public JWK operations are restricted to verification.

Source verification: CapsuleSigningKeyEndpointTests covers cold Ed25519 and EC lookup, identity before and after a real seal, provider-held custody, missing configuration, explicit provider failure, and rotation roots. Live cold-start verification is recorded separately from those tests.

7. Doctor checks

evidence-web registers five family checks plus the SDK’s three standard ones, served at GET /doctor/evidence/checks behind ops.health. The worker deliberately adopts none: it is a replica role over the same database, so the web role already answers for the family.

All five are registered by concrete implementation type through TryAddEnumerable; the interface-only overload would de-duplicate four of them away. The explicit ServiceDoctorContext is registered before the standard checks, because an implicit bind produces a false green (“no database configured”) or binds the wrong pool.

Registration with the Platform doctor registry is default-off and fails closed on a blank Authority, client id or client secret — compose passes unset variables through as empty strings, so “missing” and “blank” had to be the same failure.

Verified-by: src/Evidence/StellaOps.Evidence.WebService/Doctor/EvidenceDoctorChecks.cs:89,153,253,344,449,572-613; src/Evidence/StellaOps.Evidence.WebService/Program.cs:200-215,217-234,274-329,380; src/Evidence/__Tests/StellaOps.Evidence.WebService.Tests/EvidenceDoctorAdoptionTests.cs.

The worker exposes /health/liveness and /health/readiness through the shared worker health library. Readiness verifies host startup; it does not claim that every queue or schedule has processed work. These endpoints are local operational probes and do not register a second Doctor surface.

8. Compose home and the infrastructure members

The two roles are defined in devops/compose/docker-compose.evidence.yml, with the reviewed immutable digests pinned in the committed devops/compose/docker-compose.local-evidence.yml. The pin lives in a committed file on purpose: a container whose recorded compose chain names a tmp/ path cannot be recreated once that directory is swept.

Two members of the family keep their own compose keys in the canonical docker-compose.stella-services.yml and are family members by ownership, not by key:

The canonical wrapper merges infrastructure, application services and docker-compose.evidence.yml in one include.path list. Compose merges the shared network declaration before importing the application model, so the default stack and generated release bundle both contain evidence-web and evidence-worker. The standalone family declaration remains available for isolated rehearsals. The compliance lane imports this canonical model once and then applies its disposable-estate override.

Re-verify: docker compose --env-file NUL -f devops/compose/docker-compose.stella-ops.yml config --no-interpolate --services (use /dev/null on Unix), then bash tools/scripts/validate/check-bundle-generated.sh. The topology does not imply a live deployment or a successful compliance journey.

Carried volumes, with their physical names, which is what makes the carry real rather than nominal: compose_evidence-data at /data/evidence, compose_attestor-proofchain-cas at /app/data/proofchain-cas, compose_tsa-data at /var/lib/tsa.

The service base creates /data/evidence with the application UID/GID before a fresh named volume is initialized. The 2026-09-12 native-export preflight found the older live volume empty and owned by root (0755), which prevented UID10001 from storing packages. Its scoped repair changes only the empty volume root owner to10001:10001; fresh-volume ownership is covered by an actual image probe.

Verified-by: devops/compose/docker-compose.evidence.yml; devops/compose/docker-compose.local-evidence.yml; devops/compose/docker-compose.stella-services.yml; EvidenceConsolidationComposeConformanceTests (17/17).

9. Predecessors

The three source schemas the data was copied from are still intact in stellaops_platform. Dropping them is a separate destructive approval tracked on EVD-10b, gated on the restore-tested full dump the runbook requires.

10. Known gaps, measured

Each of these is measured, not suspected, and each belongs to a named row rather than to this document.

1. Capsule signing is verified on the live host as of 2026-09-12. The public JWKS returns 200. Snapshot 17f789a5-62db-4d54-a29a-5b4c48b27d0d carries a 64-byte Ed25519 signature from bouncycastle.ed25519 with purpose-configured key evidence-locker-capsule-key. Stella Ops cryptography verifies the retained payload against that public key and refuses a one-byte tamper. The initial provider key lookup failure is followed by configured-key loading; it does not mean that snapshot signing failed. This supersedes the 2026-09-05 observation.
2. Twenty-six carried attestations verify ok:false on any configuration. Twenty-three are demo fixture rows for a transparency log that does not exist, and three were signed by a dev key retired for having been committed. Making them pass would mean trusting a retired key. The Console’s “TAMPERED” label on them is correct; the label’s wording is a UI defect worth its own row.
3. Application namespace retirement is source-complete as of 2026-09-12. The carried types, their direct references, default namespaces and runtime type lookup use the consolidated names described in §1. Route inventory, authorization and serialization acceptance are checked separately from CLR naming. No applied SQL, schema, assembly name or HTTP route is renamed.
4. Timestamp Assurance is composed in both roles; runtime assurance remains conditional on the installed provider and trust inputs.
5. Six cross-program caller repoints remain open, tracked in caller-repoints.tsv and owned by sprints 007/010/017/018/025/026.

11. Related documents

• attestor/consolidation-design.md— the consolidation design and the direct-HTTP-caller census. It stays at that path because EvidenceDirectHttpCallerConformanceTests parses it there.
• attestor/README.mdand evidence-locker/README.md— predecessor pointer stubs.
• ../../runbooks/evidence/evidence-evd9-data-move.md— the cutover runbook, including the isolated scratch rehearsal.
• ../../architecture/database-ownership-matrix.md— the ownership tiebreaker.
• ../signer/README.md— the separate key-custody service.