Audience: guild leads and engineers planning per-module test work for the 2026 H1 Testing Master Plan. Purpose: a visual map of test-model requirements per module, the quality gates each module must satisfy, and the sprint-to-model relationships that drive the program.
- ✅ Required (from TEST_CATALOG.yml)
- 🟡 Optional (recommended but not mandatory)
- ⬜ Not Applicable
| Model | Description | Key Tests |
|---|
| L0 | Library/Core | Unit, property, snapshot, determinism |
| S1 | Storage/Postgres | Integration, migrations, idempotency, query ordering |
| T1 | Transport/Queue | Protocol roundtrip, fuzz invalid, delivery semantics, backpressure |
| C1 | Connector/External | Fixtures, snapshot, resilience, security |
| W1 | WebService/API | Contract, authz, OTel, negative |
| WK1 | Worker/Indexer | End-to-end, retries, idempotency, OTel |
| AN1 | Analyzer/SourceGen | Diagnostics, codefixes, golden generated |
| CLI1 | Tool/CLI | Exit codes, golden output, determinism |
| PERF | Benchmarks | Benchmark, perf smoke, regression thresholds |
| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|---|
| Scanner | ✅ | ✅ | ✅ | ⬜ | ✅ | ✅ | ✅ | ⬜ | ✅ | 5100.0009.0001 | 25 |
| Concelier | ✅ | ✅ | ⬜ | ✅ | ✅ | ⬜ | ✅ | ⬜ | ⬜ | 5100.0009.0002 | 18 |
| Excititor | ✅ | ✅ | ⬜ | ✅ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | 5100.0009.0003 | 21 |
| Policy | ✅ | ✅ | ⬜ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0004 | 15 |
| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|---|
| Authority | ✅ | ⬜ | ⬜ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0005 | 17 |
| Signer | ✅ | ⬜ | ⬜ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0006 | 17 |
| Attestor | ✅ | ⬜ | ⬜ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0007 | 14 |
| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|---|
| Scheduler | ✅ | ✅ | ⬜ | ⬜ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | 5100.0009.0008 | 14 |
| Notify | ✅ | ✅ | ⬜ | ✅ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | 5100.0009.0009 | 18 |
| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|---|
| CLI | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ | ✅ | ⬜ | 5100.0009.0010 | 13 |
| UI | ⬜ | ⬜ | ⬜ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0011 | 13 |
| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|---|
| EvidenceLocker | ✅ | ✅ | ⬜ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0010.0001 | 16 |
| Graph/Timeline | ✅ | ✅ | ⬜ | ⬜ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | 5100.0010.0002 | 15 |
| Router/Messaging | ✅ | ✅ | ✅ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0010.0003 | 14 |
| AirGap | ✅ | ✅ | ⬜ | ⬜ | ✅ | ⬜ | ✅ | ✅ | ⬜ | 5100.0010.0004 | 17 |
| Model | Modules Using | Percentage | Complexity |
|---|
| L0 (Library/Core) | 13/15 modules | 87% | HIGH (property tests, snapshots) |
| W1 (WebService) | 13/15 modules | 87% | MEDIUM (contract tests, auth) |
| S1 (Storage) | 10/15 modules | 67% | HIGH (migrations, idempotency) |
| C1 (Connectors) | 5/15 modules | 33% | MEDIUM (fixtures, resilience) |
| WK1 (Workers) | 5/15 modules | 33% | MEDIUM (end-to-end, retries) |
| AN1 (Analyzers) | 3/15 modules | 20% | HIGH (Roslyn, diagnostics) |
| T1 (Transport) | 2/15 modules | 13% | HIGH (protocol compliance) |
| CLI1 (CLI Tools) | 2/15 modules | 13% | LOW (exit codes, snapshots) |
| PERF (Performance) | 1/15 modules | 7% | MEDIUM (benchmarks, regression) |
High Complexity (>15 tasks per sprint):
- Scanner (25 tasks: L0+AN1+S1+T1+W1+WK1+PERF)
- Excititor (21 tasks: C1+L0+S1+W1+WK1)
- Concelier (18 tasks: C1+L0+S1+W1+AN1)
- Notify (18 tasks: L0+C1+S1+W1+WK1)
- Authority (17 tasks: L0+W1+C1)
- Signer (17 tasks: L0+W1+C1)
- AirGap (17 tasks: L0+AN1+S1+W1+CLI1)
Medium Complexity (10-15 tasks):
- Policy (15 tasks: L0+S1+W1)
- EvidenceLocker (16 tasks: L0+S1+W1)
- Graph/Timeline (15 tasks: L0+S1+W1+WK1)
- Scheduler (14 tasks: L0+S1+W1+WK1)
- Attestor (14 tasks: L0+W1)
- Router/Messaging (14 tasks: L0+T1+W1+S1)
- CLI (13 tasks: CLI1)
- UI (13 tasks: W1)
| Module | Quality Gates | Enforced By |
|---|
| Scanner | determinism, reachability_evidence, proof_spine | Sprint 5100.0009.0001 Tasks 7-10, 23-25 |
| Concelier | fixture_coverage, normalization_determinism, no_lattice_dependency | Sprint 5100.0009.0002 Tasks 1-7, 8-10, 18 |
| Excititor | preserve_prune_source, format_snapshots, no_lattice_dependency | Sprint 5100.0009.0003 Tasks 6-11, 21 |
| Policy | unknown_budget, verdict_snapshot | Sprint 5100.0009.0004 Tasks 2, 4, 14-15 |
| Authority | scope_enforcement, sign_verify | Sprint 5100.0009.0005 Tasks 3-5, 16-17 |
| Signer | canonical_payloads, sign_verify | Sprint 5100.0009.0006 Tasks 1-3, 15-17 |
| Attestor | rekor_receipts, dsse_verify | Sprint 5100.0009.0007 Tasks 6-8, 2 |
| Scheduler | idempotent_jobs, retry_backoff | Sprint 5100.0009.0008 Tasks 4, 3, 12 |
| Notify | connector_snapshots, retry_semantics | Sprint 5100.0009.0009 Tasks 1-6, 16 |
| CLI | exit_codes, stdout_snapshots | Sprint 5100.0009.0010 Tasks 1-4, 5-8 |
| UI | contract_snapshots, e2e_smoke | Sprint 5100.0009.0011 Tasks 1-2, 7-10 |
| Gate | Applies To | Enforced By |
|---|
| Determinism Contract | Scanner, Excititor, Signer, CLI, AirGap, Concelier | Sprint 5100.0007.0003 (Determinism Gate) |
| Architecture Boundaries | Concelier, Excititor (must NOT reference Scanner lattice) | Sprint 5100.0007.0007 (Architecture Tests) |
| Contract Stability | All WebServices (13 modules) | Sprint 5100.0007.0006 (WebService Contract) |
| Storage Idempotency | All S1 modules (10 modules) | Sprint 5100.0007.0004 (Storage Harness) |
| Connector Resilience | All C1 modules (5 modules) | Sprint 5100.0007.0005 (Connector Fixtures) |
| CI Lane | Models | Modules | Sprint Tasks | Est. Runtime |
|---|
| Unit | L0, AN1, CLI1 | All 15 modules | ~120 tasks | <5 min |
| Contract | W1 | 13 modules | ~50 tasks | <2 min |
| Integration | S1, WK1, T1 | 12 modules | ~100 tasks | 10-15 min |
| Security | C1 (security tests), W1 (auth tests) | 5 connectors + 13 WebServices | ~60 tasks | 5-10 min |
| Performance | PERF | Scanner only | ~3 tasks | 3-5 min |
| Live | C1 (live smoke tests) | Concelier, Excititor, Notify, Authority, Signer | ~5 tasks (opt-in) | 5-10 min (nightly) |
PR Gate (Must Pass):
├─ Unit Lane (L0, AN1, CLI1) ← Fast feedback
├─ Contract Lane (W1) ← API stability
├─ Architecture Lane (Sprint 5100.0007.0007) ← Boundary enforcement
└─ Integration Lane (S1, WK1, T1) ← Testcontainers
Merge Gate (Must Pass):
├─ All PR Gate lanes
├─ Security Lane (C1 security, W1 auth)
└─ Determinism Lane (Sprint 5100.0007.0003)
Nightly (Optional):
├─ Performance Lane (PERF)
└─ Live Lane (C1 live smoke)
Weekly (Optional):
└─ Competitor Parity (Sprint 5100.0008.0001)
| Epic Sprint | Models Enabled | Consuming Modules | Tasks |
|---|
| 5100.0007.0002 (TestKit) | ALL (L0, S1, T1, C1, W1, WK1, AN1, CLI1, PERF) | ALL 15 modules | 13 |
| 5100.0007.0003 (Determinism) | L0 (determinism), CLI1 (determinism) | Scanner, Excititor, Signer, CLI, AirGap, Concelier | 12 |
| 5100.0007.0004 (Storage) | S1 | 10 modules | 12 |
| 5100.0007.0005 (Connectors) | C1 | Concelier, Excititor, Authority, Signer, Notify | 12 |
| 5100.0007.0006 (WebService) | W1 | 13 modules | 12 |
| 5100.0007.0007 (Architecture) | (Cross-cutting) | Concelier, Excititor | 17 |
| Test Category | Model Coverage | Estimated Test Count | CI Lane |
|---|
| Unit | L0, AN1 | ~150 tests across 13 modules | Unit |
| Property | L0 (subset) | ~40 tests (Scanner, Policy, Scheduler, Router) | Unit |
| Snapshot | L0, C1, CLI1 | ~80 tests (all modules with canonical outputs) | Unit/Contract |
| Integration | S1, WK1, T1 | ~120 tests across 12 modules | Integration |
| Contract | W1 | ~50 tests (13 WebServices × avg 4 endpoints) | Contract |
| Security | C1 (security), W1 (auth) | ~60 tests | Security |
| Performance | PERF | ~3 tests (Scanner only) | Performance |
| Live | C1 (live smoke) | ~5 tests (opt-in, nightly) | Live |
Performance Testing: Only Scanner has PERF model
- Recommendation: Add PERF to Policy (policy evaluation latency), Concelier (merge performance), Scheduler (scheduling overhead)
Transport Testing: Only Router/Messaging has T1 model
- Recommendation: Scanner has T1 in TEST_CATALOG.yml but should validate Valkey transport for job queues
Live Connector Tests: Only 5 modules have C1 live smoke tests (opt-in)
- Recommendation: Run weekly, not nightly; treat as early warning system for schema drift
| Module | Missing Model | Justification | Priority |
|---|
| Policy | PERF | Policy evaluation latency critical for real-time decisioning | HIGH |
| Concelier | PERF | Merge performance affects ingestion throughput | MEDIUM |
| Scheduler | PERF | Scheduling overhead affects job execution latency | MEDIUM |
| Scanner | T1 (validate) | Job queue transport (Valkey) should have compliance tests | HIGH |
| Authority | S1 | Token storage/revocation should have migration tests | MEDIUM |
Total Test Models: 9 Total Modules Covered: 15 Total Module Test Sprints: 15 (11 module + 4 infrastructure) Total Epic Sprints: 6 Total Quality Gate Sprints: 1 (Competitor Parity)
Model Usage:
- L0: 13 modules (87%)
- W1: 13 modules (87%)
- S1: 10 modules (67%)
- C1: 5 modules (33%)
- WK1: 5 modules (33%)
- AN1: 3 modules (20%)
- T1: 2 modules (13%)
- CLI1: 2 modules (13%)
- PERF: 1 module (7%)
Estimated Total Tests: ~500 tests across all modules and models
The following dimensions track adoption of Turn #6 testing practices across modules:
| Dimension | Description | Target Coverage |
|---|
| Intent Tags | Tests with [Intent] attribute declaring regulatory/safety/performance/competitive/operational | 100% non-trivial tests in Policy, Authority, Signer, Attestor |
| Observability Contracts | W1 tests with OTel schema validation, log field contracts | 100% of W1 tests |
| Evidence Traceability | Tests with [Requirement] attribute linking to requirements | 100% of regulatory-tagged tests |
| Longevity Tests | Memory stability, counter drift, connection pool tests | Scanner, Scheduler, Notify workers |
| Interop Tests | N-1/N+1 version compatibility tests | EvidenceLocker, Policy (schema-dependent) |
| Environment Skew | Tests across infrastructure profiles (network latency, resource limits) | Integration tests |
| Module | Intent Tags | Observability | Evidence | Longevity | Interop | Skew |
|---|
| Policy | ✅ Pilot | 🟡 | ✅ Pilot | 🟡 | 🟡 | ⬜ |
| EvidenceLocker | 🟡 | 🟡 | ✅ Pilot | 🟡 | ✅ | 🟡 |
| Scanner | 🟡 | ✅ Pilot | 🟡 | ✅ | 🟡 | 🟡 |
| Authority | 🟡 | 🟡 | 🟡 | ⬜ | 🟡 | ⬜ |
| Signer | 🟡 | 🟡 | 🟡 | ⬜ | 🟡 | ⬜ |
| Attestor | 🟡 | 🟡 | 🟡 | ⬜ | 🟡 | ⬜ |
| Scheduler | 🟡 | 🟡 | 🟡 | ✅ | ⬜ | 🟡 |
| Notify | 🟡 | 🟡 | 🟡 | ✅ | ⬜ | ⬜ |
Legend:
- ✅ Pilot implementation complete
- 🟡 Recommended, not yet implemented
- ⬜ Not applicable
| Component | Location | Purpose | Status |
|---|
IntentAttribute | TestKit/Traits/IntentAttribute.cs | Tag tests with intent | ✅ Complete |
IntentAnalyzer | TestKit.Analyzers/IntentAnalyzer.cs | Detect missing intent tags | ✅ Complete |
OTelContractAssert | TestKit/Observability/OTelContractAssert.cs | Span/attribute validation | ✅ Complete |
LogContractAssert | TestKit/Observability/LogContractAssert.cs | Log field validation | ✅ Complete |
MetricsContractAssert | TestKit/Observability/MetricsContractAssert.cs | Cardinality bounds | ✅ Complete |
RequirementAttribute | TestKit/Evidence/RequirementAttribute.cs | Link tests to requirements | ✅ Complete |
EvidenceChainAssert | TestKit/Evidence/EvidenceChainAssert.cs | Hash/immutability validation | ✅ Complete |
EvidenceChainReporter | TestKit/Evidence/EvidenceChainReporter.cs | Traceability matrix | ✅ Complete |
IncidentTestGenerator | TestKit/Incident/IncidentTestGenerator.cs | Post-incident test scaffolds | ✅ Complete |
SchemaVersionMatrix | TestKit/Interop/SchemaVersionMatrix.cs | Version compatibility | ✅ Complete |
VersionCompatibilityFixture | TestKit/Interop/VersionCompatibilityFixture.cs | N-1/N+1 testing | ✅ Complete |
StabilityMetrics | TestKit/Longevity/StabilityMetrics.cs | Memory/counter tracking | ✅ Complete |
StabilityTestRunner | TestKit/Longevity/StabilityTestRunner.cs | Time-extended tests | ✅ Complete |
EnvironmentProfile | TestKit/Environment/EnvironmentProfile.cs | Infrastructure profiles | ✅ Complete |
SkewTestRunner | TestKit/Environment/SkewTestRunner.cs | Cross-profile testing | ✅ Complete |
New categories added to TestCategories.cs:
| Category | Filter | CI Lane | Gating |
|---|
PostIncident | Category=PostIncident | Release | P1/P2 block |
EvidenceChain | Category=EvidenceChain | Merge | Block |
Longevity | Category=Longevity | Nightly | Warning |
Interop | Category=Interop | Release | Block |
EnvironmentSkew | Category=EnvironmentSkew | Nightly | Warning |
| Dimension | Current Baseline | Target | Tracking |
|---|
| Intent Tags (Policy, Authority, Signer, Attestor) | 5 tests | 100% non-trivial | IntentCoverageReport |
| Observability Contracts (W1 tests) | 5 tests | 100% | OTelContractAssert usage |
| Evidence Traceability (Regulatory tests) | 3 tests | 100% | EvidenceChainReporter |
| Longevity Tests (Worker modules) | 0 tests | 1 per worker | StabilityTestRunner usage |
| Interop Tests (Schema modules) | 0 tests | 1 per schema | SchemaVersionMatrix usage |
Prepared by: Project Management Date: 2026-01-27 Next review: 2026-02-03 (Turn #6 adoption review) Sources: the test manifest (TEST_MANIFEST.yml, planning-era name TEST_CATALOG.yml), sprint files 5100.0009.* and 5100.0010.*, and SPRINT_0127_002_DOCS_testing_enhancements_turn6.md.
