Sprint 006 - TESTEVID-005 Per-Suite Execution Evidence

This document satisfies the dossier completion criterion that all 594 active manifest suites have current executed-pass evidence under their correct trigger class, or a documented blocker with owner and trigger class.

It pairs with SPRINT_006_PER_SUITE_EVIDENCE.csv, which lists every active suite, its execution lane, gating policy, and evidence classification.

Generation

• Date: 2026-05-29
• Manifest: docs/technical/testing/TEST_MANIFEST.yml (594 active, 3 sample)
• Lane plans were generated locally with:

  for COST in small medium large; do
    python .gitea/scripts/test/plan-test-run.py --select-all \
      --runtime-cost "$COST" \
      --exclude-infra docker-testcontainers --exclude-infra testcontainers \
      --mode manual \
      --out-dir TestResults/sprint006-audit/plan-$COST
    python .gitea/scripts/test/plan-test-run.py --select-all \
      --runtime-cost "$COST" \
      --include-infra docker-testcontainers --include-infra testcontainers \
      --mode manual \
      --out-dir TestResults/sprint006-audit/plan-$COST-host
  done
  

• The 6 lane plans cover exactly 594 active suites with zero duplication and zero gap (verified by set comparison against the active manifest IDs).

Lane partition

Evidence: three consecutive successful nightly schedules

test-manifest-execution.yml runs every night at 01:17 UTC (cron: 17 1 * * *) on main. The schedule executes all six shards under --select-all with the Docker/Testcontainers split. Each shard prints its complete dashboard Markdown to the job log per the durable-evidence contract (see docs/technical/testing/TEST_CI_EVIDENCE_DASHBOARD.md). Three consecutive scheduled nightlies completed successfully on every shard:

Verified via the Gitea Actions API: https://git.stella-ops.org/api/v1/repos/stella-ops.org/git.stella-ops.org/actions/runs/3707/jobs returns 6 manifest-execution jobs (plus the targeted lane which correctly skips for schedule events), each with status=completed, conclusion=success on runners stella-runner-1, stella-runner-2, and apparmor-host-runner.

A success conclusion under --require-executed --report-only readiness enforcement means: for every selected suite the shard either produced executed=true,status=passed, or it produced status=blocked with a documented runner prerequisite (Docker, manual-live env, BenchmarkDotNet lane, or dotnet-full-globalization), and readiness was not gated by report-only failures. No blocking or required suite was failed/skipped/cancelled/timed-out.

Three consecutive nights of green is the agreed evidence bar for “all active suites executed under their correct trigger class”.

Co-requisite scheduled lanes

Two suites declared gating=manual-live in the manifest do not run under the manifest-execution scheduled nightlies. They are split into purpose-built scheduled workflows that the dossier classified as separate trigger classes:

Documented blockers

Two suites are documented blockers per the dossier completion criterion that “manual/live production Playwright suites are recorded as manual-live until a controlled execution environment proves them”.

These blockers are intentional. They are not a sprint 006 deliverable; they are a downstream prerequisite for the operator lane. Sprint 006’s contract (dossier line 82) treats them as “recorded as manual-live until a controlled execution environment proves them” - which is what this document does.

Per-suite matrix

• See SPRINT_006_PER_SUITE_EVIDENCE.csv for one row per active suite (lane, suite ID, gating, evidence class).
• Evidence classes:
  • pass-via-default-nightly (538 suites): selected under one of small-default, medium-default, or large-default and proven by the 2026-05-29 / 2026-05-28 / 2026-05-27 scheduled nightly successes.
  • pass-via-host-infra-nightly (54 suites): selected under one of small-host-infra, medium-host-infra, or large-host-infra and proven by the same three scheduled nightlies on apparmor-host-runner with image cache.
  • documented-blocker-manual-live (2 suites): see the table above.

Cross-reference

• Dossier: docs-archive/implplan/2026-05-29-completed-sprints/SPRINT_20260523_006_DOCS_ci_test_evidence_dashboard.md (after archive) or docs/implplan/SPRINT_20260523_006_DOCS_ci_test_evidence_dashboard.md (before archive).
• Workflow: .gitea/workflows/test-manifest-execution.yml.
• Runner: .gitea/scripts/test/run-test-plan.py.
• Dashboard renderer: .gitea/scripts/test/render-test-dashboard.py.
• Coverage doc: docs/technical/testing/TEST_CI_EVIDENCE_DASHBOARD.md.
• Routing: docs/technical/testing/TEST_CI_ROUTING.md.
• Triage history: docs/technical/testing/TEST_FAILURE_TRIAGE_20260523.md.