Test Project and CI Inventory

Snapshot date: 2026-05-22 UTC

This report inventories the current automated test surface for consolidation planning. It is based only on repository-local files: project files, package manifests, test runner configs, existing testing docs, and active Gitea workflows under .gitea/workflows/.

The per-project appendix is generated as:

docs/technical/testing/test-project-ci-inventory.csv

The scenario-level appendix for browser E2E and Scanner SCM corpus coverage is:

docs/technical/testing/test-e2e-scenario-inventory.csv

The deeper consolidation inventory is:

docs/technical/testing/TEST_SUITE_CONSOLIDATION_DEEP_DIVE.md
docs/technical/testing/test-suite-layer-inventory.csv
docs/technical/testing/test-fixture-corpus-inventory.csv

Inventory Method

Sources reviewed:

.NET project files under src/** (all test trees consolidated under src/__Tests/** and src/<Module>/__Tests/** as of 2026-05-30) and runnable docs samples, excluding docs-archive/**, bin/**, obj/**, node_modules/**, dist/**, and generated docs-content/**.
Frontend runners in src/Web/StellaOps.Web/package.json, Vitest configs, and Playwright configs.
Live/operator Playwright runner in src/__Tests/playwright/.
Go module workflow and the two real Go module roots.
Active Gitea workflows and shared test scripts: .gitea/workflows/*.yml, .gitea/scripts/test/run-test-category.sh, and .gitea/scripts/test/detect-affected-modules.sh.

CSV columns:

Column	Meaning
`role`	Runnable test project, support library, fixture input, or non-.NET runner.
`module`	Top-level `src/<module>` owner, top-level `tests/<area>`, or docs/root classification.
`path`	Repository-relative project, package, or corpus path.
`runner`	Runner family (`dotnet`, `npm/vitest`, `npm/playwright`, `go test`, etc.).
`test_type`	Derived type from project/config naming and package references.
`infra`	Required runtime resources inferred from packages/configs.
`seed_data`	Nearby fixture, snapshot, golden, seed, or content resources.
`ci`	Direct workflow mapping plus generic sweep/fanout mapping where applicable.
`ci_triggers`	Trigger notes for mapped workflows.
`ci_where`	Workflow file/job or matched workflow lines.

Current Counts

Role	Count	Notes
`runnable-dotnet-test`	590	.NET/xUnit or MTP test projects and runnable cross-cutting packs.
`dotnet-test-support`	48	TestKit, fixture libraries, shared test infrastructure, benchmark/support projects.
`fixture-input`	19	.NET fixture projects used as analyzer/corpus inputs, not test runners.
`sample-template-test`	3	Runnable docs sample/template tests.
`frontend-runner`	1	`src/Web/StellaOps.Web` Vitest and Playwright runner.
`playwright-live-runner`	1	`src/__Tests/playwright` operator-trust live-stack runner.
`go-test-module`	2	Real Go modules covered by `go-tests.yml`.
`tooling-test-runner`	1	Root package scripts for docs/API/SDK smoke checks.
`fixture-corpus`	1	Scanner analyzer corpus under `src/__Tests/e2e/scanner-analyzer-corpus/fixtures`.
`qa-fixture-corpus`	1	QA feature-check fixtures under `docs/qa/feature-checks/fixtures`.
Total CSV rows	667	Full row-level appendix in the CSV.

Runnable .NET test projects by module are listed in the CSV. High-volume owners are Concelier (87), Scanner (79), shared __Libraries (78), __Tests cross-cutting packs (44), BinaryIndex (33), Attestor (28), ReleaseOrchestrator (27), and Notify (23).

Production E2E Scenario Inventory

The project-level CSV intentionally records runners and projects, not every browser spec or generated SCM scenario. For consolidation, the second appendix adds that scenario-level view:

Scenario role	Count	Runtime / CI notes
`production-playwright-spec`	16	`src/__Tests/playwright` operator-baseline/operator-trust specs; live-stack runner through `STELLA_BASE_URL`; no direct active workflow currently found.
`web-playwright-spec`	131	`src/Web/StellaOps.Web/tests/e2e/` and `src/Web/StellaOps.Web/e2e/`; split across source-served, docker/live-stack, integration, air-gap, and live-search configs.
`scanner-scm-corpus-seed`	216	Committed Scanner SCM seed fixtures under `src/__Tests/e2e/scanner-analyzer-corpus/fixtures/<language>/<scenario>/`.
`scanner-scm-generated-corpus-summary`	1	Summary row for the ignored/generated seed-42 comprehensive audit corpus.
Total scenario rows	364	Full row-level appendix in `test-e2e-scenario-inventory.csv`.

The Scanner SCM corpus is the surface that matches “set up SCM with repositories and check vulnerable-library call paths.” It is not Playwright. It is driven by StellaOps.E2E.SyntheticRepos, devops/scripts/e2e/provision-corpus.ps1, and StellaOps.E2E.AnalyzerCoverage, with CI in .gitea/workflows/e2e-analyzer-coverage.yml.

Current committed Scanner SCM seed counts:

Language	Seed scenarios	Call-path scenarios
bun	12	7
ccpp	15	10
dart	12	7
deno	12	7
dotnet	18	12
elixir	14	9
go	15	10
java	15	10
node	18	13
php	16	11
python	19	14
ruby	16	11
rust	22	16
swift	12	7
Total	216	144

The generated comprehensive audit corpus is intentionally ignored, but the documented seed-42 profile materializes 2,585 git repositories, 1,821 distinct coverage cases, 392 source scenarios, 870 call-path reachability repositories, and 319 secret-tagged repositories.

CI Integration Map

Surface	CI location	Trigger state	Runner / job	What it runs
Generic .NET sweep	`.gitea/workflows/dotnet-nightly-sweep.yml`	`workflow_dispatch` only; schedule currently disabled in comments	`apparmor-host`, job `sweep`, report-only	`.gitea/scripts/test/run-test-category.sh --no-trait-filter --report-only`; starts shared Postgres; supports scope/shard inputs.
.NET PR fanout	`.gitea/workflows/dotnet-pr-tests.yml`	`workflow_dispatch` only; PR trigger currently disabled in comments	`ubuntu-latest` non-infra and `apparmor-host` infra jobs, report-only	`detect-affected-modules.sh` maps changed files to `src/<Module>`; `run-test-category.sh --by-path src/<Module>/*`.
FE unit/spec drift	`.gitea/workflows/fe-web-tests.yml`	`pull_request` path-filtered	`ubuntu-latest`, job `drift-gate`	Runs `npx vitest run --config vitest.codex.config.ts` on HEAD and base; fails only for new failing spec files.
FE default Playwright	`.gitea/workflows/fe-e2e-tests.yml`	`workflow_dispatch` only	`apparmor-host`, report-only	Runs default `src/Web/StellaOps.Web/playwright.config.ts` against source-served `npm run serve:test`.
Air-gap federation	`.gitea/workflows/airgap-federation-e2e.yml`	`pull_request`, nightly schedule, manual	PR: `ubuntu-latest`; live: `apparmor-host`	Tier 0 .NET in-process tests, Tier 1 notification Playwright specs, Tier 2 live Docker e2e.
Go modules	`.gitea/workflows/go-tests.yml`	`pull_request`, `push`, daily schedule, manual	`ubuntu-latest`, report-only	`go test ./... -v -count=1` for JobEngine worker SDK and generated Attestor Go types.
OpenAPI conformance	`.gitea/workflows/openapi-conformance.yml`	`pull_request`, nightly schedule, manual	`ubuntu-latest`	Dedicated `src/__Tests/architecture/StellaOps.OpenApi.Conformance.Tests` project. Excluded from generic .NET sweep.
Architecture conformance	`.gitea/workflows/test-architecture.yml`	`pull_request`, `push main`, manual	configured Linux runner	Dedicated architecture contracts pack.
E2E analyzer coverage	`.gitea/workflows/e2e-analyzer-coverage.yml`	nightly schedule, manual, PR smoke	`ubuntu-latest`	Builds and runs `StellaOps.E2E.AnalyzerCoverage`; full Gitea stack path on scheduled/manual gitea target.
E2E notify coverage	`.gitea/workflows/e2e-notify-coverage.yml`	nightly schedule, manual, PR smoke	PR `ubuntu-latest`; full run `apparmor-host`	Builds Notify coverage harness and connector tests; full run uses Mailpit and echo receiver compose.
Chaos router	`.gitea/workflows/chaos-router.yml`	schedule, PR paths, manual	`apparmor-host`	Builds and runs `src/__Tests/chaos/StellaOps.Chaos.Router.Tests`.
Workflow renderer	`.gitea/workflows/workflow-renderer.yml`	PR paths, schedule/manual for explicit tests	`ubuntu-latest`	Fast and explicit `StellaOps.Workflow.Renderer.Tests` lanes.
Workflow Oracle/Redis	`.gitea/workflows/workflow-oracle-tests.yml`	PR paths, manual	`ubuntu-latest`	Oracle and Redis integration pack with service images.
Plugin sandbox	plugin `-sandbox` and `-confinement` workflows	Linux paths on PR/push/manual; Windows manual-only	Linux apparmor or Windows runners	Filtered `StellaOps.Plugin.Sandbox.Tests` categories/classes for overlay, AppArmor, filesystem, OS confinement.
Runtime instrumentation agents	`.gitea/workflows/runtime-instrumentation-host-agents.yml`	PR/push/manual	`ubuntu-latest`	Agent core/Linux/Tetragon test slices.
OpenAPI truthfulness and docs parity	`.gitea/workflows/openapi-truthfulness.yml`, `.gitea/workflows/docs-cli-parity.yml`	path-filtered PR/push/manual	`ubuntu-latest`	Scripted docs/API/CLI conformance checks, not general test projects.

Generic .NET Sweep Eligibility

run-test-category.sh does not discover every runnable .NET project. Its active discovery pattern is limited to:

*.Tests.csproj
*UnitTests.csproj
*SmokeTests.csproj
*FixtureTests.csproj
*IntegrationTests.csproj

It searches under src/, excludes *Testing.csproj and StellaOps.TestKit.csproj, and explicitly excludes the OpenAPI conformance project because that project has its own workflow.

Inventory impact:

Classification	Count
Runnable .NET projects detected	590
Eligible for generic `.NET Test Sweep`	567
Eligible for intended `.NET PR Tests` module fanout	461
Runnable .NET projects without direct active CI mapping	20

The uncovered runnable projects are mostly cross-cutting packs named without a *.Tests.csproj suffix:

src/__Tests/Integration/StellaOps.Integration.*/*.csproj
src/__Tests/e2e/*/*.csproj
src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/*.csproj
src/BinaryIndex/__Tests/StellaOps.BinaryIndex.Benchmarks/*.csproj
module-owned src/Doctor/__Tests/...

The module-owned src/AirGap/__Tests/StellaOps.AirGapFederation.IntegrationTests project has a dedicated workflow and is not part of the generic sweep.

Seed Data and Fixtures

Seed data is spread across several conventions:

Project-local Fixtures, TestData, Golden, Snapshots, __snapshots__, samples, and seed directories.
Embedded or copied resources declared in project files via EmbeddedResource or Content Include.
Analyzer fixture projects under src/Scanner/**/Fixtures/** and src/__Tests/e2e/scanner-analyzer-corpus/fixtures/**.
QA feature-check corpuses under docs/qa/feature-checks/fixtures/**.
Frontend Playwright evidence/output roots under src/Web/StellaOps.Web/output and src/__Tests/playwright/output.

The CSV records nearby seed-data hints per row, but this is a structural inventory, not a semantic fixture manifest. Consolidation should add a first class fixture manifest if fixture ownership, refresh policy, and offline determinism need to become enforceable.

Consolidation Findings

The generic .NET sweep is close to the current runnable .NET surface but not complete. It misses runnable projects that do not follow the discovery suffix pattern. The stale top-level tests/Telemetry duplicate has been consolidated into the module-owned Telemetry core tests.
The .NET PR fanout is currently not automatic. The workflow is workflow_dispatch only, and all jobs are report-only.
Cross-cutting src/__Tests/**, src/__Libraries/**, and src/__Analyzers/** changes are treated as broad changes by detect-affected-modules.sh; the PR lane intentionally skips fanout and relies on the sweep.
Test filtering is inconsistent. The sweep uses --no-trait-filter because the trait taxonomy is incomplete and xUnit v3/MTP projects may ignore VSTest --filter.
Frontend coverage is split: Vitest drift is the active PR gate, while default Playwright is manual report-only and live/docker Playwright configs are handled by specialized workflows or scripts.
Go CI is intentionally narrow and correct: only the two real Go modules are tested; Go modules inside analyzer/benchmark fixture corpuses are seed data.
Several support libraries own important fixture/Testcontainers behavior but are not test projects by themselves. They should stay in the inventory, but gating should happen through their consuming test projects.

Recommended Consolidation Moves

Create a versioned test manifest from this CSV and make CI discovery consume it instead of re-deriving project lists from filename patterns.
Decide whether cross-cutting src/__Tests/Integration/** and src/__Tests/e2e/** projects should be renamed to *.Tests.csproj or explicitly included in run-test-category.sh.
Add an ownership field for fixture corpuses: owning module, refresh command, offline status, expected determinism hash/snapshot policy, and CI consumers.
Split CI state into explicit blocking, report-only, and manual-only fields. The current workflow names can imply stronger coverage than the triggers actually provide.
Keep dedicated workflows for live, browser, Docker, hardware, and OS-confinement tests, but register them in the same manifest so consolidation work can see one inventory.