Test Project and Fixture Structure
Audience: contributors adding or relocating a Stella Ops test suite, and anyone maintaining the test consolidation manifest.
This document defines the on-disk layout that test consolidation expects new and migrated suites to follow: where a project lives, where its fixtures live, and how a change to either is routed to the suites that consume it. Following these conventions keeps suites manifest-routable and keeps fixture ownership unambiguous.
Related: Test Suite Consolidation Deep Dive (why these rules exist), Test Trait Taxonomy (how suites are classified), and
TEST_MANIFEST.yml(the routing manifest).
Project Placement
| Suite type | Preferred location | Notes |
|---|---|---|
| Module unit/integration tests | src/<Module>/__Tests/<Project>.Tests/ | Keep ownership local to the module. |
| Shared library tests | src/__Libraries/__Tests/<Project>.Tests/ | Use only for cross-module libraries. |
| Cross-cutting integration/E2E packs | src/__Tests/<Family>/<Project>/ | Must have explicit manifest routing and fixture ownership. |
| Production/operator Playwright | src/__Tests/playwright/** | Manual/nightly first; live stack requirements must stay explicit. |
| Frontend tests | src/Web/StellaOps.Web/** | Split unit, browser E2E, a11y, and live-stack lanes. |
| Docs samples/templates | docs/**/samples/** or docs/dev/sdks/plugin-templates/** | Sample-only trigger policy; never product-wide by default. |
Fixture Placement
| Fixture kind | Preferred location | Policy |
|---|---|---|
| Module fixtures | Adjacent Fixtures/, TestData/, Golden/, or Snapshots/ under the owning test project. | Manifest entry required. |
| Shared seed data | src/__Tests/__Datasets/** | Refresh command and deterministic policy required. |
| E2E analyzer corpus | src/__Tests/e2e/scanner-analyzer-corpus/fixtures/** | Committed seed corpus only; generated large audit corpus remains ignored. |
| Connector feed fixtures | src/Concelier/__Tests/**/Fixtures/** | Offline snapshots; live fetch is not a PR prerequisite. |
| Generated output | TestResults/, output/, or ignored module output directories. | Must not become source-of-truth fixtures. |
Fixture Manifest Files
Focused fixture manifests live under docs/technical/testing/fixtures/.
| Manifest | Scope | Current roots |
|---|---|---|
scanner-scm-corpus.fixture-manifest.yml | Scanner SCM vulnerable-library and call-path corpus | 1 |
concelier-connectors.fixture-manifest.yml | Concelier connector feed fixtures | 52 |
cross-cutting-tests.fixture-manifest.yml | src/__Tests/** fixtures and seed roots | 17 |
The files use JSON-compatible YAML for the same reason as TEST_MANIFEST.yml: validation must work with Python standard library only on Gitea runners.
Change Routing Rules
- A fixture-only change selects only suites that consume that fixture root.
- A docs sample/template change selects only sample/template suites.
- A generated corpus refresh must be deterministic and must not trigger product-wide suites unless shared code also changes.
- A live fixture refresh is manual/nightly evidence, not PR blocking, unless the manifest explicitly says otherwise.
Commands
python .gitea/scripts/test/refresh-fixture-manifests.py
python .gitea/scripts/test/validate-fixture-manifests.py \
--out TestResults/test-fixture-manifests/summary.json