Test Trait Taxonomy

Audience: contributors tagging tests and maintainers wiring CI lane selection.

This is the authoritative catalog of Stella Ops test traits. It makes Category the primary CI selection key; Lane remains compatibility metadata during migration and must not be the source of truth for new routing. For how CI consumes these traits, see ci-lane-filters.md; for where suites and fixtures live, see TEST_PROJECT_AND_FIXTURE_STRUCTURE.md.

Canonical Categories

Canonical values are defined in src/__Libraries/StellaOps.TestKit/TestCategories.cs and mirrored in TEST_MANIFEST.schema.json. The canonical set is exactly the 43 public const string values in TestCategories.cs, because validate-test-traits.py extracts every public const string NAME = "VALUE"; declaration in that file. Thirty-three of those are top-level TestCategories constants; the remaining ten (Advisories, Api, Auth, Compliance, Crypto, Evidence, Integrations, Persistence, RiskPolicy, Scanning) are declared inside the nested TestCategories.BlastRadius class but are matched by the same regex, so they are accepted both as Category values and as BlastRadius values.

Allowed Category values (43 total):

Category	Primary use
`Unit`	Fast in-memory component tests.
`Advisories`	Advisory ingestion, normalization, and decision tests.
`Api`	HTTP API and OpenAPI surface checks.
`Auth`	Authentication, authorization, token, and tenant-boundary checks.
`Compliance`	Regulatory and compliance-pack behavior checks.
`Crypto`	Cryptographic provider, key, signature, and verification checks.
`Evidence`	Evidence locker, provenance, and attestation artifact checks.
`Integrations`	External connector, adapter, and plugin integration checks.
`Persistence`	Repository, storage, and database access checks.
`RiskPolicy`	Risk and policy decision logic checks.
`Scanning`	Scanner, analyzer, SBOM, and artifact ingestion checks.
`Property`	FsCheck/generative invariant tests.
`Snapshot`	Snapshot or golden-master regression tests.
`Integration`	Database, Testcontainers, WebApplicationFactory, or service integration tests.
`Contract`	API, schema, or compatibility contract tests.
`Security`	Cryptographic, vulnerability, or security assertion tests.
`Performance`	Load, latency, throughput, or SLO checks.
`Live`	Tests requiring live external services; disabled by default in PR CI.
`Architecture`	Dependency, layering, naming, and forbidden-reference checks.
`Golden`	Corpus-based expected-output regression tests.
`Benchmark`	BenchmarkDotNet or benchmark harness runs.
`AirGap`	Offline and air-gap behavior checks.
`E2E`	Full workflow, browser, API, or service-boundary tests.
`Chaos`	Fault injection and recovery tests.
`Determinism`	Reproducibility, stable ordering, and idempotency tests.
`Resilience`	Retry, timeout, circuit-breaker, and degraded-mode tests.
`Observability`	OpenTelemetry spans, metrics, structured logging, and diagnostics contracts.
`StorageConcurrency`	Storage locking and concurrent access tests.
`StorageIdempotency`	Storage duplicate/retry idempotency tests.
`QueryDeterminism`	Stable query ordering and repeated-query consistency tests.
`StorageMigration`	Schema migration and versioning tests.
`SchemaEvolution`	Backward/forward compatibility across schema versions.
`ConfigDiff`	Behavioral delta tests for configuration changes.
`HLC`	Hybrid logical clock and distributed ordering tests.
`Federation`	Multi-site synchronization or cross-region replication tests.
`Latency`	Network latency and cross-region timing tests.
`Immutability`	Artifact integrity, build reproducibility, and digest verification tests.
`Parity`	Competitor comparison or parity validation tests.
`PostIncident`	Production-incident-derived regression tests.
`EvidenceChain`	Requirement-to-test-to-artifact traceability tests.
`Longevity`	Sustained-run stability tests.
`Interop`	Cross-version or N-1/N+1 interoperability tests.
`EnvironmentSkew`	Varied infrastructure/profile skew tests.

Current manifest usage:

Category	Suite count	CI posture
`Unit`	384	PR/main candidate when targeted and static traits are present.
`Integration`	152	Targeted PR/main or nightly depending infra and stability.
`AirGap`	25	Nightly/manual unless explicitly lightweight and targeted.
`Benchmark`	25	Weekly/manual, not PR blocking.
`E2E`	7	Nightly/manual first; live E2E is never PR blocking.
`Chaos`	4	Nightly/manual only.

The manifest currently exercises only 6 of the 43 canonical categories across its 597 suites; the remaining values are reserved for suites that have not yet been classified into the manifest.

validate-test-traits.py fails if TEST_MANIFEST.yml uses a category not present in TestCategories.cs.

Trait Migration Rules

Every new .NET test must use [Trait("Category", TestCategories.<Value>)].
Existing .NET suites without static Category traits must appear in test-trait-migration.csv.
The manifest category is authoritative only while a suite has migration_status=manifest-exception.
P0 rows must get static traits before CI routing.
P1 rows must get static traits before promotion from report-only.
P2 rows must get static traits on the next meaningful module touch.

Current triage:

Triage	Count
Trait-missing .NET suites	146
P0 rows	8
P1 rows	78
P2 rows	60

Secondary Traits

Use secondary traits for intent and diagnostics, not primary CI routing:

Trait	Purpose
`Intent`	Why the behavior exists, using `TestIntents` (`src/__Libraries/StellaOps.TestKit/Traits/TestIntents.cs`).
`BlastRadius`	Operational surface affected by failure, using `TestCategories.BlastRadius`.
`Lane`	Temporary compatibility metadata during migration.

Intent values (TestIntents): Regulatory, Safety, Performance, Competitive, Operational.

BlastRadius values (TestCategories.BlastRadius): Auth, Scanning, Evidence, Compliance, Advisories, RiskPolicy, Crypto, Integrations, Persistence, Api.

Skip Triage

Static skips are allowed only when they are visible and owned.

Existing skip pockets must appear in test-skip-audit-triage.csv.
Before promotion, each static skip must carry a sprint/task reference, owner, reason, and expiry or removal path.
New static skips without a task reference should fail review and should not be added to blocking lanes.

Current skip triage:

Triage	Count
Skip-audit suites	10
Total static skipped tests represented by those suites	34

Commands

python .gitea/scripts/test/refresh-test-triage.py
python .gitea/scripts/test/validate-test-traits.py \
  --out TestResults/test-trait-validation/summary.json