Mutation Testing Baselines
Audience: engineers raising test quality on critical Stella Ops modules, and reviewers tracking mutation-score progress against targets.
Mutation testing (via Stryker.NET) measures test effectiveness: it injects small code changes (“mutants”) and checks whether the test suite catches them. A high score means the tests genuinely assert behavior rather than merely executing code. This document tracks the agreed baselines and targets for critical modules; for how to write and run the tests, see the Mutation Testing Guide.
Sprint: SPRINT_0353_0001_0001_mutation_testing_integration Task: MUT-0353-005
Baseline Scores
These are point-in-time baselines, not live scores — re-run Stryker for current numbers. Initial Score is the value recorded when the baseline was established; Target Score is the gate the module is working toward.
| Module | Initial Score | Target Score | Date Established |
|---|---|---|---|
| Scanner.Core | 72% | ≥ 80% | 2025-12-16 |
| Policy.Engine | 68% | ≥ 80% | 2025-12-16 |
| Authority.Core | 75% | ≥ 85% | 2025-12-16 |
| Signer.Core | 70% | ≥ 80% | TBD |
| Attestor.Core | 65% | ≥ 80% | TBD |
| Reachability.Core | 60% | ≥ 75% | TBD |
Threshold Configuration
See stryker-thresholds.json for per-module threshold configuration.
Mutation Operators Applied
| Operator | Description | Enabled |
|---|---|---|
| Arithmetic | Replace +, -, *, /, % | ✓ |
| Boolean | Flip true/false | ✓ |
| Comparison | Replace <, >, <=, >=, ==, != | ✓ |
| Logical | Replace &&, | |
| String | Mutate string literals | ✓ |
| Linq | Mutate LINQ methods | ✓ |
| NullCoalescing | Mutate ?? operators | ✓ |
| Assignment | Mutate assignment operators | ✓ |
Exclusions
The following patterns are excluded from mutation testing:
**/Migrations/**- Database migrations (tested via integration tests)**/Generated/**- Generated code**/*.g.cs- Source-generated files**/Models/**- Simple data transfer objects**/Exceptions/**- Exception types (tested via integration)
Running Mutation Tests
Local Execution
# Run mutation tests for a specific module
cd src/Scanner/__Libraries/StellaOps.Scanner.Core
dotnet stryker
# Run with specific configuration
dotnet stryker -f stryker-config.json --reporter html
# Quick mode (fewer mutations, faster feedback)
dotnet stryker --since:main
CI Execution
Mutation tests run on:
- Merge requests targeting main
- Weekly scheduled runs (comprehensive)
Results are uploaded as artifacts and published to the mutation testing dashboard.
Improving Mutation Score
- Add missing test cases - Cover edge cases revealed by surviving mutants
- Strengthen assertions - Replace weak assertions with specific ones
- Test boundary conditions - Cover off-by-one and boundary scenarios
- Add negative tests - Test that invalid inputs are rejected
References
- Mutation Testing Guide — writing and running mutation tests
- CI Quality Gates — how gate thresholds enforce in CI
- Testing Strategy Models
- Stryker.NET Documentation