Mutation Testing Guide

Audience: Stella Ops developers and test automation engineers hardening critical modules.

This guide explains how to run and interpret Stryker.NET mutation testing in the Stella Ops monorepo, and how to close the test gaps it reveals.

Overview

Mutation testing measures test-suite effectiveness by introducing small code changes (mutants) and verifying that tests detect them. Unlike line coverage, mutation testing answers the question that actually matters: “Would my tests catch this bug?”

Per-module baseline and target scores are tracked separately in Mutation Testing Baselines.

Installation

Stryker.NET is configured as a local dotnet tool:

# Restore tools (includes Stryker.NET)
dotnet tool restore

# Verify installation
dotnet stryker --version

Configuration

Solution-Level Configuration

Base configuration is at stryker-config.json in the solution root. Module-specific configs override these settings.

Module Configurations

Running Mutation Tests

Single Module

# Navigate to module directory
cd src/Scanner/__Libraries/StellaOps.Scanner.Core

# Run mutation testing
dotnet stryker

# With specific config
dotnet stryker --config-file stryker-config.json

All Configured Modules

# From solution root
dotnet stryker --solution StellaOps.Router.slnx

CI Mode (Threshold Enforcement)

# Fails if mutation score below threshold
dotnet stryker --break-at-score 60

Understanding Results

Mutation Score

Mutation Score = (Killed Mutants / Total Mutants) × 100

• Killed: Test failed when mutant was introduced (good!)
• Survived: Test passed with mutant present (test gap!)
• No Coverage: No test covered the mutated code
• Timeout: Test timed out (usually treated as killed)

Thresholds

Example Output

All mutants have been tested, and your mutation score has been calculated
╔═══════════════════════════════════════════════════════════════════════╗
║ Mutation Testing Report                                                ║
╠═══════════════════════════════════════════════════════════════════════╣
║ Mutants tested: 156                                                    ║
║ Mutants killed: 134                                                    ║
║ Mutants survived: 18                                                   ║
║ Mutants no coverage: 4                                                 ║
║ Mutation score: 85.90%                                                 ║
╚═══════════════════════════════════════════════════════════════════════╝

Common Mutators

Fixing Survived Mutants

1. Analyze the Report

Open the HTML report in .stryker/output/<module>/mutation-report.html.

2. Identify the Gap

Look at the survived mutant:

// Original
if (score >= threshold) { return "PASS"; }

// Mutant (survived!)
if (score > threshold) { return "PASS"; }

3. Add Missing Test

[Fact]
public void Should_Pass_When_Score_Equals_Threshold()
{
    var score = 60;
    var threshold = 60;
    
    var result = EvaluateScore(score, threshold);
    
    result.Should().Be("PASS"); // Now kills the >= to > mutant
}

Best Practices

1. Focus on Critical Modules First

Prioritize mutation testing for:

• Security-critical code (Authority, Signer)
• Business logic (Policy decisions, Scanner matching)
• Boundary conditions

2. Don’t Chase 100%

Some mutants are false positives or equivalent mutants. Aim for 80%+ on critical modules.

3. Use Baseline Mode

Enable baseline to only test changed files:

dotnet stryker --with-baseline:main

4. Exclude Non-Critical Code

Exclude from mutation testing:

• DTOs and models
• Generated code
• Migrations
• UI components

CI Integration

Mutation testing runs in CI:

mutation-test:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - name: Run Stryker
      run: |
        dotnet tool restore
        dotnet stryker --break-at-score 60

Troubleshooting

Slow Execution

• Use --concurrency to control parallelism
• Enable coverage-analysis: perTest for smarter mutant selection
• Use --since:main to only test changed code

Out of Memory

• Reduce --concurrency value
• Exclude large test projects

Timeout Issues

• Adjust --timeout setting
• Some infinite loop mutants may timeout (this is expected)

References

• Mutation Testing Baselines - per-module baseline and target scores
• Test Suite Overview - how mutation testing fits the wider Stella Ops test strategy
• Stryker.NET Documentation
• Mutation Testing Theory