Cross-Cutting Testing Standards Guide

This guide is for engineers writing tests anywhere in the Stella Ops monorepo. It documents the four cross-cutting standards that apply across modules — blast-radius annotations, schema-evolution testing, dead-path detection, and config-diff testing — what each one is for, how to author tests against it, and how it is enforced.

Sprint reference: SPRINT_20260105_002_005_TEST_cross_cutting (archived).

Table of Contents

1. Overview
2. Blast-Radius Annotations
3. Schema Evolution Testing
4. Dead-Path Detection
5. Config-Diff Testing
6. CI Workflows
7. Best Practices

Overview

Cross-cutting testing standards ensure consistent test quality across all modules:

Blast-Radius Annotations

Purpose

Blast-radius annotations categorize tests by the operational surfaces they affect. During incidents, this enables targeted test runs for specific areas (e.g., run only Auth-related tests when investigating an authentication issue).

Categories

Usage

using StellaOps.TestKit;
using Xunit;

// Single blast-radius
[Trait("Category", TestCategories.Integration)]
[Trait("BlastRadius", TestCategories.BlastRadius.Auth)]
public class TokenValidationTests
{
    [Fact]
    public async Task ValidToken_ReturnsSuccess()
    {
        // Test implementation
    }
}

// Multiple blast-radii (affects multiple surfaces)
[Trait("Category", TestCategories.Integration)]
[Trait("BlastRadius", TestCategories.BlastRadius.Auth)]
[Trait("BlastRadius", TestCategories.BlastRadius.Api)]
public class AuthenticatedApiTests
{
    // Tests that affect both Auth and Api surfaces
}

Requirements

• Integration tests: Must have at least one BlastRadius annotation
• Contract tests: Must have at least one BlastRadius annotation
• Security tests: Must have at least one BlastRadius annotation
• Unit tests: BlastRadius optional but recommended

Running Tests by Blast-Radius

# Run all Auth-related tests
dotnet test --filter "BlastRadius=Auth"

# Run tests for multiple surfaces
dotnet test --filter "BlastRadius=Auth|BlastRadius=Api"

# Run incident response test suite
dotnet run --project src/__Libraries/StellaOps.TestKit \
  -- run-blast-radius Auth,Api --fail-fast

Schema Evolution Testing

Purpose

Schema evolution tests verify that code remains compatible with previous database schema versions. This prevents breaking changes during:

• Rolling deployments (new code, old schema)
• Rollbacks (old code, new schema)
• Migration windows

Schema Versions

Using SchemaEvolutionTestBase

using StellaOps.Testing.SchemaEvolution;
using Testcontainers.PostgreSql;
using Xunit;

[Trait("Category", TestCategories.SchemaEvolution)]
public class ScannerSchemaEvolutionTests : PostgresSchemaEvolutionTestBase
{
    public ScannerSchemaEvolutionTests()
        : base(new SchemaEvolutionConfig
        {
            ModuleName = "Scanner",
            CurrentVersion = new SchemaVersion("v2.1.0",
                DateTimeOffset.Parse("2026-01-01")),
            PreviousVersions =
            [
                new SchemaVersion("v2.0.0",
                    DateTimeOffset.Parse("2025-10-01")),
                new SchemaVersion("v1.9.0",
                    DateTimeOffset.Parse("2025-07-01"))
            ],
            ConnectionStringTemplate =
                "Host={0};Port={1};Database={2};Username={3};Password={4}"
        })
    {
    }

    [Fact]
    public async Task ReadOperations_CompatibleWithPreviousSchema()
    {
        var result = await TestReadBackwardCompatibilityAsync(
            async (connection, version) =>
            {
                // Test read operations against old schema
                var repository = new ScanRepository(connection);
                var scans = await repository.GetRecentScansAsync(10);
                return scans.Count >= 0;
            });

        Assert.True(result.IsSuccess);
    }

    [Fact]
    public async Task WriteOperations_CompatibleWithPreviousSchema()
    {
        var result = await TestWriteForwardCompatibilityAsync(
            async (connection, version) =>
            {
                // Test write operations
                var repository = new ScanRepository(connection);
                await repository.CreateScanAsync(new ScanRequest { /* ... */ });
                return true;
            });

        Assert.True(result.IsSuccess);
    }
}

Versioned Container Images

Build versioned PostgreSQL images for testing:

# Build all versions for a module
./devops/docker/schema-versions/build-schema-images.sh scanner

# Build specific version
./devops/docker/schema-versions/build-schema-images.sh scanner v2.0.0

# Use in tests
docker run -d -p 5432:5432 ghcr.io/stellaops/schema-test:scanner-v2.0.0

Dead-Path Detection

Purpose

Dead-path detection identifies uncovered code branches. This helps:

• Find untested edge cases
• Identify potentially dead code
• Prevent coverage regression

How It Works

1. Tests run with branch coverage collection (Coverlet)
2. Cobertura XML report is parsed
3. Uncovered branches are identified
4. New dead paths are compared against baseline
5. CI fails if new dead paths are introduced

Baseline Management

The baseline file (dead-paths-baseline.json) tracks known dead paths:

{
  "version": "1.0.0",
  "activeDeadPaths": 42,
  "totalDeadPaths": 50,
  "exemptedPaths": 8,
  "entries": [
    {
      "file": "src/Scanner/Services/AnalyzerService.cs",
      "line": 128,
      "coverage": "1/2",
      "isExempt": false
    }
  ]
}

Exemptions

Add exemptions for intentionally untested code in coverage-exemptions.yaml:

exemptions:
  - path: "src/Authority/Emergency/BreakGlassHandler.cs:42"
    category: emergency
    justification: "Emergency access bypass - tested in incident drills"
    added: "2026-01-06"
    owner: "security-team"

  - path: "src/Scanner/Platform/WindowsRegistryScanner.cs:*"
    category: platform
    justification: "Windows-only code - CI runs on Linux"
    added: "2026-01-06"
    owner: "scanner-team"

ignore_patterns:
  - "*.Generated.cs"
  - "**/Migrations/*.cs"

Using BranchCoverageEnforcer

using StellaOps.Testing.Coverage;

var enforcer = new BranchCoverageEnforcer(new BranchCoverageConfig
{
    MinimumBranchCoverage = 80,
    FailOnNewDeadPaths = true,
    ExemptionFiles = ["coverage-exemptions.yaml"]
});

// Parse coverage report
var parser = new CoberturaParser();
var coverage = await parser.ParseFileAsync("coverage.cobertura.xml");

// Validate
var result = enforcer.Validate(coverage);
if (!result.IsValid)
{
    foreach (var violation in result.Violations)
    {
        Console.WriteLine($"Violation: {violation.File}:{violation.Line}");
    }
}

// Generate dead-path report
var report = enforcer.GenerateDeadPathReport(coverage);
Console.WriteLine($"Active dead paths: {report.ActiveDeadPaths}");

Config-Diff Testing

Purpose

Config-diff tests verify that configuration changes produce only expected behavioral deltas. This prevents:

• Unintended side effects from config changes
• Config options affecting unrelated behaviors
• Regressions in config handling

Using ConfigDiffTestBase

using StellaOps.Testing.ConfigDiff;
using Xunit;

[Trait("Category", TestCategories.ConfigDiff)]
public class ConcelierConfigDiffTests : ConfigDiffTestBase
{
    [Fact]
    public async Task ChangingCacheTimeout_OnlyAffectsCacheBehavior()
    {
        var baselineConfig = new ConcelierOptions
        {
            CacheTimeoutMinutes = 30,
            MaxConcurrentDownloads = 10
        };

        var changedConfig = baselineConfig with
        {
            CacheTimeoutMinutes = 60
        };

        var result = await TestConfigIsolationAsync(
            baselineConfig,
            changedConfig,
            changedSetting: "CacheTimeoutMinutes",
            unrelatedBehaviors:
            [
                async config => await GetDownloadBehavior(config),
                async config => await GetParseBehavior(config),
                async config => await GetMergeBehavior(config)
            ]);

        Assert.True(result.IsSuccess,
            $"Unexpected changes: {string.Join(", ", result.UnexpectedChanges)}");
    }

    [Fact]
    public async Task ChangingRetryPolicy_ProducesExpectedDelta()
    {
        var baseline = new ConcelierOptions { MaxRetries = 3 };
        var changed = new ConcelierOptions { MaxRetries = 5 };

        var expectedDelta = new ConfigDelta(
            ChangedBehaviors: ["RetryCount", "TotalRequestTime"],
            BehaviorDeltas:
            [
                new BehaviorDelta("RetryCount", "3", "5", null),
                new BehaviorDelta("TotalRequestTime", "increase", null,
                    "More retries = longer total time")
            ]);

        var result = await TestConfigBehavioralDeltaAsync(
            baseline,
            changed,
            getBehavior: async config => await CaptureRetryBehavior(config),
            computeDelta: ComputeBehaviorSnapshotDelta,
            expectedDelta: expectedDelta);

        Assert.True(result.IsSuccess);
    }
}

Behavior Snapshots

Capture behavior at specific configuration states:

var snapshot = CreateSnapshotBuilder("baseline-config")
    .AddBehavior("CacheHitRate", cacheMetrics.HitRate)
    .AddBehavior("ResponseTime", responseMetrics.P99)
    .AddBehavior("ErrorRate", errorMetrics.Rate)
    .WithCapturedAt(DateTimeOffset.UtcNow)
    .Build();

CI Workflows

Available Workflows

Status note: the dedicated workflow files below have been retired to .gitea/workflows-archived/. The cross-cutting checks they ran are folded into the active PR-gating lanes (see LOCAL_CI_GUIDE and test-architecture.yml). Treat the filenames here as the historical home of each check rather than a live trigger.

Workflow Outputs

Each workflow posts results as PR comments:

## Test Infrastructure :white_check_mark: All checks passed

| Check | Status | Details |
|-------|--------|---------|
| Blast-Radius | :white_check_mark: | 0 violations |
| Dead-Path Detection | :white_check_mark: | Coverage: 82.5% |
| Schema Evolution | :white_check_mark: | Compatible: N-1,N-2 |
| Config-Diff | :white_check_mark: | Tested: Concelier,Authority,Scanner |

Running Locally

# Blast-radius validation
dotnet test --filter "Category=Integration" | grep BlastRadius

# Dead-path detection
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura

# Schema evolution (requires Docker)
docker-compose -f devops/compose/schema-test.yml up -d
dotnet test --filter "Category=SchemaEvolution"

# Config-diff
dotnet test --filter "Category=ConfigDiff"

Best Practices

General Guidelines

1. Test categories: Always categorize tests correctly

   • Unit tests: Pure logic, no I/O
   • Integration tests: Database, network, external systems
   • Contract tests: API contracts, schemas
   • Security tests: Authentication, authorization, injection

2. Blast-radius: Choose the narrowest applicable category

   • If a test affects Auth only, use BlastRadius.Auth
   • If it affects Auth and Api, use both

3. Schema evolution: Test both read and write paths

   • Read compatibility: Old data readable by new code
   • Write compatibility: New code writes valid old-schema data

4. Dead-path exemptions: Document thoroughly

   • Include justification
   • Set owner and review date
   • Remove when no longer applicable

5. Config-diff: Focus on high-impact options

   • Security-related configs
   • Performance-related configs
   • Feature flags

Code Review Checklist

• [ ] Integration/Contract/Security tests have BlastRadius annotations
• [ ] Schema changes include evolution tests
• [ ] New branches have test coverage
• [ ] Config option tests verify isolation
• [ ] Exemptions have justifications

Troubleshooting

Blast-radius validation fails:

# Find tests missing BlastRadius
dotnet test --filter "Category=Integration" --list-tests | \
  xargs -I {} grep -L "BlastRadius" {}

Dead-path baseline drift:

# Regenerate baseline
dotnet test /p:CollectCoverage=true
python extract-dead-paths.py coverage.cobertura.xml
cp dead-paths-report.json dead-paths-baseline.json

Schema evolution test fails:

# Check schema version compatibility
docker run -it ghcr.io/stellaops/schema-test:scanner-v2.0.0 \
  psql -U stellaops_test -d stellaops_schema_test \
  -c "SELECT * FROM _schema_metadata;"

Related Documentation

• Test Infrastructure Overview
• Database Schema Specification
• Determinism Developer Guide
• Local CI Testing Guide
• CI Workflow Triggers
• Module Testing Agents