CI Lane Integration Guide

Audience: engineers wiring Stella Ops .NET test projects into Gitea CI workflows, or adding a new test category lane.

This guide explains how to integrate Stella Ops’ standardized test-category filtering into CI workflows: how categories are declared on tests, how the canonical runner sweeps them, and how the live and archived workflows are structured.

Status note (reconciled against src/ and .gitea/). The mechanism this guide describes is the xUnit Category trait (see src/__Libraries/StellaOps.TestKit/TestCategories.cs), filtered with dotnet test --filter "Category=<name>". Earlier drafts of this guide referenced a Lane= filter, per-lane [UnitTest]/[IntegrationTest] attributes, a scripts/test-lane.sh helper, a scripts/ci/aggregate-test-results.sh aggregator, a .gitea/workflows/test-lanes.yml workflow, and a docs/technical/testing/TEST_CATALOG.yml catalog. None of those exist in the repo. They have been corrected below and the as-built reality documented:

  • The trait key is Category, not Lane.
  • Categories are applied with [Trait("Category", TestCategories.<Name>)] — there are no per-lane attribute types.
  • The canonical runner is .gitea/scripts/test/run-test-category.sh(not scripts/test-lane.sh).
  • The standalone test-lanes.yml workflow has been archived to .gitea/workflows-archived/test-lanes.yml; the live .NET test surface is exercised by .gitea/workflows/dotnet-pr-tests.yml (per-module, report-only) and .gitea/workflows/dotnet-nightly-sweep.yml (whole-repo sweep), with dedicated category lanes (benchmarks, conformance, determinism, etc.) in their own workflows.

Overview

StellaOps categorizes tests with the xUnit Category trait defined in src/__Libraries/StellaOps.TestKit/TestCategories.cs. The categories most relevant to CI filtering are:

TestCategories.cs defines many more categories (Property, Snapshot, Golden, Determinism, AirGap, E2E, Chaos, Resilience, Observability, storage-specific traits, distributed-systems traits, and a BlastRadius sub-trait set for incident-targeted runs). Use the constant that matches the test’s behavior rather than inventing a new string.

PR-gating reality. As of this reconciliation the live PR workflow (dotnet-pr-tests.yml) is report-only — every test job sets continue-on-error: true, so a failing test does not block a merge; the per-module summary.json artifacts carry the signal. The “PR gating ✅ Required” labels below describe the intended gating policy for each category, not the current enforcement. Treat them as aspirational until the workflow’s pull_request trigger and blocking behavior are restored.

Using Category Filters in CI

Using the Test Runner Script

The canonical runner is .gitea/scripts/test/run-test-category.sh, which sweeps the matching test projects and applies the Category=<name> trait filter. It takes the category as its first argument plus path/scope and reporting flags (run with no arguments to see the full option list):

- name: Run Unit category tests
  run: |
    chmod +x .gitea/scripts/test/run-test-category.sh
    .gitea/scripts/test/run-test-category.sh Unit \
      --results-dir ./TestResults

Caveat — MTP ignores --filter. xUnit v3 projects built on Microsoft.Testing.Platform (MTP) ignore the VSTest --filter flag, so a Category= trait clause is best-effort on those projects. The repo’s CI works around this with --no-trait-filter (run whole projects, selected --by-path) and, for projects that must filter, the MTP-native -- --filter-trait "Category=<name>" / --filter-class forms (see .gitea/workflows/plugin-linux-overlay-sandbox.yml). VSTest-based projects still honour --filter.

Direct dotnet test Filtering

For a single VSTest-based project, use dotnet test with a Category trait filter directly:

- name: Run Integration category tests
  run: |
    dotnet test \
      --filter "Category=Integration" \
      --configuration Release \
      --logger "trx;LogFileName=integration-tests.trx" \
      --results-directory ./test-results

For an MTP (xUnit v3) project, pass the filter after --:

- name: Run Unit category tests (MTP project)
  run: |
    dotnet test path/to/Project.csproj \
      --configuration Release \
      -- --filter-trait "Category=Unit"

Lane-Based Workflow Pattern

Reference Implementation (archived)

A per-lane reference workflow with separate jobs for Unit, Architecture, Contract, Integration, Security, Performance, and Live lanes exists at .gitea/workflows-archived/test-lanes.yml(it was authored under SPRINT 5100.0007.0001, then archived rather than activated). It is no longer wired into CI and depends on a scripts/test-lane.sh helper that was never committed — read it as a design sketch only.

The live .NET test workflows are:

Key features (as designed in the archived workflow):

Job Structure

The archived reference uses the shared runner label and the (uncommitted) helper script. The live workflows instead call run-test-category.sh directly and run on ubuntu-latest (non-infra jobs) or apparmor-host (Testcontainers/Npgsql/WebApplicationFactory jobs). An illustrative non-infra job:

unit-tests:
  name: Unit Tests
  runs-on: ${{ vars.LINUX_RUNNER_LABEL || 'ubuntu-latest' }}
  timeout-minutes: 15
  steps:
    - uses: actions/checkout@v4
    - name: Setup .NET
      uses: actions/setup-dotnet@v4
      with:
        global-json-file: global.json
    - name: Run Unit category
      run: |
        chmod +x .gitea/scripts/test/run-test-category.sh
        .gitea/scripts/test/run-test-category.sh Unit --results-dir ./TestResults
    - name: Upload results
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: unit-test-results
        path: ./TestResults

Lane Execution Guidelines

The “PR gating” rows describe the intended policy per category. See the PR-gating reality note in the Overview: the live PR workflow is currently report-only, so none of these categories block a merge today.

Unit Lane (Category=Unit)

Architecture Lane (Category=Architecture)

Contract Lane (Category=Contract)

Integration Lane (Category=Integration)

Security Lane (Category=Security)

Performance Lane (Category=Performance / Category=Benchmark)

Live Lane (Category=Live)

Per-Project vs Category-Based Execution

Note on the as-built model. The repo did not fully replace per-project execution with category lanes. The live PR workflow (dotnet-pr-tests.yml) is per-module — it maps changed files to affected src/<Module> directories and sweeps each one with run-test-category.sh --by-path "src/<Module>/*". Category-based lanes coexist as dedicated workflows (benchmarks, conformance, architecture, determinism, etc.) rather than as a single test-lanes.yml. This section contrasts the two execution styles; pick per the workflow you are editing.

Per-module sweep (the current PR pattern)

- name: Test affected module
  run: |
    chmod +x .gitea/scripts/test/run-test-category.sh
    .gitea/scripts/test/run-test-category.sh \
      --by-path "src/${m}/*" \
      --no-trait-filter --no-infra --report-only \
      --results-dir "TestResults/${m}"

Category-based sweep

- name: Run Unit category
  run: .gitea/scripts/test/run-test-category.sh Unit --results-dir ./TestResults

- name: Run Integration category
  run: .gitea/scripts/test/run-test-category.sh Integration --results-dir ./TestResults

- name: Run Security category
  run: .gitea/scripts/test/run-test-category.sh Security --results-dir ./TestResults

Benefits of category-based grouping:

Best Practices

1. Parallel Execution

Run PR-gating lanes in parallel for faster feedback:

jobs:
  unit-tests:
    # ...
  integration-tests:
    # ...
  security-tests:
    # ...

2. Conditional Execution

Use workflow inputs for optional lanes:

on:
  workflow_dispatch:
    inputs:
      run_performance:
        type: boolean
        default: false

jobs:
  performance-tests:
    if: github.event.inputs.run_performance == 'true'
    # ...

3. Test Result Aggregation

Create a summary job that depends on all category jobs. There is no committed aggregator script — the archived reference workflow generates the summary with inline shell over the downloaded artifacts, writing to $GITHUB_STEP_SUMMARY:

test-summary:
  needs: [unit-tests, architecture-tests, contract-tests, integration-tests, security-tests]
  if: always()
  steps:
    - name: Download all results
      uses: actions/download-artifact@v4
      with:
        path: all-test-results
    - name: Generate summary
      run: |
        echo "## Test Category Results" >> "$GITHUB_STEP_SUMMARY"
        for cat in unit architecture contract integration security; do
          if [ -d "all-test-results/${cat}-test-results" ]; then
            echo "### ${cat}: passed" >> "$GITHUB_STEP_SUMMARY"
          else
            echo "### ${cat}: failed or skipped" >> "$GITHUB_STEP_SUMMARY"
          fi
        done

For the report-only PR/sweep workflows, per-shard summary.json artifacts are turned into remediation sprints by .gitea/scripts/test/triage-sweep-results.sh rather than by an in-workflow aggregator.

4. Timeout Configuration

Set appropriate timeouts per lane:

unit-tests:
  timeout-minutes: 15  # Fast

integration-tests:
  timeout-minutes: 30  # Testcontainers startup

performance-tests:
  timeout-minutes: 45  # Benchmark execution

5. Environment Isolation

Use Testcontainers for the Integration category, not GitHub Actions service containers. In the live PR workflow the Testcontainers/Npgsql/WebApplicationFactory subset runs on the apparmor-host runner (Docker available), separate from the non-infra ubuntu-latest jobs:

integration-tests:
  runs-on: apparmor-host
  steps:
    - name: Run Integration tests
      env:
        POSTGRES_TEST_IMAGE: postgres:16-alpine
      run: .gitea/scripts/test/run-test-category.sh Integration --infra-only --results-dir ./TestResults

Testcontainers provides:

Troubleshooting

Tests Not Found

Problem: dotnet test --filter "Category=Unit" finds no tests

Solution: Ensure tests carry the Category trait, and remember that MTP (xUnit v3) projects ignore the VSTest --filter (use -- --filter-trait "Category=Unit" there, or sweep whole projects with run-test-category.sh --no-trait-filter):

using StellaOps.TestKit; // TestCategories

[Fact]
[Trait("Category", TestCategories.Unit)]  // This trait is required for Category=Unit filtering
public void MyTest() { }

There is no [UnitTest] / [IntegrationTest] attribute type in the codebase. Categorization is the xUnit [Trait("Category", ...)] attribute with constants from TestCategories. (StellaOps.TestKit.Traits does provide a separate [Intent(...)] attribute for the Intent trait — that is unrelated to category lanes.)

Wrong Category Assignment

Problem: An Integration test is tagged as Unit

Solution: Check the Category trait value:

// Bad: a database test tagged Unit
[Fact]
[Trait("Category", TestCategories.Unit)]
public async Task DatabaseTest() { /* uses Postgres */ }

// Good: use the Integration category for database tests
[Fact]
[Trait("Category", TestCategories.Integration)]
public async Task DatabaseTest() { /* uses Testcontainers */ }

Testcontainers Timeout

Problem: Integration tests timeout waiting for containers

Solution: Increase job timeout and ensure Docker is available:

integration-tests:
  timeout-minutes: 30  # Increased from 15
  steps:
    - name: Verify Docker
      run: docker info

Live Tests in PR

Problem: Live lane tests failing in PRs

Solution: Never run Live tests in PRs:

live-tests:
  if: github.event_name == 'workflow_dispatch' && github.event.inputs.run_live == 'true'
  # Never runs automatically on PR

Integration with Existing Workflows

The earlier build-test-deploy.yml workflow referenced here has been archived to .gitea/workflows-archived/build-test-deploy.yml. New work should plug into the live workflows (dotnet-pr-tests.yml, dotnet-nightly-sweep.yml) and the dedicated category workflows.

Prefer the shared run-test-category.sh runner over ad-hoc per-solution dotnet test invocations:

# Ad-hoc per-solution (avoid):
- name: Run Concelier tests
  run: dotnet test src/Concelier/StellaOps.Concelier.sln

# Shared runner (recommended) -- per-module sweep:
- name: Run Concelier module tests
  run: .gitea/scripts/test/run-test-category.sh --by-path "src/Concelier/*" --no-trait-filter --report-only --results-dir TestResults/Concelier

# Or a single category across the repo:
- name: Run all Unit tests
  run: .gitea/scripts/test/run-test-category.sh Unit --results-dir TestResults

Gradual Adoption Strategy

  1. Phase 1: Add [Trait("Category", TestCategories.<Name>)] traits to existing tests.
  2. Phase 2: Route module test execution through run-test-category.sh.
  3. Phase 3: Monitor the report-only PR/sweep workflows for stability.
  4. Phase 4: Restore the pull_request trigger and blocking behavior once the host-safe per-project build has a proven clean run (the workflow header in dotnet-pr-tests.yml tracks this).