WebService Test Discipline

Audience: backend engineers and test-automation owners adding or maintaining tests for a Stella Ops WebService.

This document defines the testing discipline for Stella Ops WebService projects. Every web service must follow these patterns to deliver consistent test coverage, contract stability, telemetry verification, and security hardening.

See the WebService Test Rollout Plan for the order in which services adopt this discipline.

Overview

WebService tests use WebServiceFixture<TProgram> from StellaOps.TestKit and WebApplicationFactory<TProgram> from Microsoft.AspNetCore.Mvc.Testing. Tests are organized into four categories:

1. Contract Tests — OpenAPI schema stability
2. OTel Trace Tests — Telemetry verification
3. Negative Tests — Error handling validation
4. Auth/AuthZ Tests — Security boundary enforcement

1. Test Infrastructure

WebServiceFixture Pattern

using StellaOps.TestKit.Fixtures;

public class ScannerWebServiceTests : WebServiceTestBase<ScannerProgram>
{
    public ScannerWebServiceTests() : base(new WebServiceFixture<ScannerProgram>())
    {
    }
    
    // Tests inherit shared fixture setup
}

Fixture Configuration

Each web service should have a dedicated fixture class that configures test-specific settings:

public sealed class ScannerTestFixture : WebServiceFixture<ScannerProgram>
{
    protected override void ConfigureTestServices(IServiceCollection services)
    {
        // Replace external dependencies with test doubles
        services.AddSingleton<IStorageClient, InMemoryStorageClient>();
        services.AddSingleton<IQueueClient, InMemoryQueueClient>();
    }
    
    protected override void ConfigureTestConfiguration(IDictionary<string, string?> config)
    {
        config["scanner:storage:driver"] = "inmemory";
        config["scanner:events:enabled"] = "false";
    }
}

2. Contract Tests

Contract tests ensure OpenAPI schema stability and detect breaking changes.

Pattern

[Fact]
[Trait("Lane", "Contract")]
public async Task OpenApi_Schema_MatchesSnapshot()
{
    // Arrange
    using var client = Fixture.CreateClient();
    
    // Act
    var response = await client.GetAsync("/swagger/v1/swagger.json");
    var schema = await response.Content.ReadAsStringAsync();
    
    // Assert
    await ContractTestHelper.AssertSchemaMatchesSnapshot(schema, "scanner-v1");
}

[Fact]
[Trait("Lane", "Contract")]
public async Task Api_Response_MatchesContract()
{
    // Arrange
    using var client = Fixture.CreateClient();
    var request = new ScanRequest { /* test data */ };
    
    // Act
    var response = await client.PostAsJsonAsync("/api/v1/scans", request);
    var result = await response.Content.ReadFromJsonAsync<ScanResponse>();
    
    // Assert
    ContractTestHelper.AssertResponseMatchesSchema(result, "ScanResponse");
}

Snapshot Management

• Snapshots stored in Snapshots/ directory relative to test project
• Schema format: <service>-<version>.json
• Update snapshots intentionally when breaking changes are approved

3. OTel Trace Tests

OTel tests verify that telemetry spans are emitted correctly with required tags.

Pattern

[Fact]
[Trait("Lane", "Integration")]
public async Task ScanEndpoint_EmitsOtelTrace()
{
    // Arrange
    using var otelCapture = Fixture.CaptureOtelTraces();
    using var client = Fixture.CreateClient();
    var request = new ScanRequest { ImageRef = "nginx:1.25" };
    
    // Act
    await client.PostAsJsonAsync("/api/v1/scans", request);
    
    // Assert
    otelCapture.AssertHasSpan("scanner.scan");
    otelCapture.AssertHasTag("scanner.scan", "scan.image_ref", "nginx:1.25");
    otelCapture.AssertHasTag("scanner.scan", "tenant.id", ExpectedTenantId);
}

Required Tags

All WebService endpoints must emit these tags:

Service-specific tags are documented in each module’s architecture doc.

4. Negative Tests

Negative tests verify proper error handling for invalid inputs.

Pattern

[Fact]
[Trait("Lane", "Security")]
public async Task MalformedContentType_Returns415()
{
    // Arrange
    using var client = Fixture.CreateClient();
    var content = new StringContent("{}", Encoding.UTF8, "text/plain");
    
    // Act
    var response = await client.PostAsync("/api/v1/scans", content);
    
    // Assert
    Assert.Equal(HttpStatusCode.UnsupportedMediaType, response.StatusCode);
}

[Fact]
[Trait("Lane", "Security")]
public async Task OversizedPayload_Returns413()
{
    // Arrange
    using var client = Fixture.CreateClient();
    var payload = new string('x', 10_000_001); // Exceeds 10MB limit
    var content = new StringContent(payload, Encoding.UTF8, "application/json");
    
    // Act
    var response = await client.PostAsync("/api/v1/scans", content);
    
    // Assert
    Assert.Equal(HttpStatusCode.RequestEntityTooLarge, response.StatusCode);
}

[Fact]
[Trait("Lane", "Unit")]
public async Task MethodMismatch_Returns405()
{
    // Arrange
    using var client = Fixture.CreateClient();
    
    // Act (POST endpoint, but using GET)
    var response = await client.GetAsync("/api/v1/scans");
    
    // Assert
    Assert.Equal(HttpStatusCode.MethodNotAllowed, response.StatusCode);
}

Required Coverage

5. Auth/AuthZ Tests

Auth tests verify security boundaries and tenant isolation.

Pattern

[Fact]
[Trait("Lane", "Security")]
public async Task AnonymousRequest_Returns401()
{
    // Arrange
    using var client = Fixture.CreateClient(); // No auth
    
    // Act
    var response = await client.GetAsync("/api/v1/scans");
    
    // Assert
    Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode);
}

[Fact]
[Trait("Lane", "Security")]
public async Task ExpiredToken_Returns401()
{
    // Arrange
    using var client = Fixture.CreateAuthenticatedClient(tokenExpired: true);
    
    // Act
    var response = await client.GetAsync("/api/v1/scans");
    
    // Assert
    Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode);
}

[Fact]
[Trait("Lane", "Security")]
public async Task TenantIsolation_CannotAccessOtherTenantData()
{
    // Arrange
    using var tenantAClient = Fixture.CreateTenantClient("tenant-a");
    using var tenantBClient = Fixture.CreateTenantClient("tenant-b");
    
    // Create scan as tenant A
    var scanResponse = await tenantAClient.PostAsJsonAsync("/api/v1/scans", new ScanRequest { /* */ });
    var scan = await scanResponse.Content.ReadFromJsonAsync<ScanResponse>();
    
    // Act: Try to access as tenant B
    var response = await tenantBClient.GetAsync($"/api/v1/scans/{scan!.Id}");
    
    // Assert
    Assert.Equal(HttpStatusCode.NotFound, response.StatusCode); // Tenant isolation
}

Required Coverage

6. Test Organization

Directory Structure

src/<Module>/__Tests/StellaOps.<Module>.WebService.Tests/
├── StellaOps.<Module>.WebService.Tests.csproj
├── <Module>ApplicationFactory.cs       # WebApplicationFactory implementation
├── <Module>TestFixture.cs              # Shared test fixture
├── Contract/
│   └── OpenApiSchemaTests.cs
├── Telemetry/
│   └── OtelTraceTests.cs
├── Negative/
│   ├── ContentTypeTests.cs
│   ├── PayloadLimitTests.cs
│   └── MethodMismatchTests.cs
├── Auth/
│   ├── AuthenticationTests.cs
│   ├── AuthorizationTests.cs
│   └── TenantIsolationTests.cs
└── Snapshots/
    └── <module>-v1.json                # OpenAPI schema snapshot

Test Trait Assignment

7. CI Integration

WebService tests run in the appropriate CI lanes. The lane workflow shown below is currently archived under .gitea/workflows-archived/test-lanes.yml and is not wired into the active gate today; treat the example as the intended structure until the workflow is restored.

# .gitea/workflows-archived/test-lanes.yml
jobs:
  contract-tests:
    steps:
      - run: ./scripts/test-lane.sh Contract
  
  security-tests:
    steps:
      - run: ./scripts/test-lane.sh Security
  
  integration-tests:
    steps:
      - run: ./scripts/test-lane.sh Integration

All lanes are PR-gating. Failed tests block merge.

8. Rollout Checklist

When adding WebService tests to a new module:

• [ ] Create <Module>ApplicationFactory extending WebApplicationFactory<TProgram>
• [ ] Create <Module>TestFixture extending WebServiceFixture<TProgram> if needed
• [ ] Add contract tests with OpenAPI schema snapshot
• [ ] Add OTel trace tests for key endpoints
• [ ] Add negative tests (content type, payload, method)
• [ ] Add auth/authz tests (anonymous, expired, tenant isolation)
• [ ] Verify all tests have appropriate [Trait("Lane", "...")] attributes
• [ ] Run locally: dotnet test --filter "Lane=Contract|Lane=Security|Lane=Integration"
• [ ] Verify CI passes on PR

Known build breaks (2026-05-02)

StellaOps.Concelier.WebService.Tests currently fails to build because StellaOps.ReleaseOrchestrator.Plugin.Context.ReleaseOrchestratorPluginContext (src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.Plugin/Context/ReleaseOrchestratorPluginContext.cs:10) has not been updated to implement two members added to IPluginContext (src/Plugin/StellaOps.Plugin.Abstractions/Context/IPluginContext.cs:31,37):

• IFilesystemPolicy Filesystem { get; }
• IProcessSpawnPolicy ProcessSpawn { get; }

This is a build error in a transitive dependency, not a test failure. The test runner reports exit code 1 because the project never compiled; treating it as an OOM or test bug is wrong. See sprint SPRINT_20260501_003_Plugin_sandbox_trusted_level for the interface change that introduced the break. Until the impl lands, WebService.Tests verification on Concelier is blocked at compile.

References

• WebServiceFixture implementation
• ContractTestHelper implementation
• WebServiceTestBase implementation
• Test-lanes CI workflow (archived): .gitea/workflows-archived/test-lanes.yml
• CI lane filters documentation
• WebService test rollout plan

Last updated: 2026-05-02 · Sprint 5100.0007.0006