Testing workflows

Test a workflow at three levels: the definition alone (fast, no database), the definition running in the engine against a real store (the behaviour), and the host over HTTP (the composition). This page shows each level with code from Tutorial 9.

LevelNeedsCatches
DefinitionNothingCompile errors, invalid definitions, a missing module, the wrong structure
RuntimePostgreSQL (a container)Wrong branches, wrong state, wrong calls, permission rules
HostThe host and PostgreSQLAuthentication, scopes, configuration, the actor claims

The examples use NUnit; the code works the same with xUnit or MSTest.

Definition tests

Compile the workflow, then validate the result with the engine’s validator. No store and no host are needed:

[Test]
public void TheDefinitionCompilesToAValidCanonicalDefinition()
{
    var compilation = WorkflowCanonicalDefinitionCompiler.Compile(new DocumentReviewWorkflow());

    Assert.That(compilation.Succeeded, Is.True, string.Join("; ", compilation.Diagnostics.Select(d => d.Message)));

    // The engine's validator: the schema, the expressions, and the modules the definition needs.
    using var services = new ServiceCollection()
        .AddLogging()
        .AddWorkflowEngineCoreServices(new ConfigurationBuilder().Build())
        .BuildServiceProvider();
    using var scope = services.CreateScope();
    var validation = scope.ServiceProvider.GetRequiredService<WorkflowCanonicalDefinitionService>()
        .Validate(new WorkflowCanonicalValidateRequest
        {
            CanonicalDefinitionJson = WorkflowCanonicalJsonSerializer.Serialize(compilation.Definition!),
        });

    var issues = validation.SchemaIssues.Concat(validation.SemanticIssues).Concat(validation.ModuleIssues);
    Assert.That(validation.IsValid, Is.True, string.Join("; ", issues.Select(issue => $"{issue.Path}: {issue.Message}")));
}

Assert on the structure of the compiled definition when a rule matters, for example who may work a task or where a decision leads:

[Test]
public void ChangesRequestedSendsTheDocumentBackToTheAuthor()
{
    var definition = WorkflowCanonicalDefinitionCompiler.Compile(new DocumentReviewWorkflow()).Definition!;

    var review = definition.Tasks.Single(task => task.TaskName == "Review document");
    var json = WorkflowCanonicalJsonSerializer.SerializeFragment(review.OnComplete);

    Assert.That(review.TaskRoles, Is.EqualTo(new[] { "reviewer" }));
    Assert.That(review.TimeoutSeconds, Is.EqualTo(259200));
    Assert.That(json, Does.Contain("\"Revise document\""));
}

A definition test also guards the canonical JSON of a workflow: serialize the compiled definition and compare it with a reviewed file, so that a change to the C# shows up as a change to the JSON.

Runtime tests

Run the workflow in the engine with the PostgreSQL store, in a disposable container (Testcontainers), and replace the transports with recording ones.

The test host

[OneTimeSetUp]
public async Task StartEngine()
{
    database = new PostgreSqlBuilder().WithImage("postgres:16-alpine").Build();
    try
    {
        await database.StartAsync();
    }
    catch (Exception exception) when (exception is not OperationCanceledException)
    {
        Assert.Ignore($"Docker is not available: {exception.Message}");
    }

    var builder = Host.CreateApplicationBuilder();
    builder.Configuration.AddInMemoryCollection(new Dictionary<string, string?>
    {
        ["STELLAOPS_POSTGRES_WORKFLOW_CONNECTION"] = database.GetConnectionString(),
        ["WorkflowBackend:Provider"] = "Postgres",
        ["WorkflowRuntime:DefaultProvider"] = "Stella.Engine",
    });
    builder.Services.AddWorkflowEngineCoreServices(builder.Configuration);
    builder.Services.AddWorkflowPostgresDataStore(builder.Configuration);
    // The signal pump resumes instances that wait for a child workflow, a timer or a signal.
    builder.Services.AddWorkflowEngineHostedServices();
    // The recording transport answers `endpoint` calls. A definition requires the module of each address
    // kind it uses; a transport package installs it, a test registers it with the stand-in.
    builder.Services.AddWorkflowModule("transport.endpoint", "1.0.0");
    builder.Services.AddScoped<IWorkflowEndpointTransport>(_ => transport);
    builder.Services.AddWorkflowRegistration<DocumentReviewWorkflow, DocumentReviewRequest>();
    builder.Services.AddWorkflowRegistration<SendInvoiceWorkflow, SendInvoiceRequest>();

    host = builder.Build();
    await host.StartAsync(); // applies the schema script, then starts the hosted services
}

Recording the calls

A recording transport implements the transport interface of one address kind, records each call and answers from scripted responses. Register it after the engine, in place of the HTTP transport:

/// <summary>
/// A test stand-in for the <c>endpoint</c> address kind: it records every call and answers from
/// scripted responses, keyed by service name and path. Register it in place of the HTTP transport.
/// </summary>
public sealed class RecordingEndpointTransport : IWorkflowEndpointTransport
{
    private readonly ConcurrentDictionary<string, WorkflowEndpointResponse> responses = new(StringComparer.OrdinalIgnoreCase);
    private readonly ConcurrentQueue<WorkflowEndpointRequest> calls = new();

    public IReadOnlyCollection<WorkflowEndpointRequest> Calls => calls.ToArray();

    public RecordingEndpointTransport Respond(string serviceName, string path, object? payload)
    {
        responses[Key(serviceName, path)] = new WorkflowEndpointResponse { Succeeded = true, Payload = payload };
        return this;
    }

    public RecordingEndpointTransport Fail(string serviceName, string path, long statusCode, string error)
    {
        responses[Key(serviceName, path)] = new WorkflowEndpointResponse
        {
            Succeeded = false,
            StatusCode = statusCode,
            Error = error,
        };
        return this;
    }

    public Task<WorkflowEndpointResponse> ExecuteAsync(
        WorkflowEndpointRequest request,
        CancellationToken cancellationToken = default)
    {
        calls.Enqueue(request);
        return Task.FromResult(responses.TryGetValue(Key(request.ServiceName, request.Path), out var response)
            ? response
            : new WorkflowEndpointResponse { Succeeded = true });
    }

    private static string Key(string serviceName, string path) => $"{serviceName} {path.TrimStart('/')}";
}

The interfaces are IWorkflowEndpointTransport, IWorkflowHttpTransport, IWorkflowGraphqlTransport and IWorkflowRabbitTransport. A scripted failure (Succeeded = false) runs the step’s failure branch.

A definition requires the module of each address kind it uses (transport.endpoint, transport.http, …). A transport package installs its module; a test that registers only a recording transport installs the module itself with AddWorkflowModule, as the test host above does. Without it, the host stops at start with WFIMP010.

A human task, end to end

[Test]
public async Task AChangeRequestOpensTheRevisionTaskAndTheRevisionReturnsToTheReviewer()
{
    using var scope = host!.Services.CreateScope();
    var runtime = scope.ServiceProvider.GetRequiredService<WorkflowRuntimeService>();

    var started = await runtime.StartWorkflowAsync(new StartWorkflowRequest
    {
        WorkflowName = "DocumentReview",
        Payload = new Dictionary<string, object?> { ["documentId"] = "DOC-7", ["title"] = "Travel policy", ["author"] = "author-1" },
    });
    Assert.That(started.NextTasks.Select(task => task.TaskName), Is.EqualTo(new[] { "Review document" }));

    var changes = await runtime.CompleteTaskAsync(new WorkflowTaskCompleteRequest
    {
        WorkflowTaskId = started.NextTasks.Single().WorkflowTaskId,
        ActorId = "reviewer-1",
        ActorRoles = ["reviewer"],
        Payload = new Dictionary<string, object?> { ["decision"] = "changes", ["comments"] = "Add the approval limits." },
    });
    Assert.That(changes.NextTasks.Select(task => task.TaskName), Is.EqualTo(new[] { "Revise document" }));

    var revised = await runtime.CompleteTaskAsync(new WorkflowTaskCompleteRequest
    {
        WorkflowTaskId = changes.NextTasks.Single().WorkflowTaskId,
        ActorId = "author-1",
        ActorRoles = ["author"],
    });
    Assert.That(revised.NextTasks.Select(task => task.TaskName), Is.EqualTo(new[] { "Review document" }));
    Assert.That(revised.WorkflowState["revision"]?.ToString(), Is.EqualTo("2"));
}

A service call

[Test]
public async Task TheInvoiceWorkflowCallsTheBillingService()
{
    using var scope = host!.Services.CreateScope();
    var runtime = scope.ServiceProvider.GetRequiredService<WorkflowRuntimeService>();

    var started = await runtime.StartWorkflowAsync(new StartWorkflowRequest
    {
        WorkflowName = "SendInvoice",
        Payload = new Dictionary<string, object?> { ["orderId"] = "ORD-42" },
    });

    var instance = await runtime.GetInstanceAsync(new WorkflowInstanceGetRequest
    {
        WorkflowInstanceId = started.WorkflowInstanceId,
    });
    Assert.That(instance.Instance.Status, Is.EqualTo("Completed"));
    Assert.That(transport.Calls.Any(call => call.ServiceName == "billing" && call.Path.EndsWith("invoices", StringComparison.Ordinal)), Is.True);
}

A permission rule

[Test]
public void ACallerWithoutTheTaskRoleCannotCompleteTheTask()
{
    Assert.ThrowsAsync(Is.InstanceOf<Exception>(), async () =>
    {
        using var scope = host!.Services.CreateScope();
        var runtime = scope.ServiceProvider.GetRequiredService<WorkflowRuntimeService>();
        var started = await runtime.StartWorkflowAsync(new StartWorkflowRequest
        {
            WorkflowName = "DocumentReview",
            Payload = new Dictionary<string, object?> { ["documentId"] = "DOC-8", ["title"] = "Leave policy", ["author"] = "author-1" },
        });

        await runtime.CompleteTaskAsync(new WorkflowTaskCompleteRequest
        {
            WorkflowTaskId = started.NextTasks.Single().WorkflowTaskId,
            ActorId = "author-1",
            ActorRoles = ["author"],
            Payload = new Dictionary<string, object?> { ["decision"] = "approve" },
        });
    });
}

Host tests

Test the composition of your host with WebApplicationFactory or the ASP.NET Core test server: a token without a scope gets 403, a start answers 200, a task action ignores the body’s roles. The host of Getting started is a host to test this way.

Cases

Pitfalls