Durable execution
The core engine already survives restarts: every wait is a stored signal. Durable execution adds the features that a process spanning several services needs: a start that happens exactly once even when its message arrives twice, signals that race against a deadline, work handed to external handlers through a transactional outbox, bounded fan-out over collections, child-execution links and an execution history. This page explains when you need it and how to enable it.
When to use it
| You need | Use |
|---|---|
| Human tasks, service calls, timers, signals, child workflows | The core engine; no durable registration |
| To start workflows from a message queue, with each message starting at most one instance | Durable start |
| To wait for a reply from another system with a deadline, where exactly one of reply and deadline wins | Registered signals |
| To hand long-running work (minutes to days) to a worker service and resume when it reports back | External activities |
| To process every item of a collection with bounded concurrency | Maps |
Enabling it
Durable execution needs a store (PostgreSQL or MongoDB; both carry the durable tables), two registrations and a message transport of your own:
/// <summary>
/// Durable execution: durable start, registered signals, external activities, maps, child links and
/// history (AddWorkflowEngineDurableServices), and the lane handlers with the outbox dispatcher
/// (AddWorkflowDurableMessaging), which publishes through the host's message transport.
/// </summary>
public static IServiceCollection AddWorkflowDurableExecution<TPublisher>(this IServiceCollection services, IConfiguration configuration)
where TPublisher : class, IWorkflowOutboxMessagePublisher
{
services.AddWorkflowEngineDurableServices(configuration);
services.AddWorkflowDurableMessaging();
services.AddSingleton<IWorkflowOutboxMessagePublisher, TPublisher>();
return services;
}
AddWorkflowEngineDurableServices(configuration)adds durable start, registered signals, external activities, maps, child executions, the history, the maintenance service and the observability.AddWorkflowDurableMessaging()adds the handlers of the incoming messages and the outbox dispatcher with its hosted service.IWorkflowOutboxMessagePublisheris yours:PublishAsync(message)sends one outbox message on your transport (a queue, a topic) and returnstruewhen the transport confirmed it. An unconfirmed message stays in the outbox and is sent again.
Your message consumers feed the incoming lanes to the handlers and acknowledge each message by the disposition the handler returns.
Messages and lanes
Messages leave the engine only through the transactional outbox: the outbox rows are written in the same transaction as the state change that produces them, so no message is lost or sent for a change that was rolled back.
| Lane | Direction | Content |
|---|---|---|
workflow.start-execution.v1 | in | A start request |
workflow.start-execution-result.v1 | out | The result of a start |
workflow.root-continuation.v1 | in | A continuation of a root execution |
workflow.prestart-terminal-ack.v1 | out | The acknowledgement of a start that was ended before it arrived |
workflow.activity.execute.v1 | out | An external activity to run |
workflow.activity.completed.v1 | in | The completion of an external activity |
workflow.activity.cancel.v1 | out | The cancellation of an external activity |
Each incoming message gets a disposition, which your consumer maps to its acknowledgement:
| Disposition | Consumer action |
|---|---|
Applied, TerminalHandoffCommitted, AlreadyApplied, AlreadyTerminal, Late, Ignored | Acknowledge |
RetryableFailure, WaitNotReady | Deliver again later |
Rejected, Poison | Do not retry; move to your dead-letter queue |
The messages are canonical JSON (RFC 8785) with SHA-256 digests; the contracts and their JSON Schemas are in StellaOps.Workflow.Contracts.Durability.
Durable start
A workflow.start-execution.v1 message names the definition by code, version and SHA-256, and carries a start request id. The result is one of:
| Disposition | When |
|---|---|
STARTED | The start was admitted; the execution, its first history event, the receipt and the outbox messages were written together |
ALREADY_STARTED | A start with the same request id was recorded; the recorded result is returned. Different content under the same request id is a conflict |
REJECTED_NOT_CREATED | The admission policy refused the start (for example DEFINITION_NOT_PERMITTED), or an open execution of the definition holds the business key |
FENCED_NOT_CREATED | The caller ended the start before it arrived |
The default admission policy admits a start only when:
- the definition code is listed in
WorkflowDurableStart:Definitionswith the audience allowed to start it; - the store holds that version, with the requested content hash;
- the version is a managed definition (
originMANAGED_EMBEDDED, amanagedCodeequal to the definition code, and non-zerosourceBuildSha256andcompatibilitySha256) and is active (Managed import).
A missing, unmanaged or inactive definition is a transient refusal (DEFINITION_NOT_READY, DEFINITION_NOT_ACTIVE): the consumer delivers the message again later, without end, so a definition imported without the managed fields keeps its starts waiting. Replace the policy with your own IWorkflowDurableStartAdmissionPolicy.
{
"WorkflowDurableStart": {
"Definitions": { "OrderApproval": "order-service" }
}
}
Registered signals
A registered signal is a wait with an owner and a deadline. Exactly one of the signal and the deadline wins: the wait ends SIGNAL_WON or DEADLINE_WON. Raising a signal answers CORRELATED, ALREADY_RESOLVED, WAIT_NOT_READY, RETRYABLE_FAILURE or REJECTED.
{
"WorkflowRegisteredSignals": {
"ServiceSubjects": { "svc-payments": "payments" },
"Signals": { "payment.settled": "payments" }
}
}
ServiceSubjects maps an authenticated service subject to its audience; Signals maps a signal code to the only audience that may wait for it and raise it.
External activities
An external-activity step hands work to a registered handler and waits for its completion, with an optional deadline:
{ "$type": "external-activity", "stepName": "Generate invoice documents",
"destinationCode": "documents", "operationCode": "generate", "contractVersion": "1",
"requestExpression": { "$type": "path", "path": "state.invoice" },
"resultKey": "documents", "deadlineSeconds": 3600 }
The C# builder method is ExternalActivity(stepName, destinationCode, operationCode, contractVersion, requestExpression, resultKey, deadlineSeconds).
The registry. The engine accepts an external activity only when it is listed in a signed registry manifest: destination, operation, contract version, request and result schemas, the handler’s build digest, and its lanes. At start the engine canonicalises the manifest, compares its SHA-256 with ManifestSha256 and verifies the signature (RSA PKCS #1 v1.5, SHA-256) with the configured public key.
{
"WorkflowExternalActivityRegistry": {
"ManifestPath": "/etc/contoso/workflow/activities.json",
"ManifestSha256": "<sha-256 of the canonical manifest>",
"SignaturePath": "/etc/contoso/workflow/activities.sig",
"PublicKeyPath": "/etc/contoso/workflow/activities-public.pem"
}
}
Instead of the three files, the manifest, the signature and the key can be given inline: ManifestJson, SignatureBase64 and PublicKeyPem.
Without a manifest the registry is empty, and a definition with an external-activity step is rejected (WFVAL080); an activity not in the manifest is rejected (WFVAL081). A call-transport step that targets a registered external lane is rejected (WFVAL082): use call-transport for short, read-only, idempotent queries and external-activity for work.
Maps
A map step runs its body once per item, with bounded concurrency, and a deterministic fan-in (Defining workflows). The map coordinator writes an item and its dispatch in one transaction, so an item is never visible without its work, nor finished without the fan-in state that follows it. The host limits apply to every map: WorkflowRuntime:MaxMapConcurrency (default 8, at most 64) and WorkflowRuntime:MaxMapItems (default 512, at most 10000).
History and maintenance
The execution history is served in pages of 1 to 1000 events after a sequence number. Compaction removes old history, never for an execution under legal hold.
The maintenance service runs from your scheduler: it verifies integrity and compacts delivered outbox and inbox rows older than WorkflowMaintenance:CompactionRetentionDays (default 30, clamped to 7 to 3650).
Pitfalls
- Durable start of a definition that is only registered in code, or imported plainly. The admission policy requires a stored, managed, active version. Import it with the managed fields (Managed import) and activate it; the publisher cannot do this.
- A publisher that returns
truetoo early. Returntrueonly after your transport confirmed the message; otherwise a crash loses it. - Retrying
RejectedorPoison. These never succeed; dead-letter them. - External activities without a registry. The definitions are rejected at validation; provide the signed manifest first.
Related
- Persistence and signals
- Governed activation and publishing
- Configuration reference
- .NET API reference:
WorkflowDurableServiceCollectionExtensions,WorkflowDurableMessagingServiceCollectionExtensions,IWorkflowOutboxMessagePublisher,IWorkflowDurableStartAdmissionPolicyand theStellaOps.Workflow.Contracts.Durabilitypackage
