Persistence and signals
The engine keeps instances, tasks, task events, runtime state, signals, dead letters, stored definitions and the Designated Workflow ledger in one store. This page describes the two stores, PostgreSQL and MongoDB, the signal drivers that wake the signal pump, and running several nodes on one store.
Choosing a store
| PostgreSQL | MongoDB | |
|---|---|---|
| Package | StellaOps.Workflow.DataStore.PostgreSQL (in StellaOps.Workflow) | StellaOps.Workflow.DataStore.MongoDB |
WorkflowBackend:Provider | Postgres (the default) | Mongo |
| Connection | STELLAOPS_POSTGRES_WORKFLOW_CONNECTION | ConnectionStrings:WorkflowMongo |
| Schema | The workflow schema, created at start | The collections, created at first use |
| Native signal wake-up | LISTEN/NOTIFY | Change streams |
Register the store library after the engine. Each store library registers itself only when WorkflowBackend:Provider names it, so a host can register both and choose by configuration:
/// <summary>
/// Both store libraries are registered; WorkflowBackend:Provider ("Postgres" or "Mongo") decides which
/// one takes effect, so one build serves either store.
/// </summary>
public static IServiceCollection AddWorkflowStores(this IServiceCollection services, IConfiguration configuration)
{
services.AddWorkflowEngineCoreServices(configuration);
services.AddWorkflowPostgresDataStore(configuration);
services.AddWorkflowMongoDataStore(configuration);
services.ValidateWorkflowProviderRegistrations(configuration);
return services;
}
PostgreSQL
Connection
The store reads its connection string from one key, STELLAOPS_POSTGRES_WORKFLOW_CONNECTION, from any configuration source. Without it the host stops at start with an error that names the key; there is no fallback to ConnectionStrings.
STELLAOPS_POSTGRES_WORKFLOW_CONNECTION="Host=db.contoso.example;Port=5432;Database=workflow;Username=workflow_app;Password=<password>;SSL Mode=VerifyFull"
Pool settings are in WorkflowBackend:Postgres (MinPoolSize 1, MaxPoolSize 100, Pooling, ConnectionIdleLifetimeSeconds 300, ApplicationName stellaops-workflow).
Schema and permissions
At start, the store applies its schema script: it creates the workflow schema, its tables and a workflow.schema_migrations table that records the applied script. The schema name and the table names are fixed.
| Phase | The role needs |
|---|---|
| First start | CREATE on the database (to create the schema), or ownership of a workflow schema that an administrator created for it |
| Every start | Ownership of the workflow schema, so the script can check and extend it |
| Running | Read and write on the tables of the workflow schema |
To keep schema changes with your database administrator, turn the startup migration off and apply the script yourself:
/// <summary>The schema script is applied by the database administrator, not by the host.</summary>
public static IServiceCollection AddWorkflowStoreWithManagedSchema(this IServiceCollection services, IConfiguration configuration)
{
services.AddWorkflowEngineCoreServices(configuration);
services.AddWorkflowPostgresDataStore(configuration, options => options.ApplyMigrationsOnStartup = false);
return services;
}
The script is embedded in the StellaOps.Workflow.Persistence assembly as StellaOps.Workflow.Persistence.Migrations.001_workflow_schema.sql. It is idempotent (CREATE ... IF NOT EXISTS). Apply the script of the package version you install before you start that version.
Tables
| Class | Tables | Lifetime |
|---|---|---|
| Operational state | Instances, tasks, task events, runtime states, and the durable-execution tables | Purged by the retention job after completion (Operations) |
| Source of truth | Definitions, definition audit, signal queue, dead letters, wake outbox, host locks, the first-seen ledger | Changed only by the operation the row describes; never deleted by age |
The first-seen ledger (wf_first_seen_workflows) is insert-only: a trigger refuses updates and deletes.
MongoDB
{
"WorkflowBackend": {
"Provider": "Mongo",
"Mongo": { "DatabaseName": "workflow" }
},
"ConnectionStrings": {
"WorkflowMongo": "mongodb://workflow-app:<password>@mongo-1.contoso.example,mongo-2.contoso.example/?replicaSet=rs0&tls=true"
}
}
WorkflowBackend:Mongo:ConnectionStringName(defaultWorkflowMongo) names the connection string. A missing connection string fails when the client is first used.WorkflowBackend:Mongo:DatabaseNamedefaults tostellaops_workflow_store. Set it explicitly.- Every collection name is an option of
WorkflowBackend:Mongo(InstancesCollectionNameand so on), withworkflow_*defaults (Configuration reference). - Run MongoDB as a replica set: the store writes an instance, its tasks and its signals in one transaction, and the native signal driver reads change streams. A standalone server supports neither.
Signals and the signal pump
A step that waits leaves a signal in the store: a timer, a task deadline, an external signal, the end of a child workflow, or the start of a ContinueWith workflow. The signal pump (AddWorkflowEngineHostedServices) claims due signals and resumes their instances. Timers, external signals and task deadlines run on the Stella.Engine runtime provider, the default (Hosting).
| Setting | Default | Effect |
|---|---|---|
WorkflowEngine:MaxConcurrentSignalHandlers | processor count | Pump workers per node |
WorkflowAq:MaxDeliveryAttempts | 10 | Attempts before a signal is dead-lettered |
WorkflowBackend:Postgres:BlockingWaitSeconds | 30 | The longest wait of a worker for a wake-up before it checks the store again |
WorkflowBackend:Postgres:ClaimTimeoutSeconds | 60 | The claim lease; a claim not completed within it becomes claimable again |
WorkflowBackend:Mongo:BlockingWaitSeconds, ClaimTimeoutSeconds | 30, 60 | The same two settings for the MongoDB store |
A signal is processed at least once. A signal whose wait is gone, a duplicate for example, is ignored, so a duplicate never applies twice; a signal that is applied again runs the steps after its wait again, service calls included. A signal whose processing keeps failing is moved to the dead letters after MaxDeliveryAttempts (Operations). The store keeps the delivery count with the signal and raises it at each claim, so the limit counts every delivery on every node, before and after a restart; a replayed dead letter starts again from zero.
Signal drivers
The signal driver wakes the pump workers when a signal becomes due, so that they do not wait for BlockingWaitSeconds.
WorkflowSignalDriver:Provider | Wake-up | Use |
|---|---|---|
Native (default) | PostgreSQL LISTEN/NOTIFY, or MongoDB change streams | One store, any number of nodes |
Redis | A Redis channel | Many nodes when the database should not carry the notifications |
With Redis, the signals stay in the store; only the wake-up travels through Redis. The host provides the Redis connection, and AddWorkflowRedisSignaling registers everything else:
/// <summary>
/// Redis carries the wake-up notifications between nodes (WorkflowSignalDriver:Provider = "Redis"); the
/// signals themselves stay in the store. The host provides the Redis connection; the driver announces
/// each stored signal on the Redis channel once the store's transaction commits.
/// </summary>
public static IServiceCollection AddWorkflowRedisWakeUps(this IServiceCollection services, IConfiguration configuration)
{
services.AddSingleton<IConnectionMultiplexer>(_ =>
ConnectionMultiplexer.Connect(configuration["Redis:Configuration"]
?? throw new InvalidOperationException("Set Redis:Configuration.")));
services.AddWorkflowRedisSignaling(configuration);
return services;
}
{
"WorkflowSignalDriver": {
"Provider": "Redis",
"Redis": { "ChannelName": "contoso:workflow:signals", "BlockingWaitSeconds": 30 }
},
"Redis": { "Configuration": "redis.contoso.example:6380,ssl=true,password=<password>" }
}
The driver announces each stored signal on the Redis channel once the store’s transaction commits, so a waiting pump worker claims it at once. A worker that receives no announcement looks at the store again after BlockingWaitSeconds (default 30). A timer is announced when it is scheduled, not when it falls due, so a due timer can start up to BlockingWaitSeconds late; lower the value when timers must be prompt.
Several nodes
Any number of nodes can share one store:
- Signals. Each pump worker claims signals with a lease, so one signal is processed by one node at a time.
- The retention job. It runs under a host lock (
WorkflowRetentionHostedJob:LockName); one node purges at a time. - Instances. Each change of an instance is checked against its version. A signal whose change loses that check to a concurrent change, for example the two branches of a fork signalled on two nodes, is applied again on the reloaded instance, up to five times. If the conflict lasts, the signal returns to the queue for another delivery, which counts as a delivery attempt.
- Workflows. Every node must register the same workflows at the same versions. A node that does not know a workflow cannot resume its instances.
Pitfalls
- One database for several services. Give the engine its own database and role; the schema and table names are fixed.
- The connection string in
appsettings.json. It holds a password. Use an environment variable or a secret store. - A role without
CREATEat the first start. The schema script fails and the host stops. Grant it once, or create the schema for the role. - Redis selected but not registered. With
WorkflowSignalDriver:Providerset toRedisand noAddWorkflowRedisSignaling, or noIConnectionMultiplexer, the host fails at start. - Mixed versions on one store. Upgrade all nodes together (Upgrading).
