Troubleshooting

Find the symptom, read the cause, take the action. Each row names the check that confirms the cause.

The host does not start

SymptomCauseCheck and action
“The PostgreSQL workflow backend requires a connection string and none is configured”STELLAOPS_POSTGRES_WORKFLOW_CONNECTION is not set where the host runsSet it in the environment or the secret store of that host
“Workflow backend provider ‘X’ is selected but no backend plugin registered it”WorkflowBackend:Provider names a store whose library is not registeredRegister the store library, or correct the provider name (Postgres, Mongo)
“Workflow signal driver provider ‘X’ is selected but no signal driver plugin registered it”WorkflowSignalDriver:Provider is Redis without AddWorkflowRedisSignalingRegister the driver, or set Native
“WorkflowRuntime.MaxMapConcurrency must be between 1 and 64” (and similar)An engine option is outside its rangeCorrect the value (Configuration reference)
The schema migration fails with permission denied for databaseThe role may not create the workflow schemaGrant CREATE on the database once, or create the schema for the role (Persistence and signals)
The build fails with WF001 to WF010The analyzer found C# in a Spec that cannot be a canonical definitionRead the message; the analyzer reference gives the fix
The build fails with WF020A class implements IDeclarativeWorkflow<T> without a Spec propertyAdd the Spec (analyzer reference)
The build reports WF021, or fails on it with warnings as errorsThe generator cannot evaluate a construct at build time; the workflow is left out of the build-time bundle and the publisher compiles it when it startsNothing for a workflow registered in code; or <WarningsNotAsErrors>WF021</WarningsNotAsErrors> (analyzer reference)
The publisher reports a workflow as failed with WFCD codesThe workflow uses a C# delegate, which has no canonical definitionUse the WorkflowExpr form (Defining workflows)

A workflow does not behave as written

SymptomCauseCheck and action
A start answers 404 with WorkflowDefinitionNotFoundThe workflow is not registered in this host, or the name differsGET /api/workflow/definitions lists the registered workflows
An imported definition cannot be startedImports are stored, not registeredRegister the workflow in code (Governed activation and publishing)
The host stops at start, or a definition is refused, with WFIMP010 ... Required workflow module 'transport.endpoint' is not installedNo transport package for an address kind the workflow usesRegister the transport (Calling services); a test with its own transport registers the module with AddWorkflowModule
A step after a timer or a signal never runsThe signal pump does not run on any nodeCall AddWorkflowEngineHostedServices on at least one node; check GET /api/workflow/signals/pump/stats
With the Redis driver, timers start up to 30 seconds after they fall dueA timer is announced when it is scheduled, not when it falls due; the pump finds it on its next look at the storeLower WorkflowSignalDriver:Redis:BlockingWaitSeconds (Persistence and signals)
An instance stops and its signal is in the dead lettersA step failed MaxDeliveryAttempts timesRead the error, fix the cause, replay (Operations)
A state change is refused with WORKFLOW_STATE_JSON_*The state is too large, too deep, or has a property name with a forbidden fragmentKeep large data and secrets out of the state

Calls fail

SymptomCauseCheck and action
“Endpoint transport has no base address for service ‘X’”WorkflowEndpointTransport:Services:X:Url is missingConfigure the service
The publisher fails every workflow with 404Endpoint or CommandPath does not reach the target’s import routeEndpoint is the target’s base URL; CommandPath defaults to /api/workflow/definitions/import
The host stops at start with an options validation error for WorkflowPublisherAn enabled publisher has no absolute http or https Endpoint, or another option breaks a ruleSet the key the error names, or Enabled: false or DryRun: true (Governed activation and publishing)
An error names a missing transport for a kindThe transport package of that kind is not registeredRegister it (Calling services)
A rabbit call fails at onceWorkflowRabbitTransport:HostName is not setConfigure the broker
A rabbit command runs the timeout branchNo reply within ReplyTimeoutSeconds, or the step’s timeoutSeconds passedCheck the consumer and the reply mode
A call returns 404The path in the definition, or the base URL, is wrongThe request goes to the base URL joined with the path; check both

HTTP API

SymptomCauseCheck and action
401 on every routeNo token, an expired token, another issuer or audienceCheck Authentication:Schemes:Bearer and the token
403 on a routeThe token lacks the scope of the route’s policyHosting
403 although the token has the scopeThe scopes are in a claim the handler does not readPut them in scope (space-delimited) or scp; set MapInboundClaims = false
A task action answers 403 with WorkflowTaskActionDeniedThe caller’s roles do not include a role of the taskCheck the token’s role claims and WorkflowAuthorization:RoleClaimTypes
A task action answers 403 with WorkflowActorUnresolvedThe token has none of the actor ID claim typesCheck WorkflowAuthorization:ActorIdClaimTypes (default sub, then the name identifier)
An import or activation answers 409 or 400A governance refusal, named in the body’s error (WORKFLOW_DEFINITION_VERSION_HASH_CONFLICT, WORKFLOW_DEFINITION_ACTIVATION_FENCE_LOST, and others)Governed activation and publishing, Refusals
A request answers 500WorkflowRuntimeFailed in the body, or no body: an unexpected errorThe host log has the exception
A diagram request takes minutesThe default layout effort is BestAsk for layoutEffort=Draft or Balanced (Rendering)

Licence messages

MessageMeaning
Status within-free-limitThe count is at or below the notice level; information only
Status above-notice-levelThe count is above the notice level and within the free limit; a notice, nothing is refused
Status record-matchesA licence entry covers the running version, and the count is within its capacity or the free limit; information only
Status embedded-scope-not-establishedAn embedded licence entry decides; the local record does not establish its product and end-customer scope. Check the Order; nothing is refused
Status no-matching-record, count-exceeds-recorded-capacity, development-build, count-unavailableSee Licensing; nothing is refused, stopped or restricted

Collecting information for support

  1. The package version of every StellaOps.Workflow* package (all must be the same).
  2. The host’s configuration with secrets removed.
  3. The log of the start and of the failing operation, at Information.
  4. GET /readyz, GET /api/workflow/signals/pump/stats, and the dead letters of the instance.