Release Orchestrator Architecture

Technical architecture specification for the Release Orchestrator — Stella Ops Suite’s central release control plane for non-Kubernetes container estates.

Status: Active Development (backend substantially implemented; API surface layer in progress)

Implementation reality (updated 2026-04-27; project counts re-verified 2026-05-30): The backend is substantially complete. As of 2026-05-30 the module spans 67 .csproj projects (32 of them test projects). Core libraries (Release, Promotion, Deployment, Workflow, Evidence, PolicyGate, Progressive, ProgressiveDelivery, Federation, Compliance) are implemented with comprehensive tests. Seven deployment target types are operational (Compose, Docker, SSH, WinRM, ECS, Nomad, Ansible), alongside build/deploy capability agents (BuildDocker, BuildScript, DeployCompose, DeployNative). Compatibility HTTP surfaces now exist across Platform, JobEngine, and Scanner for environment management, deployment monitoring, evidence inspection, dashboard promotion decisions, and registry search. The standalone WebApi now owns /api/v1/release-orchestrator/environments, /targets, and /freeze-windows for the environment-management slice, with startup migrations wired into the release schema; Platform’s no-PostgreSQL runtime now proxies this owning API instead of falling back to local in-memory environment stores. The standalone WebApi also owns and auto-migrates the scripts PostgreSQL schema used by /api/v2/scripts; that surface now exposes full entry-point/dependency/version metadata, emits X-Total-Count for list parity, accepts script:read / script:write alongside the older orchestrator scopes, and evaluates compatibility against persisted script metadata, declared variables, dependency inventory, target metadata, and available secrets instead of returning a fabricated success response. Script registry writes now compute sha256: content hashes from canonical framed content plus ordered dependency name/source/version/development metadata, and persist the same digest to the current script row and version row. Script search is store-backed through the PostgreSQL script store and does not maintain a process-local index in production wiring. Platform’s no-PostgreSQL scripts path now proxies this owning API instead of falling back to a local in-memory catalog. The real environment-management UI is now reachable at /releases/environments, with live authenticated detail/settings/targets/freeze-window flows backed by the owning persisted API; the older legacy release-management entry points redirect into that screen. The live /releases/deployments route now reads persisted state from the owning release_orchestrator.deployments table and survives service restarts instead of seeding fake compatibility rows on first access. Release and approval compatibility list/detail/create/update/delete/ready/promote/approve/reject/dashboard/v2 approval paths now use the release_orchestrator release-truth tables rather than compatibility SeedData; evidence endpoints read stored canonical bytes from release_orchestrator.release_evidence; and deploy gate checks hydrate durable release_orchestrator.gate_decisions before deciding. Promote/deploy compatibility endpoints fail closed when approval, gate-decision, or verified evidence truth is missing, and direct release deploy creates a pending persisted deployment request instead of projecting an immediate deployed state. The release_orchestrator schema includes release, component, event, approval, gate-decision, and release-evidence tables for the durable release truth cutover. Remaining gaps: some legacy compatibility surfaces still need full execution-engine backing rather than compatibility projections.

Overview

The Release Orchestrator transforms Stella Ops Suite from a vulnerability scanning platform into a centralized, auditable release control plane. It sits between CI systems and runtime targets, governing promotion across environments, enforcing security and policy gates, and producing verifiable evidence for every release decision.

Ownership boundary: Platform owns the Stella Ops installation control plane (installation identity/settings, setup, tenants, and cross-service read projections). ReleaseOrchestrator exclusively owns deployment environments, targets, deployed image digests/inventory, promotions, and execution state. Platform may expose a compatibility facade or read projection, but it delegates mutations to this WebApi and does not run this module’s environment migrations. Sharing one PostgreSQL server/database does not change schema ownership.

Core Value Proposition

• Release orchestration — UI-driven promotion (Dev → Stage → Prod), approvals, policy gates, rollbacks
• Security decisioning as a gate — Scan on build, evaluate on release, re-evaluate on CVE updates
• OCI-digest-first releases — Immutable digest-based release identity
• Toolchain-agnostic integrations — Plug into any SCM, CI, registry, secrets system
• Auditability + standards — Evidence packets, SBOM/VEX/attestation support, deterministic replay

Design Principles

1. Digest-First Release Identity — A release is an immutable set of OCI digests, never mutable tags. Tags are resolved to digests at release creation time.

2. Pluggable Everything, Stable Core — Integrations are plugins; the core orchestration engine is stable. Plugins contribute UI screens, connector logic, step types, and agent types.

3. Evidence for Every Decision — Every deployment/promotion produces an immutable evidence record containing who, what, why, how, and when.

4. No Feature Gating — All plans include all features. Limits are only: environments, new digests/day, fair use on deployments.

5. Offline-First Operation — All core operations work in air-gapped environments. Plugins may require connectivity; core does not.

6. Immutable Generated Artifacts — Every deployment generates and stores immutable artifacts (compose lockfiles, scripts, evidence).

7. Truthful Runtime Placeholders — Production-root runtime placeholders must fail closed. Test doubles for registry connectors, event publishers, agent certificates, inventory collection, and progressive-delivery metrics live under adjacent test support with Test* names. Unconfigured agent/orchestrator and workflow-step bindings throw explicit configuration errors or return explicit failed/unavailable results instead of reporting success.

Runtime Fail-Closed Boundaries (2026-04-28)

• AgentManager.ExecuteTaskAsync is an active deployment dispatch surface. Production DI registers the IAgentTaskTransport boundary and the release_orchestrator_agent_task_transport readiness check. The default transport is UnavailableAgentTaskTransport, which is unhealthy and returns a failed TaskResult; deployment collectors must treat that as a failed task, not a successful deployment. When ReleaseOrchestrator:AgentTaskTransport:Enabled=true, the standalone WebApi registers MtlsPollingAgentTaskTransport and exposes /api/v1/release-orchestrator/agent-runtime/agents/{agentId}/tasks:poll plus /tasks/{taskId}/result. The runtime requires HTTPS configuration, mTLS client certificates bound to the registered agent certificate thumbprint, version agent-task-transport/v1, matching task ids, and a result signature verified with the mTLS certificate public key before task success is accepted. The mapped execution targets are Compose deploy (compose.up), Compose preflight/rollback, single-container Docker deploy (docker.run), SSH (ssh.execute), WinRM (winrm.execute), ECS service deploy (ecs.deploy), and Nomad job deploy (nomad.deploy); unsupported deployment task types fail closed.
• Infrastructure binding connectivity checks are not satisfied by binding existence alone. InfrastructureBindingService.TestBindingAsync calls the bound IntegrationHub connector test when an IIntegrationManager is configured. It fails closed when the integration is missing, disabled, role/type mismatched, the connector test throws, or no connector-backed tester is configured. Topology readiness gates for registry, vault, and settings-store bindings consume that result.
• Policy-gate bundle simulation is a preview surface only. If promotion artifact hydration is not configured, SimulateBundleAsync returns Error, AllGatesPassed=false, and a blocking gate named promotion_artifact_hydration_unavailable; it is not a policy decision record.
• Compliance control categories without an automated validator fail closed as failed controls with remediation guidance. They must not be counted as not applicable unless the framework map explicitly defines them as not applicable.

Current Deployment Runtime Snapshot (2026-05-11)

The Release Orchestrator runtime is no longer a stub-only deployment surface. The current WebApi can create persisted environments, targets, agents, releases, components, approvals, gate decisions, deployment jobs, and per-target execution state. The repeatable VM gate npm run test:e2e:live:deployment-vms passed 20/20 on 2026-05-09 against the local Hyper-V lab and Docker Desktop host. That run proved Docker/Compose runtime preflight, true runtime-missing failures, registry/pull/health/digest negative cases, Linux SSH and Windows WinRM script/container deployment, multi-service Compose with pre/post hook evidence, rolling/canary/A/B strategy API behavior, native target-agent ComposeHost seed/update/rollback, and a second Linux target fixture. See docs/qa/agent-targets/e2e-test-runbook.md for current command and evidence ids. Subsequent 2026-05-11 targeted Playwright runs added two-Linux rolling progression, required traffic-adapter execution, a lab Nginx data-plane cutover, and native DockerHost target-agent seed/update proof through docker.run.

Agent Transport Modes

Agent Local OCI Registry

StellaOps.Agent.Host wires the local OCI registry when either Agent:LocalRegistry:Enabled=true or Agent:LocalRegistry:OciLayoutBundlePath is set. OciLayoutBundlePath points at an extracted OCI image-layout oci/ directory from an AirGap bundle. On startup, OciLayoutBundleImportHostedService validates oci-layout, index.json, manifest descriptor resolution, and every blobs/sha256/<hex> file by recomputing SHA-256 before importing through OciImageLayoutImporter into IOciContentStore; any mismatch throws and aborts startup. In BundleOnly mode with no fetcher, the /v2/ listener serves only imported content and returns 404 on cache misses without outbound registry access. The agent registry keeps this verification inline and does not take a production dependency on StellaOps.AirGap.Bundle.

In pull-through mode the registry persists fetched content under /var/lib/stellaops/agent-registry-cache. The hardened agent image creates this directory and gives ownership to the non-root application user at build time; a pre-existing volume with root-only ownership makes the registry fail closed instead of serving partially cached content.

Agent registration endpoints live under /api/v1/release-orchestrator/agents and registration tokens under /api/v1/release-orchestrator/agents/registration-tokens. Agent task polling uses /api/v1/release-orchestrator/agent-runtime/agents/{agentId}/tasks:poll and completion uses /tasks/{taskId}/result.

Console agent reuse is target-assignment driven: the environment target form reads registered agents and the tenant’s existing /environments/{id}/targets rows, then derives “used in” environment chips from target agentId assignments. This is intentionally separate from Platform’s /api/v2/topology/* release-control read model, which can be empty until bundle materialization produces topology projections. The Topology > Agents page falls back to the same Release Orchestrator registry view when the Platform topology projection has no agent/target rows.

Target-to-agent assignment is owned by ReleaseOrchestrator at both the topology compatibility alias POST /api/v1/targets/{id}/assign-agent and the owning PUT /api/v1/release-orchestrator/targets/{id}/agent route. Both routes use the same fail-closed eligibility check before persisting the assignment: the target must belong to the authenticated tenant, and the agent must exist in that same tenant, be Active, and advertise the executor capability selected by the deployment route (DockerHost -> Docker, ComposeHost -> Compose, SshHost -> Ssh, WinRmHost -> WinRm, EcsService -> Ecs, NomadJob -> Nomad, AnsibleHost -> Ansible). An foreign target returns non-leaking 404 target_not_found; an absent or foreign-tenant agent returns non-leaking 404 agent_not_found; a non-active agent or missing executor capability returns 409. Heartbeat freshness remains a health/execution concern rather than an enrollment precondition. A target deleted between eligibility validation and persistence still returns 404 target_not_found rather than escaping as a server error. Agent state can change between this validation and target persistence, so health checks and deployment dispatch must continue to verify the current agent and fail closed; assignment eligibility is not a liveness lease.

Target Type Runtime Matrix

ComposeHost has three explicit endpoint modes. The default useRemoteDockerEndpoint=false with no SSH configuration uses the assigned agent’s local Docker socket. SSH configuration selects DOCKER_HOST=ssh://user@host. Remote Docker TCP requires useRemoteDockerEndpoint=true and sets DOCKER_HOST=tcp://host:port; with useTls=true, the orchestrator resolves the CA/client-certificate/client-key references and the agent materializes ca.pem, cert.pem, and key.pem under the task root before setting DOCKER_TLS_VERIFY=1 and DOCKER_CERT_PATH. Partial certificate sets fail closed, and all task certificate files are deleted after preflight/deploy/rollback. NX-1 behavioral coverage uses a real mutually authenticated fake remote daemon, an actual Compose deployment, independent host-side container inspection, and credential-root deletion. NX-2 coverage sends prerequisite-first four-service Compose waves through the production Linux agent’s Docker-over-SSH path and verifies the complete suite is running.

Compose registry preflight has two explicit contracts. preloaded is cache-only: it runs docker image inspect on the deployment daemon and skips registry login and manifest inspection even when registry credentials are available. Registry mode authenticates with the task-scoped deployment credential before inspecting the manifest. Failure evidence retains a stable reason code: registry_authentication_failed, registry_credentials_missing, registry_image_missing, registry_unreachable, or registry_check_failed; the deployment evaluator converts those codes to operator-specific remediation while preserving fail-closed behavior. Plain-HTTP registries require the explicit component setting runtime.registryAllowInsecure=true; only then does the agent add Docker’s manifest-inspection --insecure opt-in. The default remains TLS-only, and the opt-in is recorded in preflight evidence.

For SSH-mode Compose, the task scope writes the private key and an initially empty known_hosts file with owner-only permissions. A task-local ssh wrapper is placed first on PATH because Docker’s ssh:// transport does not consistently honor DOCKER_SSH_COMMAND; the wrapper enforces IdentitiesOnly=yes, BatchMode=yes, StrictHostKeyChecking=accept-new, and the task-local known-hosts file. This is first-contact TOFU, not pre-pinned host-key verification. Environments that require pre-pinning must provide a separately governed host-key admission contract before changing this boundary. The wrapper, key, Docker config, and learned host key are deleted together on every terminal task path.

Ansible Delivery Engine

AnsibleHost is the first-class bridge for estates whose deployment authority remains an operator-supplied playbook. The control plane owns release identity, gates, target/agent selection, bounded configuration transport, deployment-scoped credential authority, dispatch, and evidence. It deliberately does not generate, rewrite, or vendor customer playbooks or roles. The configured PlaybookBasePath must already be visible inside the agent runtime, and DefaultPlaybookPath can be overridden per component by ansible.playbook.

Target-specific non-secret inventory and vars text belongs in AnsibleHostConfig.ManagedArtifacts. Artifact names are portable relative paths; the agent enforces count and character bounds, rejects rooted/traversal paths, materializes the files below the task temp root, resolves managed://<path> and {managedArtifactRoot}, and deletes the root on every terminal path. This replaces workstation bind mounts and direct database edits without turning target configuration into a secret store. Secret-bearing values remain references in target secret-ref fields or component ansible.environment.* configuration.

For pull-mode deployments with a minted ADR-034 capability, builtin://, vault://, openbao://, and authref:// environment values remain unresolved until the agent calls the orchestrator broker. The capability is deployment-scoped and reference-bound; the agent receives no standing secret-provider token. Target SSH, Vault-password, and become-password references are delivered through the scoped credential channel as ansible.sshPrivateKey, ansible.vaultPassword, and ansible.becomePassword. Environment values and resolved credentials are redacted from stdout, stderr, raw proof, and process errors.

The target health surface proves only that its assigned agent is active, fresh, and advertises Ansible. Acceptance still requires a live forcing function: the unchanged playbook must converge on the intended host and independent target inspection must confirm the expected digest-pinned workload. The complete setup and verification procedure is the Ansible delivery environment onboarding runbook; the payload and allow-list details remain in Execution Plugins.

Approval Policy Runtime

The current approval policy evaluator is DB-backed by default. Operator-managed rows live in release_orchestrator.approval_policies and map source environment, target environment, and optional release-name glob predicates to auto-approve, require-manual, or deny. Matching is tenant-scoped and deterministic: enabled policies are ranked by priority DESC, then predicate specificity, then creation time.

The management surface is /api/v1/release-orchestrator/approval-policies and the UI editor is reachable at /releases/policies. Read operations require release.policy.read; mutations require release.policy.manage and emit audit actions for create, update, and delete. Delete is an audit-preserving soft disable.

PromotionRequestProcessor persists the matched policy id, mode, reason, gate result, and required-approval override into durable release truth. Deny returns 409 and deployment remains fail-closed because the deployment truth guard refuses blocked approvals. The legacy ReleaseOrchestrator:PromotionApprovalPolicy config rules remain only as a transition fallback when ReleaseOrchestrator:PromotionApprovalPolicy:UseDbBackedStore=false or no policy repository is registered.

Promotion persistence has a strict durability order: the gate-decision row is committed first, then the resulting approval status, quorum, and complete gate results are committed before optional OCI referrer and policy-evidence work. Those optional side effects still honor request cancellation, but cancellation during them cannot strand a pending approval with empty gate results after gate truth has committed.

Approval mutation has one backend safety boundary: ApprovalSafetyGuard. The normal, batch, v2, and dashboard approve routes all call it before quorum can change. Empty gate rows, waiting, advisory/skipped incomplete gate truth, a non-passing gate, missing evidence, or an invalid/expired exception return a typed conflict without mutating the approval. The dedicated gate-exception route accepts any such transition-blocking gate result, including advisory/skipped, but only with the gate-bypass scope and a signed, time-limited operator decision. Reject remains independent so an operator can still stop an unsafe promotion. Batch approval verifies the operator decision once, preflights every selected approval, and only then persists decisions, preventing partial batch mutation.

Protected environments and signing-required environments add a stronger contract: the durable gate decision must contain a self-consistent GateDecisionBasis, a DSSE envelope bound to that canonical decision, and a policy-decision evidence packet whose status was earned through EvidenceVerificationService. Open environments retain the advisory evidence rollout and do not require signed gate-decision evidence unless global Approve signing or the per-environment signing list is enabled.

Operator-signed release decisions (Sprint 20260608_014)

Release decision signing is wired behind default-off per-decision flags under ReleaseOrchestrator:OperatorDecisionEnforcement. The flags are Approve, Reject, Promote, Deploy, and Rollback; all default to false so existing operator flows continue to work until a signing client is available. When a flag is enabled, the corresponding endpoint requires an operatorDecision body carrying an operator-decision@v1 DSSE envelope. The Release Orchestrator does not perform local signature verification: it calls the WS3 IOperatorDecisionAttestationService, which verifies through IOperatorDecisionVerifier and the configured ICryptoProviderRegistry. Fresh-auth is evaluated with the shared FreshAuthenticationGate (RequireFreshAuthentication=true, five-minute default window, optional MinimumAcr).

Production verification resolves the DSSE envelope keyid through the Authority IssuerDirectory client. Set IssuerDirectory:Client:BaseAddress and ReleaseOrchestrator:OperatorDecisionEnforcement:IssuerId before enabling a decision-enforcement flag. The resolver calls the IssuerDirectory operator-signing-key read API, maps the public material into the WS3 EnrolledKey contract, and fails closed when the client, issuer id, key row, algorithm id, or public material is unavailable.

Approve=true applies to every approval surface, including batch, v2, and the dashboard route. A narrower rollout can instead list target environments under ReleaseOrchestrator:ApprovalSafety:SigningRequiredEnvironments; those environments require both a verified operator approval envelope and verified signed gate-decision evidence. Batch clients send the envelope in the optional operatorDecision member of the batch body.

The release truth schema has additive nullable operator-decision columns on both release_orchestrator.approvals and release_orchestrator.gate_decisions: decision_envelope, decision_keyid, decision_algorithm, decision_signed_at, and decision_digest. Unsigned/default-off rows keep these fields null. Signed rows persist the verified envelope metadata and the real sha256: digest of the submitted compact DSSE envelope. Approval evidence and v2 replay surfaces expose the real digest plus keyid; unsigned approvals return null / not_available and no longer synthesize sha256:decision-* or policy-decision-*.dsse placeholders.

release_orchestrator.release_evidence remains the store for release evidence bytes and release-evidence signatures. The operator-decision envelope is stored on the approval/gate-decision row because the enforcement and replay paths need per-decision truth by approval/gate id, and deploy/rollback decisions are lifecycle decisions rather than standalone release-evidence packets. The digest column preserves the compact envelope content address even though the JSONB envelope column may be rendered differently by PostgreSQL.

Digest-keyed deployment-decision artifacts (WF5)

The promotion gate emits the WF5 referrer after the durable gate-decision upsert in PromotionRequestProcessor.ProcessAsync. Emission is keyed by component image digest, so one signed deployment-decision referrer is attached per valid sha256: release component digest rather than per release. The predicate captures the gate verdict, policy id/version, tenant, masked image reference, SBOM readiness, and reachable CVE hints available from the gate results. Registry placement is derived from the component image reference host/repo when available and falls back to the internal metadata registry namespace. Referrer emission is fire-and-log: signing or OCI attachment failures are logged per component and never roll back the already-persisted promotion decision.

In connected profiles, the WebApi registers ScannerReferrerSink when Scanner:BaseUri is configured. The sink posts the deployment-decision referrer to Scanner’s POST /api/v1/oci-referrers ingest surface with X-StellaOps-TenantId, so the signed verdict lands in the Scanner OCI metadata store that MetadataRegistryGateway.ResolveDecisionAsync later reads through the /v2/<repo>/referrers/<digest> surface. When ReleaseOrchestrator:SbomReadinessGate:ServiceIdentity:Enabled=true, the Scanner sink and metadata gateway clients also attach a tenant-scoped stellaops-release-dispatch client-credentials token. The default requested scope set is scanner:read scanner:scan scanner:write, covering both deployment-decision ingest and OCI referrer read-back off the trusted bypass network. When Scanner:BaseUri is absent, the WebApi keeps the local FileSystemOciContentStore sink for offline and bundle-only profiles.

Deployed-estate deviations keep a local structured warning as the fail-visible baseline. When ReleaseOrchestrator:DeviationNotify:Enabled=true, HttpDeviationEventPublisher additionally posts the canonical generic-event envelope to Notify at /api/v1/events/releaseorchestrator. The HTTP client mints a per-tenant notify.operator client-credentials token from the shared stellaops-release-dispatch identity, carries the same tenant in X-StellaOps-TenantId, and treats transport/non-success responses as non-fatal so notification availability cannot change deployment, rollback, heartbeat, or estate re-touch state. The shipped compose profile enables this path; operators can set RO_DEVIATION_NOTIFY_ENABLED=false while retaining the local warning trail. Agent heartbeats persist drift under the agent’s canonical tenant UUID. Before an outbound Notify hop, Release Orchestrator resolves that UUID through shared.tenants to the registered Authority tenant key and uses that one key consistently in the token request, header, body, and idempotency key. Unknown UUIDs fail closed before HTTP dispatch while retaining the local structured warning.

Deployment-decision signing can be supplied from runtime configuration with ReleaseOrchestrator:DeploymentDecision:Signing:KeyId and one of two key sources (checked in this precedence order):

1. ReleaseOrchestrator:DeploymentDecision:Signing:Es256PrivateKeyPkcs8Base64 (env alias STELLAOPS_DEPLOYMENT_DECISION_ES256_PRIVATE_KEY_PKCS8_B64) — an inline base64 PKCS#8 key, kept for back-compat with environment-injected keys.
2. ReleaseOrchestrator:DeploymentDecision:Signing:Es256PrivateKeyPemFile (env alias STELLAOPS_DEPLOYMENT_DECISION_ES256_PRIVATE_KEY_PEM_FILE) — a path to a mounted EC (P-256) PEM. This is the primary source for the deployed stack: the base compose points it at a per-lab dev PEM generated by devops/compose/scripts/ensure-dev-certs.sh (gitignored under devops/etc/release-orchestrator/keys/, mounted read-only), mirroring the scanner scan-attestation key custody pattern. A plain docker compose up recreate therefore keeps signing without re-injecting a key from the environment. Production replaces the mounted file with a custody-managed PEM at the same container path.

The key id defaults to STELLAOPS_DEPLOYMENT_DECISION_KEY_ID. When a key is present the WebApi imports the P-256 private key into the configured crypto provider and sets DSSE signing defaults to ES256 for WF5 deployment-decision artifacts; the corresponding SPKI public key is exported for agent registry verification. When neither source yields usable key material the signer degrades honestly to decision_algorithm='unsigned' with a NULL envelope — never a silent pass, and never a repo-baked key.

The live agent pull-through registry path imports those same decision referrers before it stores the subject manifest locally. On a pull-through miss, OrchestratorImageFetcher asks the orchestrator content-by-digest channel for /deployments/{id}/registry/decision-referrers/{subjectDigest}; the WebApi re-reads Scanner’s /v2/{repo}/referrers/{digest} surface, digest-verifies the referrer manifest and layer bytes, and returns the bundle over the deployment capability channel. The agent stores the referrer blobs and manifests in its local OCI cache before writing the image manifest, so local registry enforcement sees the same signed Scanner SoR decision that connected promotion emitted.

OCI image indexes and Docker manifest lists are treated as first-class subjects: the pull-through fetcher stores the top-level index/list, fetches and stores the referenced child manifests and blobs, and keeps the deployment-decision referrer bound to the top-level subject digest. When a runtime pulls a child manifest, the offline verifier walks the cached parent index/list back to the signed parent decision and rejects the child transitively when the parent verdict is block or deny.

StellaOps.ReleaseOrchestrator.DecisionArtifact defines the shared contract for the WF5 deployment-decision referrer. The canonical predicate JSON uses alphabetically ordered camel-case fields, is signed through IDsseSigner as an application/vnd.in-toto+json DSSE payload with predicate type https://stellaops.io/attestation/v1/deployment-decision, and is attached as an OCI referrer with artifact type application/vnd.stellaops.deployment-decision.v1. Verification is fail-closed: the DSSE envelope must parse, the signature must verify against the trusted EC public key, subjectDigest must match the deployed digest, and verdict=block or verdict=deny is rejected even when the signature is valid.

Gate plugin migration (Sprint 20260527_023 GPM-003 → GPM-006)

The two built-in promotion gates (approval-policy and reachability-gate) were migrated to the plugin-registry path in Sprint 023. The canonical implementations are ApprovalPolicyGatePlugin and ReachabilityGatePlugin (in WebApi/GatePlugins/); each wraps the legacy domain-service evaluator behind a BuiltInGatePluginEnvelope.ContributionKey envelope so the plugin path produces byte-identical GateDecisionDto.GateResults[] rows (pinned by PluginPathGateResultsParityTests). The default IGateDecisionEvaluator is now PluginBackedGateDecisionEvaluator. The legacy CompositeGateDecisionEvaluator path remains wired for one release behind the feature flag ReleaseOrchestrator:GateEvaluator:UsePluginRegistry=false as an operator escape hatch. The domain-service evaluators (PolicyDrivenGateDecisionEvaluator, ReachabilityGateEvaluator) are marked [Obsolete] and slated for removal in the release after the plugin path has run hot for one milestone.

SBOM readiness gate (Sprint 20260531_054)

Component-version sync now asks Scanner whether each newly discovered digest is already tracked before enqueueing work. HttpSbomScannerClient calls Scanner’s GET /api/v1/sbom-readiness contract with the tenant header and submits POST /api/v1/scans only when the state is absent; pending, scanning, ready, and failed states are not re-submitted. The scan submit payload carries the digest-pinned image reference, component id, version id, registry integration id, and deterministic clientRequestId so a registry re-sync cannot create a scan storm.

Promotion gate evaluation has a distinct waiting state for SBOM gaps. The built-in SbomReadinessGatePlugin wraps SbomReadinessGateEvaluator and runs before approval-policy and reachability gates. If any release component digest is pending, scanning, or absent, the contribution short-circuits with mode=waiting, gate type sbom-readiness, and an operator hint such as waiting on SBOM results: digest ..., status pending. A failed Scanner state is an explicit deny with the Scanner failure reason. ready digests contribute a passing SBOM-readiness gate and allow normal policy evaluation to continue.

CVE-aware policy gate (Sprint 20260512_002 R-GATE-CVE)

Before the metadata-only approval-policy match runs, PolicyDrivenGateDecisionEvaluator (wrapped by ApprovalPolicyGatePlugin on the plugin path) calls Policy.Engine’s POST /api/policy/packs/{packId}/revisions/{version}/evaluate endpoint via IPolicyEngineGateClient. The pack is resolved from ReleaseOrchestrator:Policy:EnvironmentBindings[targetEnv] first, falling back to ReleaseOrchestrator:Policy:DefaultPackId / DefaultPackVersion. If no pack is configured, the call is skipped and the legacy approval-policy path runs unchanged (back-compat preserved). A deny verdict from Policy.Engine overrides any auto-approve approval-policy rule — the resulting gate decision row is outcome=deny, mode=deny, policyId={packId}@v{version}, with the policy bundle digest captured in the reason. By default the gate is fail-closed: Policy.Engine 5xx / timeout / network failures produce a deny so a denylisted CVE cannot slip through during a policy-engine outage. Set ReleaseOrchestrator:Policy:FailClosed=false for rollback compatibility, or ReleaseOrchestrator:Policy:Enabled=false to fully disable the CVE-aware gate.

CVE-to-release impact query (Sprint 20260627_005)

The reverse impact endpoint GET /api/v1/release-orchestrator/release-impact?identifier={cve-or-advisory} answers the FE vulnerability lens without introducing a separate projection table. It reads tenant-scoped release truth from persisted releases, approvals, gate decisions, approval evidence packets, and release evidence packets. Current impact is built only from the latest gate decision per (releaseId, targetEnvironment) plus approval security snapshots when no current gate decision covers that approval gate. Older matching gate decisions are returned under timeline.historical and do not contribute to currentImpact.

The response separates impacted releases, approvals, promotions, environments, gate verdicts, reachable CVE rows, evidence links, and timeline rows. All collections are sorted deterministically for UI diffing, tenant boundaries are enforced through the release-truth repository, and no-impact identifiers return HTTP 200 with empty=true and empty arrays. If only historical rows match, currentImpact remains empty while timeline.historical preserves the audit trail.

Environment policy bindings (Sprint 20260513_021)

Pack-to-environment binding is now operator-managed via the release_orchestrator.environment_policy_bindings table (Postgres, consolidated baseline 001_v1_releaseorchestrator_baseline.sql:563 — folded in from the pre-1.0 007_environment_policy_bindings.sql, now archived and not embedded). The CVE-aware gate’s pack resolver walks this chain in HttpPolicyEngineGateClient.ResolvePackAsync:

1. DB binding (canonical): row in release_orchestrator.environment_policy_bindings keyed by (tenant_id, environment) with enabled = TRUE.
2. Static config map (DEPRECATED back-compat): ReleaseOrchestrator:Policy:EnvironmentBindings[targetEnv] from configuration. A startup INFO/WARN log fires when the map is populated, and a runtime warning fires every time the fallback is consumed.
3. Tenant default: ReleaseOrchestrator:Policy:DefaultPackId / ReleaseOrchestrator:Policy:DefaultPackVersion.
4. Skip — gate-call is skipped and legacy approval-policy logic runs.

The bindings are administered via tenant-scoped REST endpoints (/api/v1/release-orchestrator/environments/{environment}/policy-bindings, GET / PUT / DELETE, plus /environments/policy-bindings for list). Authorization uses the named policies release-orchestrator.policy-bindings.read (GET) and release-orchestrator.policy-bindings.write (PUT / DELETE); the JWT scope tokens that satisfy those policies are release.policy-bindings.read and release.policy-bindings.write respectively (see ReleaseOrchestratorPolicies.cs). Both scopes are in the canonical StellaOpsScopes catalog (StellaOpsScopes.cs:812 ReleasePolicyBindingsRead, :818 ReleasePolicyBindingsWrite) and ReleaseOrchestratorPolicies.cs binds the constants, not literals — so tokens minted by Authority satisfy them. (Corrected 2026-07-12: this paragraph previously claimed they were service-local and absent from the catalog. They were added to the catalog and the claim went stale.) DELETE is a soft-delete that flips enabled=false; rows remain visible via ?includeDisabled=true for audit replay.

The static config map is retained for one minor-release back-compat window and will be removed in a future major release.

Deployment Engine Boundary

HTTP deployment requests now enter the canonical library DeployOrchestrator.StartAsync(promotionId, options) path through LibraryDeploymentRequestStarter. The starter resolves release-truth approval rows, creates a dedicated deployment scope for the library engine, and mirrors terminal DeploymentJob state back into release_orchestrator.deployments, including target status, progress, errors, agent id, strategy evidence, and rollback status. The previous WebApiDeployOrchestrator adapter and WebApiArtifactGenerator stub were removed from the WebApi production assembly.

Deployment creation requires an approved promotion for the resolved release and target environment. If that approval is absent, the endpoint returns HTTP 409 ProblemDetails with code=approved_promotion_required, releaseId, and environment; the condition never enters the legacy fallback path, even when fallback is enabled. This is an operator-correctable state conflict, not an internal server error.

Within the deployment library, TargetExecutor owns stateful target dispatch, secret resolution, runtime preflight, health probes, task persistence, and evidence publication. Pure deployment-value rules are isolated in the internal TargetExecutionValues collaborator: digest/config parsing, masked-reference anchoring, per-bundle plugin classification, registry-context selection, result-value projection, and deterministic identifier/hash normalization. The collaborator has no service dependencies and does not change the public TargetExecutor constructor or agent task/wire contracts.

Immutable release and deployment subject projection

The cross-module canonical subject and evidence-spine contract defines the owner ids, digest passport, Web handoff parameters, lifecycle continuity, and distinct global/aggregate/record evidence-state namespaces used by Release Orchestrator, EvidenceLocker, Attestor, Estate, and the Console.

Release-artifact identity uses only the canonical OCI form sha256:<64-hex>. Component-version sync drops malformed registry values, and release/developer-approval writes reject malformed caller values instead of persisting a sha-looking identifier. Valid values are normalized to lowercase before they enter release, approval, or deployment projections.

Promotion approvals carry the canonical target environment id and component ids alongside the release id/version. Their candidateDigest is either the exact authored digest or the one unambiguous component digest; it is never derived from a release id. Deployment summary/detail JSON persists the same approval id, candidate digest, component subjects, release id, and environment id, and deployment lifecycle events repeat those identifiers for Evidence and Web consumers. Legacy records without a usable subject return legacy_release_digest_missing, legacy_release_digest_invalid, or release_digest_ambiguous; registry availability failures retain their separate currency reasons. The deploy-engine adapter uses the exact digest for a one-component release and a deterministic digest of ordered componentId=digest subjects for a multi-component manifest. These fields are additive members of the existing JSON projections, so no PostgreSQL schema migration is required.

The local deployment and bounded UI verification are recorded in the 2026-07-12 RDT-2 QA run.

Rollback requests use WebApiDeploymentRollbackStarter, which drives the deployment pipeline rollback path directly and projects terminal rollback state through the same compatibility read model. On successful WebApi rollbacks the starter creates the canonical release.rollback.evidence.v1 JSON through RollbackEvidenceGenerator, stores it as a verified release evidence packet, and exposes the evidence id, stored content hash, proof digest, and proof status on deployment strategy evidence. The rollback evidence also carries multi-module order proof: failed module identity, original deployment order, rollback order, dependency names, and release-script proof digests when task results include them. The projection exposes orderProofDigest and orderProofStatus next to target digest proof so replay/audit tooling can detect dependency-order proof drift. The library strategy planner covers rolling, canary, blue-green, and all-at-once execution; rolling/canary/blue-green are verified against real Testcontainers sshd targets in addition to recorder tests for batch order.

Release script graph proof now has an executable bounded contract in the Scripts library. IReleaseScriptGraphProofBuilder emits release.script.graph-proof.v1 evidence with deterministic start and update orders, module/script/version dependency references, fail-closed issues for missing dependencies or version mismatches, and a sha256: proof digest. Start scripts are not allowed to depend on update scripts; update scripts may retain cross-phase references to start scripts while update-to-update dependencies drive update ordering. See deployment/runtime-contract.md — release script graph proof.

Platform Themes

The Release Orchestrator introduces ten new functional themes:

Components

Reconciled 2026-05-30 against src/ReleaseOrchestrator/. The actual layout places domain libraries under __Libraries/, the HTTP service under __Apps/, and deployment/build agents under __Agents/. There is no separate StellaOps.ReleaseOrchestrator.WebService / .Worker / .Core / .Deploy / .Integration project — those names do not exist on disk.

ReleaseOrchestrator/
├── __Libraries/
│   ├── StellaOps.ReleaseOrchestrator.Core/             # Core domain models
│   ├── StellaOps.ReleaseOrchestrator.Foundation/       # Shared primitives
│   ├── StellaOps.ReleaseOrchestrator.Environment/      # Environment/Target/FreezeWindow models + topology migrations
│   ├── StellaOps.ReleaseOrchestrator.Release/          # Release + ReleaseComponent models
│   ├── StellaOps.ReleaseOrchestrator.Promotion/        # Promotion + GateResult + approval models
│   ├── StellaOps.ReleaseOrchestrator.Workflow/         # DAG workflow engine (WorkflowTemplate/Step/Run)
│   ├── StellaOps.ReleaseOrchestrator.Deployment/       # Deployment coordination (DeployOrchestrator, strategies, artifact generators)
│   ├── StellaOps.ReleaseOrchestrator.Evidence/         # Evidence generation/collector
│   ├── StellaOps.ReleaseOrchestrator.EvidenceThread/   # Evidence thread/verdict-ledger persistence
│   ├── StellaOps.ReleaseOrchestrator.PolicyGate/       # Policy-gate simulation models
│   ├── StellaOps.ReleaseOrchestrator.Progressive/      # Progressive delivery
│   ├── StellaOps.ReleaseOrchestrator.ProgressiveDelivery/ # A/B + canary + traffic-router controllers
│   ├── StellaOps.ReleaseOrchestrator.Compliance/       # Compliance control validators
│   ├── StellaOps.ReleaseOrchestrator.Federation/       # Federation
│   ├── StellaOps.ReleaseOrchestrator.IntegrationHub/   # Integration connectors / hub
│   ├── StellaOps.ReleaseOrchestrator.Observability/    # Metric/log/trace export infrastructure
│   ├── StellaOps.ReleaseOrchestrator.Scripts/          # Release-script registry + graph proof (scripts schema)
│   ├── StellaOps.ReleaseOrchestrator.SelfHealing/      # Self-healing
│   ├── StellaOps.ReleaseOrchestrator.Agent/            # Agent domain models + agent-store migrations
│   ├── StellaOps.ReleaseOrchestrator.Persistence/      # release_orchestrator schema migrations + Postgres repositories
│   ├── StellaOps.ReleaseOrchestrator.Plugin/           # Plugin infrastructure (gate/step/integration models)
│   ├── StellaOps.ReleaseOrchestrator.Plugin.Sdk/       # Plugin SDK
│   └── StellaOps.Runtime.Contracts/                    # Runtime observation DTOs (relocated from Zastava)
├── __Apps/
│   └── StellaOps.ReleaseOrchestrator.WebApi/           # HTTP API (minimal-API endpoints, gate plugins, services)
├── __Agents/
│   ├── StellaOps.Agent.Core/                           # Agent base framework + CapabilityRegistry
│   ├── StellaOps.Agent.Host/                           # Target-resident agent host (mTLS polling)
│   ├── StellaOps.Agent.Docker/                         # Docker host agent
│   ├── StellaOps.Agent.Compose/                        # Docker Compose agent
│   ├── StellaOps.Agent.Ssh/                            # SSH agentless executor
│   ├── StellaOps.Agent.WinRM/                          # WinRM agentless executor
│   ├── StellaOps.Agent.Ecs/                            # AWS ECS agent
│   ├── StellaOps.Agent.Nomad/                          # HashiCorp Nomad agent
│   ├── StellaOps.Agent.Ansible/                        # Ansible playbook agent
│   ├── StellaOps.Agent.DeployCompose/                  # Compose deploy capability
│   ├── StellaOps.Agent.DeployNative/                   # Native deploy capability
│   ├── StellaOps.Agent.BuildDocker/                    # Docker build capability
│   └── StellaOps.Agent.BuildScript/                    # Build-script capability
├── StellaOps.ReleaseOrchestrator.Api/                  # Shared controller contracts
└── __Tests/
    ├── StellaOps.ReleaseOrchestrator.*.Tests/
    └── StellaOps.Agent.*.Tests/

Data Flow

Release Orchestration Flow

CI Build → Registry Push → Webhook → Stella Scan → Create Release →
Request Promotion → Gate Evaluation → Decision Record →
Deploy via Agent → Version Sticker → Evidence Packet

Detailed Flow

1. CI pushes image to registry by digest; triggers webhook to Stella
2. Stella scans the new digest (if not already scanned); stores verdict
3. Release created bundling component digests with semantic version
4. Promotion requested to move release from source → target environment
5. Gate evaluation runs: security verdict, approval count, freeze windows, custom policies
6. Decision record produced with evidence refs and signed
7. Deployment executed via agent to target (Docker/Compose/ECS/Nomad)
8. Version sticker written to target for drift detection
9. Evidence packet sealed and stored

Key Abstractions

Reconciled 2026-05-30 against the records under src/ReleaseOrchestrator/__Libraries/. The shapes below are the actual domain records; earlier drafts used speculative field names (Slug, PromotionOrder, FreezeWindow[] FreezeWindows, ApprovalPolicy, EnvironmentState State) that do not exist in code. Freeze windows and approval policy are modelled as separate tables/records, not inline properties of Environment.

Environment

Source: StellaOps.ReleaseOrchestrator.Environment/Models/Environment.cs.

public sealed record Environment
{
    public required Guid Id { get; init; }
    public required Guid TenantId { get; init; }
    public required string Name { get; init; }        // "dev", "staging", "production" (lowercase, 2-32 chars)
    public required string DisplayName { get; init; }
    public string? Description { get; init; }
    public Guid? RegionId { get; init; }
    public required int OrderIndex { get; init; }     // promotion order (0 = earliest)
    public required bool IsProduction { get; init; }
    public required int RequiredApprovals { get; init; }
    public required bool RequireSeparationOfDuties { get; init; }
    public Guid? AutoPromoteFrom { get; init; }
    public required int DeploymentTimeoutSeconds { get; init; }
    public DateTimeOffset CreatedAt { get; init; }
    public DateTimeOffset UpdatedAt { get; init; }
    public Guid CreatedBy { get; init; }
    public ImmutableDictionary<string, string> BundleDefaults { get; init; } // per-env push-mode bundle defaults
    public bool RequiresHumanApproval { get; init; }                          // explicit human sign-off (no auto-approve)
    public ProtectionClass ProtectionClass { get; init; }                     // Open | Gated | Protected (Sprint 20260708_004)
    public LifecycleState LifecycleState { get; init; }                       // Active | Archived | Disposable (Sprint 20260717 G9)
}

// Health is a separate enum, not a state field on Environment:
public enum EnvironmentHealthStatus { Healthy = 0, Degraded = 1, Unhealthy = 2, Unknown = 3 }

// Explicit, owner-set estate lifecycle. Set ONLY by an archive/dispose/restore action — never inferred
// from a name. Governs default visibility/ranking (Archived/Disposable are demoted out of the estate
// matrix, readiness table, and pickers); it never mutates the authoritative gate mechanism fields. The
// sibling Target record carries the same field. Defaults to Active so pre-lifecycle rows keep their
// behaviour after the backfill.
public enum LifecycleState { Active = 0, Archived = 1, Disposable = 2 }

Lifecycle-state contract (Sprint 20260717 G9): lifecycle_state is persisted as a canonical lowercase token (active | archived | disposable). Storage fails closed on both sides — the enum-to-token serializer throws on an undefined value, ParseStorage throws on an unknown persisted token, and a forward-only CHECK constraint (migration 007_environment_lifecycle_state_constraints.sql) rejects any other value at the database. The PUT .../{environments|targets}/{id}/lifecycle endpoints (Operate scope, audited as set_environment_lifecycle / set_target_lifecycle) are the only transition path; the list endpoints accept an optional ?lifecycle= filter (all [default] | active | demoted | archived | disposable) and reject any other token with 400 invalid_lifecycle_filter.

Aggregate environment operational truth

The owning WebApi exposes a bounded, tenant-scoped read model for environment readiness and protection:

• GET /api/v1/release-orchestrator/environments/operational-truth returns all ordered environments in one HTTP response. Each item includes protection, target/readiness counts, evaluated gate reports, and component-specific problems. The response status is complete, partial, failed, or empty; an empty topology is a typed 200 result rather than an error.
• POST /api/v1/release-orchestrator/environments/readiness/validate-all evaluates every known target and returns every target result plus totalCount, succeededCount, and failedCount. One failed target does not discard successful results, and the failure names its environment and target.

The WebApi also contains an opt-in TopologyReadinessScheduler that can refresh target connection health and persist readiness reports on a cadence. It is disabled by default because every pass performs outbound probes to operator-configured targets or agent bindings and writes topology health rows. When enabled, the first pass runs after InitialDelay (default 45 seconds), subsequent passes run every Interval (default 5 minutes, with a 60-second minimum), and each health/readiness step is bounded by PerTargetTimeout (default 12 seconds). A failed or timed-out connection test first persists Unreachable with a live host token; readiness is not evaluated when that failure truth cannot be stored. The connectivity_ok meta-gate requires target health to be Healthy or Degraded for every target type, including ECS, Nomad, SSH, WinRM, and Ansible targets whose Docker-specific gates are skipped.

The scheduler establishes a required ReleaseEnvironmentIdentityAccessor override for each tenant before resolving any tenant-scoped service. There is currently no distributed lease for this job. Operators must enable it on only one ReleaseOrchestrator WebApi replica; enabling multiple replicas duplicates probes and writes. In sealed or air-gapped installations the scheduler makes no public-Internet request of its own, but configured targets/bindings can still cross a network boundary. Keep it disabled unless those destinations are local or explicitly permitted, and use the authenticated validate-all operation for manual evaluation. The periodic pass is the only automatic trigger in this slice; deployment- and heartbeat-triggered immediate refresh remain outside the implemented contract. Configuration is documented in appendices/config.md.

Protection repository failures are not projected as a missing policy pack or zero reviewers. The protection object becomes partial and reports protection_policy_binding_unavailable or protection_reviewer_source_unavailable. The aggregate item carries those same problems, allowing Console to distinguish unavailable truth from an actual unprotected configuration. The bounded executive operational reads GET /environments, GET /environments/operational-truth, GET /estate/status, GET /estate/drift, GET /estate/digest/{digest}, GET /estate/unproven-cells, and the Release Command Center’s per-environment GET /environments/{id}/targets and GET /environments/{id}/freeze-windows calls accept either release:read or orch:read; all eight still require the active tenant. The estate status and drift handlers filter the internal cross-tenant drift store by the active tenant’s canonical UUID before returning rows or computing counts. Agent heartbeats persist that UUID, while the compatibility estate/verdict stores continue to use the active tenant key (for example the registered default slug); the handler resolves both forms instead of comparing unlike values. Records from another tenant therefore cannot appear in the response. The digest lookup predicates its deployed-estate query on the same active tenant; other environment/estate reads remain orch:read-only, and all mutation policies are unchanged. Validate-all requires orch:operate and is audited. Source: ReleaseOrchestratorEnvironmentOperationalTruthEndpoints.cs and ReleaseOrchestratorEnvironmentEndpoints.cs; estate filtering is in EstateEndpoints.cs.

Environment operational truth is not the source for Estate’s service x environment unproven cells: readiness gates describe target connectivity and protection, while cell proof belongs to the deployment that established the cell. GET /api/v1/release-orchestrator/estate/unproven-cells is the distinct tenant-scoped producer. PostgreSQL selects the latest completed/succeeded deployment per case-insensitive service/environment key (matching the existing Estate matrix join), computes the exact count, orders deterministically, and returns at most 50 rows (default 20) without loading tenant deployment history into the application. Each row names the service, environment, release, and deployment and carries persisted landing status, per-target non-verified verdicts, mismatch anatomy, and deployment-attestation reference. Stored mismatch plainReason, action, and action path are returned unchanged; absent cause/cure data is explicit not_recorded. The bounded rows are also enriched with authoritative ownership only: the stored release creator plus active owner grants for that release or environment, split into user and group ids. Revoked grants, other roles, and other tenants are excluded. The response marks the nested ownership assigned or unassigned; it never infers an owner from a service, release, or environment name. An attestation reference proves a record was signed/anchored, not that a later cryptographic verification passed, so the row remains unproven until such a receipt is persisted. Every row links to deployment detail and, when a canonical digest exists, the subject passport. The response never substitutes environment readiness text for deployment proof.

Release

Source: StellaOps.ReleaseOrchestrator.Release/Models/Release.cs. A release bundles an ordered array of ReleaseComponent records (not a string -> ComponentDigest dictionary), tracks lifecycle through ReleaseStatus, and carries the canonical manifest digest set on finalization. There is no Version or SourceRef field on the record.

public sealed record Release
{
    public required Guid Id { get; init; }
    public required Guid TenantId { get; init; }
    public required string Name { get; init; }        // lowercase, alphanumeric with hyphens
    public required string DisplayName { get; init; }
    public string? Description { get; init; }
    public required ReleaseStatus Status { get; init; }
    public required ImmutableArray<ReleaseComponent> Components { get; init; }
    public string? ManifestDigest { get; init; }      // sha256: of canonical manifest (set on finalize)
    public DateTimeOffset? FinalizedAt { get; init; }
    public Guid? FinalizedBy { get; init; }
    public string? DeprecationReason { get; init; }
    public DateTimeOffset? DeprecatedAt { get; init; }
    public DateTimeOffset CreatedAt { get; init; }
    public DateTimeOffset UpdatedAt { get; init; }
    public Guid CreatedBy { get; init; }
}

public enum ReleaseStatus { Draft = 0, Ready = 1, Promoting = 2, Deployed = 3, Deprecated = 4 }

public sealed record ReleaseComponent
{
    public required Guid Id { get; init; }
    public required Guid ComponentId { get; init; }
    public required string ComponentName { get; init; }
    public required string Digest { get; init; }      // sha256:abc123...
    public string? Tag { get; init; }                 // tag resolved to this digest, if any
    public string? SemVer { get; init; }
    public ImmutableDictionary<string, string> Config { get; init; }
    public int OrderIndex { get; init; }              // deployment order within the release
}

Promotion

Source: StellaOps.ReleaseOrchestrator.Promotion/Models/Promotion.cs. State is tracked through PromotionStatus (nine values — see below), not a five-value PromotionState. There is no Decision property; the promotion carries denormalized source/target/release names plus reason fields per terminal outcome, and a DeploymentId once deployment is triggered.

public sealed record Promotion
{
    public required Guid Id { get; init; }
    public required Guid TenantId { get; init; }
    public required Guid ReleaseId { get; init; }
    public required string ReleaseName { get; init; }
    public required Guid SourceEnvironmentId { get; init; }
    public required string SourceEnvironmentName { get; init; }
    public required Guid TargetEnvironmentId { get; init; }
    public required string TargetEnvironmentName { get; init; }
    public required PromotionStatus Status { get; init; }
    public string? Reason { get; init; }
    public string? RejectionReason { get; init; }
    public string? CancellationReason { get; init; }
    public string? FailureReason { get; init; }
    public ImmutableArray<ApprovalRecord> Approvals { get; init; }
    public ImmutableArray<GateResult> GateResults { get; init; }
    public Guid? DeploymentId { get; init; }
    public DateTimeOffset RequestedAt { get; init; }
    public DateTimeOffset? SubmittedAt { get; init; }
    public DateTimeOffset? ApprovedAt { get; init; }
    public DateTimeOffset? DeployedAt { get; init; }
    public DateTimeOffset? CompletedAt { get; init; }
    public Guid RequestedBy { get; init; }
    public string RequestedByName { get; init; }
}

public enum PromotionStatus
{
    Pending, AwaitingApproval, Approved, Deploying,
    Deployed, Rejected, Cancelled, Failed, RolledBack
}

Workflow

Source: StellaOps.ReleaseOrchestrator.Workflow/Models/. The template type is WorkflowTemplate (not Workflow); the dependency graph is encoded as DependsOn arrays on each step rather than a separate DependencyGraph map; and WorkflowStep has no Provider or State property (the DAG is materialized into a WorkflowRun at execution time). Step Type is a free-form string ("script", "approval", "deploy", etc.).

public sealed record WorkflowStep
{
    public required string Id { get; init; }
    public required string Type { get; init; }        // "script", "approval", "deploy", ...
    public string? DisplayName { get; init; }
    public ImmutableArray<string> DependsOn { get; init; }
    public string? Condition { get; init; }
    public ImmutableDictionary<string, object> Config { get; init; }
    public TimeSpan? Timeout { get; init; }
    public RetryConfig? Retry { get; init; }
    public bool ContinueOnError { get; init; }
}

Target

Source: StellaOps.ReleaseOrchestrator.Environment/Models/Target.cs. The target has a polymorphic ConnectionConfig, a HealthStatus (not a separate Health of type HealthStatus plus a TargetState — there is no TargetState enum), and an optional inventory snapshot. There is no Labels property on Target (labels live on Agent).

public sealed record Target
{
    public required Guid Id { get; init; }
    public required Guid TenantId { get; init; }
    public required Guid EnvironmentId { get; init; }
    public required string Name { get; init; }
    public required string DisplayName { get; init; }
    public required TargetType Type { get; init; }
    public required TargetConnectionConfig ConnectionConfig { get; init; }
    public Guid? AgentId { get; init; }               // null for in-process / agentless dev paths
    public required HealthStatus HealthStatus { get; init; }
    public string? HealthMessage { get; init; }
    public DateTimeOffset? LastHealthCheck { get; init; }
    public DateTimeOffset? LastSyncAt { get; init; }
    public InventorySnapshot? InventorySnapshot { get; init; }
    public DateTimeOffset CreatedAt { get; init; }
    public DateTimeOffset UpdatedAt { get; init; }
}

// Serialized as JSON strings (JsonStringEnumConverter). Note: EcsService (not "ECSService"),
// and AnsibleHost was added in the topology-migration refresh.
public enum TargetType
{
    DockerHost = 0,
    ComposeHost = 1,
    EcsService = 2,
    NomadJob = 3,
    SshHost = 4,
    WinRmHost = 5,
    AnsibleHost = 6
}

public enum HealthStatus { Unknown = 0, Healthy = 1, Degraded = 2, Unhealthy = 3, Unreachable = 4 }

Agent

Source: StellaOps.ReleaseOrchestrator.Agent/Models/Agent.cs. Capabilities are a closed AgentCapability enum plus a free-string ExtendedCapabilities set (used by Sprint 017 EPF deploy plugins such as build.docker, deploy.compose); the union of the two is the authoritative capability set for the task matcher. Status is AgentStatus (Pending/Active/Inactive/Stale/Revoked), not a three-value AgentState (Online/Offline/Degraded), and the heartbeat timestamp is nullable.

public sealed record Agent
{
    public required Guid Id { get; init; }
    public required Guid TenantId { get; init; }
    public required string Name { get; init; }
    public required string DisplayName { get; init; }
    public required string Version { get; init; }
    public string? Hostname { get; init; }
    public required AgentStatus Status { get; init; }
    public required ImmutableArray<AgentCapability> Capabilities { get; init; }
    public ImmutableArray<string> ExtendedCapabilities { get; init; } // EPF deploy-plugin capability names
    public bool RequiresRecapabilities { get; init; }
    public ImmutableDictionary<string, string> Labels { get; init; }
    public string? CertificateThumbprint { get; init; } // mTLS
    public DateTimeOffset? CertificateExpiresAt { get; init; }
    public DateTimeOffset? LastHeartbeatAt { get; init; }
    public AgentResourceStatus? LastResourceStatus { get; init; }
    public DateTimeOffset? RegisteredAt { get; init; }
    public DateTimeOffset CreatedAt { get; init; }
    public DateTimeOffset UpdatedAt { get; init; }
}

public enum AgentStatus { Pending = 0, Active = 1, Inactive = 2, Stale = 3, Revoked = 4 }

public enum AgentCapability
{
    Docker = 0, Compose = 1, Ssh = 2, WinRm = 3, VaultCheck = 4,
    ConsulCheck = 5, Ecs = 6, Nomad = 7, Ansible = 8
}

Database Schema

Reconciled 2026-05-30 against the embedded SQL migrations under StellaOps.ReleaseOrchestrator.{Environment,Agent,Persistence,Scripts}/Migrations/. The orchestrator owns three PostgreSQL schemas, not a single release schema. The previously documented tables release.components, release.approvals, release.workflows, release.workflow_runs, release.deployment_jobs, release.evidence_packets, and release.plugins do not exist; their real homes are in the release_orchestrator schema (or do not exist at all).

PostgreSQL pool-budget precedence

Release Orchestrator owns seven independent long-lived PostgreSQL pools: main persistence, environment, agent, identity, scripts, KEK, and plugin registry. Limiting only the main pool does not protect a shared platform database. The default aggregate ceiling is therefore 28 connections (8 main, 8 environment, 4 agent, and 2 each for identity, scripts, KEK, and plugin registry), with a 60-second idle lifetime and zero warm auxiliary connections.

ReleaseOrchestrator:Postgres:MaxPoolSize remains the explicit main-pool override. EnvironmentMaxPoolSize, AgentMaxPoolSize, and AuxiliaryMaxPoolSize allocate the other shares; the auxiliary share applies uniformly to identity, KEK, and plugin registry. Main and auxiliary idle lifetimes use ConnectionIdleLifetimeSeconds and AuxiliaryConnectionIdleLifetimeSeconds. Explicit Scripts:Postgres pool settings take precedence for the scripts catalog.

These settings override larger limits carried by a shared connection string. Without a module setting, the bounded defaults above apply; the shared PostgresOptions default of 100 must never silently expand an operator budget. Application names and runtime connection defaults continue to be normalized by PostgresConnectionStringPolicy so pg_stat_activity retains per-pool attribution.

release schema (topology / agent registry)

Owned by the Environment + Agent libraries through their embedded v1 baselines and forward-only follow-ups. Environment may create the topology-catalog form of release.agents first; Agent migration 002_environment_first_agent_shape_convergence.sql then adds the complete operational/certificate/heartbeat column set. The reverse startup order is also supported, so concurrent service startup converges to the same table shape.

Tenant-owned topology tables, including release.environments, release.regions, release.targets, and release.infrastructure_bindings, must use the current tenant id on primary-key, natural-key, scope/list, and delete/update paths. Environment names and order indexes, region names and sort orders, and target names are tenant-local; lookup methods such as GetByNameAsync, GetByOrderIndexAsync, ListOrderedAsync, ListByEnvironmentAsync, and ListByAgentAsync must predicate on the active tenant instead of relying on globally unique ids or database session state alone. The infrastructure binding store also rejects creates whose model tenant differs from the active tenant, so registry/vault/settings-store bindings cannot be read or removed across tenants by guessing a binding id, scope, or integration id.

The standalone WebApi resolves that active tenant through ReleaseEnvironmentIdentityAccessor. Ordinary principals stay claim-first. For global-admin Console requests, Sprint 20260530.033 allows the canonical X-StellaOps-TenantId header to select the tenant without re-issuing the token; this is the Release Orchestrator side of ADR-023. The live Sprint 034 forcing function created dev environments in both e2e-lab and customer-dev-lab through the same admin token and verified a tenant-scoped list only returns that tenant’s row.

Topology setup compatibility aliases

ReleaseOrchestrator is the sole HTTP, mutation, and migration owner for the deployment-topology compatibility families /api/v1/regions, /api/v1/environments, /api/v1/targets, /api/v1/agents, /api/v1/infrastructure-bindings, and /api/v1/pending-deletions. These aliases moved from Concelier on 2026-07-11 without requiring Web client URL changes; both Router configurations now target ReleaseOrchestrator and preserve authorization headers. Every relocated handler requires tenant context plus orch:read for reads or orch:operate for mutations.

The environment and agent Router rules are intentionally narrow. The environment alias covers only list/create, readiness, rename, and deletion request shapes; the agent alias covers the exact list and deletion-request routes, and the integration compatibility rule covers only deletion requests. This prevents the compatibility layer from shadowing other services’ environment, agent, and integration contracts. Agent/integration rename aliases were not carried over because the environment library cannot implement those owner-specific actions. Their deletion-request aliases remain addressable but fail closed with the rest of the deletion lifecycle. Region/environment/target rename paths remain addressable for compatibility but return 503 topology_rename_unavailable; the former implementation changed only the display label while reporting the immutable name as changed.

Pending deletion is fail-closed. Request and confirmation aliases return 503 topology_deletion_unavailable until cascade discovery and idempotent execution are implemented. ReleaseOrchestrator does not start the historical worker that marked a confirmed deletion completed without deleting its target. Existing pending records remain readable and cancellable under tenant scope.

release_orchestrator schema (release truth + deployment engine)

Owned by the Persistence library; auto-migrated by the standalone WebApi on startup.

Audit content hashing is forward-only and epoch-aware. Hash v3 is the first storage-round-trip- stable epoch for new rows; v1/v2 rows that cannot reproduce after PostgreSQL timestamp/JSON normalization retain structural coverage and return honest partial deep-verification coverage, not a false tamper verdict. Migration 018_audit_hash_epoch_guard.sql makes the persisted hash_version immutable after insert. The exact endpoint semantics and threat boundary are documented in Audit Trail.

scripts schema (release-script registry)

Owned by the Scripts library (001_initial.sql); auto-migrated by the WebApi.

Release Truth Persistence

The standalone WebApi auto-migrates the release_orchestrator schema on startup. The release-truth DDL now lives in the consolidated baseline 001_v1_releaseorchestrator_baseline.sql (folded in from the pre-1.0 003_release_truth.sql, which is archived under Migrations/_archived/pre_1.0/mig061/ and excluded from embedding; live forward migrations continue at 002–017). It provides the following durable tables for the compatibility endpoint cutover — a focused subset of the full release_orchestrator table list enumerated above:

Runtime rule (updated 2026-07-12): protected promote/deploy paths must not infer success from seed/demo data. Release and approval HTTP compatibility paths read and mutate durable release truth rows, and approval decisions are idempotent per actor so one approver cannot inflate quorum by repeating the same decision. Missing approval quorum, WAITING/empty/advisory/skipped gate truth, non-passing gates, missing verified evidence, unsigned/unbound protected decision evidence, or evidence hash mismatch returns a typed blocked/conflict response. The only override that can clear a gate is the dedicated GateException decision: release gate-bypass scope, a verified operator envelope, a 1-365 day expiry, durable reason/risk acknowledgement, evidence reference, and an explicit missingEvidence list are mandatory. Conditional/permanent exceptions are rejected until their conditions have a deterministic enforcement contract. Deployment requests are persisted as queued/pending deployment state and only executor completion may advance them to deployed/completed.

Weekly dashboard projection (updated 2026-07-18): authenticated release:read callers can read GET /api/v1/release-orchestrator/releases-this-week (plus the unversioned compatibility alias). The tenant-scoped response uses a Monday-based UTC calendar window. startInclusiveUtc is inclusive, endExclusiveUtc is the following Monday, and asOfUtc is the effective upper bound: future-dated rows inside the nominal calendar window are not reported. Shipped rows are anchored to stored deployedAt. Blocked rows are current failed releases updated in the observed window or the latest in-window approval when its stored gate result, quorum, rejection, expiry, or missing gate pass says it is blocked. Failed gate rows carry the stored gate id/name/message and a stored ruleId, policyId, or ruleName when one exists; legacy failures with no recorded reason explicitly return reasonState=not_recorded.

The response reports exact in-window totals and caps every emitted item/summary list at 50 with truncated and maxItems. Ordering is deterministic (event time descending, then release id ordinal; ownership summaries use count descending, then id ordinal). Ownership is never derived from release/service names or the promotion requester. Owner subject ids come only from persisted release createdBy and active release Owner user grants; team ids come only from active release Owner group grants. Missing ownership is explicit as state=unassigned. Owner-grant reads cover every distinct candidate in the current partial UTC week so owner/team and unassigned totals remain exact; the response lists remain hard bounded.

shippedHistory is the bounded trend contract for the same dashboard answer. It always returns eight contiguous Monday-based UTC buckets in chronological order, ending with the current week. Counts come only from persisted release deployedAt; timestamps before the oldest bucket and after asOfUtc are excluded, and the current bucket is marked partial=true. The source is stated explicitly as release.deployed_at. No blocked-history series is claimed because the current release/approval projection does not retain an immutable blocker event history; deriving one from mutable latest status would rewrite past weeks. The dashboard consumer must preserve these honesty states rather than inventing a reason, owner, team, or blocked trend.

Blocked-release explanation projection (updated 2026-07-18): authenticated release:read callers can read GET /api/v1/release-orchestrator/release-blockers (plus the unversioned compatibility alias). The endpoint answers the current all-release question, not the calendar-week rollup: PostgreSQL selects only the latest approval per release, returns failed releases plus approvals with blocking or advisory gate truth, computes the exact tenant total, and emits at most 50 rows ordered by stored observation time descending then release id ordinal. The application does not load the tenant’s unbounded release/approval history.

Each item distinguishes block from warn. Causes carry the stored gate id, name, status, message, and stored rule identity when gate details contain ruleId, policyId, ruleName, packId, or Policy Engine cveMatches[].ruleName. The latest gate decision contributes its stored policy id/mode when present. Approval quorum, rejection, expiry, missing gate-result truth, and failed release state are named as separate stored-state causes. Cause lists cap at ten with causesTruncated; absent or malformed legacy rule detail remains ruleIdentity.state=not_recorded rather than throwing or guessing.

Remediation linkage is deliberately honest. Only an explicit stored gate-detail link (fixLink, remediationLink, fixUrl, remediationUrl, remediationUri, or actionUrl) becomes remediation.state=recorded; otherwise the response says not_recorded. Release Orchestrator does not call or search Policy again while serving this projection. Owner subjects come only from the stored release creator and active release Owner user grants; teams come only from active release Owner group grants. Missing ownership is unassigned. This projection does not store or claim digest-scoped service identity or service ownership. Stable Policy rule-to-remediation linkage remains a cross-module successor; Web must preserve the explicit absence states until that contract exists.

Developer self-service rule (updated 2026-07-12): developer principals are not authorized by broad release:write alone. Release and deployment list, detail, and release-component reads are server-filtered by the caller’s scoped ownership and environment claims; a contributor token without ownership claims receives an explicit missing_ownership_claims zero-data projection rather than tenant-wide rows labelled as personal data. Non-developer principals, including admin-family roles, are not silently converted into developer principals: their list reads remain tenant-scoped and still require the authenticated tenant plus release:read; the role is never a scope bypass. Service and repository ownership uses canonical exact identity, while image namespaces use a path-boundary match, preventing neighboring names or repositories with the same leaf from matching. Deployment read filters advertised by the Web client (serviceIds, initiating developer, source type, and current/candidate digest) are applied by the backend, and the compatibility projection persists those developer-update fields. POST /api/v1/release-orchestrator/deployments applies service/environment/digest ABAC when the caller is a developer principal or carries developer ABAC claims. Direct deployment requires owned serviceIds/repositoryIds or matching image namespace, allowed environmentIds or maxEnvironmentType, directUpdateAllowed=true, and an immutable sha256: package digest. POST /api/v1/release-orchestrator/approvals/developer-requests uses the same service/environment/digest checks without requiring directUpdateAllowed, then persists an approval row with SourceType=developer_update, service id/name, candidate digest, target environment id, actor, and audit details. If a developer has a newly built digest that is not yet attached to an existing release row, the request endpoint derives a stable developer-update-{service} release key and still records the service, digest, target environment, and actor for approval routing and replay. GET /api/v1/approvals/{id}/replay returns the v2 approval replay status from the persisted decision digest and evidence packet so the approval-detail UI can verify replayability without falling back to an unimplemented compatibility route.

Deployment execution reality (updated 2026-07-16): the standalone WebApi drives HTTP deploy requests through LibraryDeploymentRequestStarter into the canonical library DeployOrchestrator. POST /deployments still returns immediately after the library job and compatibility read projection are persisted, while the starter keeps a dedicated deployment scope alive until the library engine reaches terminal state. IDeploymentJobStore is backed by release_orchestrator.deployment_jobs in production WebApi DI, so deploy-engine job state survives service restarts instead of depending on the process-local in-memory store. Task-level updates use row-locked UpdateTaskAsync writes so parallel blue-green target completion cannot overwrite sibling task state. Terminal DeploymentJob state is projected back into release_orchestrator.deployments, including the real job start/completion times and per-target status/progress/error while preserving the real target ids, names, agent ids, strategy evidence, and rollback status. The current successful job wire value is succeeded; persisted legacy completed values are equivalent success aliases for target rollups, rollback eligibility, list filters, and filtered totals. A successful row promotes its canonical candidate digest to currentDigest; the read normalization applies the same rule to pre-fix successful rows while failed or incomplete rows retain their previous current digest. Progress measures finished target work independently of outcome, so a failed deployment may truthfully be 100% finished without becoming successful. Each terminal mirror also upserts one deterministic terminal event and recomputes total duration, average target duration, and target success rate; repeated mirrors do not duplicate the event or target logs. Read paths apply the same deterministic projection to legacy rows whose serialized rollups, operation flags, terminal event, metrics, or successful current-digest subject are stale, without rewriting their stored outcome. The WebApi composition root registers the library deployment strategy factory and real artifact generator (ComposeLockGenerator, VersionStickerGenerator, DeploymentManifestGenerator) against read-through release-truth adapters for IReleaseManager and IPromotionManager. Those adapters map existing string-based release, component, environment, and approval ids to stable GUIDs, pass component ImageRef into the compose writer’s repository config, and fail closed for unsupported mutation methods; release and approval HTTP endpoints remain the mutation authority. Deployment and promotion domain events are persisted through PostgresDomainEventPublisher into release_orchestrator.domain_events with tenant, aggregate, occurrence time, and JSON payload metadata. Live verification now covers library-engine rolling/canary/blue-green dispatch through real Testcontainers sshd targets, dev in-process SSH/WinRM paths, and the target-resident StellaOps.Agent.Host ComposeHost and DockerHost paths through the mTLS-polling agent-runtime contract. StellaOps.Agent.Host can also compose SSH and WinRM capabilities for out-of-process task execution, agent registration now persists through PostgresAgentStore into release.agents, and registered agents receive CA-signed client certificates from the file-backed internal agent CA; full live proof for out-of-process SSH/WinRM remains tracked in Sprint 20260506_002 alongside compose bootstrap.

Normal deployments and per-target retries use the same authoritative in-flight mirror: while the engine operation is active, changed non-terminal job/task state is sampled from IDeploymentJobStore and projected into release_orchestrator.deployments. Terminal evidence and target-log append remain on the terminal path. This keeps API and CLI watch consumers on real target status/progress instead of leaving a retried target at compatibility pending until completion.

Deployment Share Grants (Sprint 20260608_001)

See ADR-027 for the decision record (alternatives rejected, the seven invariants, Production 4-eyes).

An operator on the Console deployment monitor (/releases/deployments/{id}) can hand a colleague a share link so they can watch — or take over — a single live deployment. The link carries an opaque grantId (dsg_<32 hex>), not a credential: opening it does nothing privileged. The recipient must sign in through the existing OIDC flow and POST .../share/{grantId}/accept; only the accepted subject is ever elevated, scoped to that one resource, time-boxed, revocable, and audited. There is no new global scope and no Authority/identity-envelope change — delegation is enforced inside ReleaseOrchestrator against the request principal.

Capability levels

The platform never issues a half-elevated grant. An OperateDeploy grant satisfies a required View; a View grant satisfies only View.

Endpoint group

Mapped under both /api/release-orchestrator and /api/v1/release-orchestrator (ShareGrantEndpoints); the whole group requires authentication + tenant + release:read. Elevated decisions are enforced in DeploymentShareGrantService against the request principal.

Resource-scoped authorization fallback

The existing operate endpoints (/deployments/{id}/pause|resume|cancel|rollback, target retry) and initiate endpoints (/releases/{id}/deploy|promote|rollback) keep their native scope check unchanged and add a fallback via ShareGrantAccessGuard: the action is authorized when the caller holds the native scope (or is a global admin) OR has an active, accepted OperateDeploy grant for this exact resource. The grant for deployment D1 never authorizes D2; a deploy-grant for (releaseR, envStage) never authorizes (releaseR, envProd) nor another release. A privileged action authorized via a grant emits a release.share.used audit event (real acting subject + grantId + capability + route + resource ids) and atomically consumes a use; the native-scope path emits no share.used. (Implementation: the per-route elevated RequireAuthorization moved into the handler so the OR-fallback is reachable — ASP.NET authorization middleware runs before the handler. The native check is byte-identical, so the native path is not weakened.)

Lifecycle and persistence

Three tables in the release_orchestrator schema, added by the embedded, idempotent migration 003_deployment_share_grants.sql (CREATE SCHEMA/TABLE/INDEX IF NOT EXISTS; CHECK-constrained enum columns; JSONB scopes_granted/subjects; FK-cascade on acceptances/approvals), auto-applied by the existing AddStartupMigrations(schema=release_orchestrator) on WebApi startup: deployment_share_grants, deployment_share_grant_acceptances, deployment_share_grant_approvals. Default lifetime 24h (cap 30 days), optional maxUses budget (consumed atomically), immediate revoke, auto-revoke on terminal deployment state, plus a periodic DeploymentShareGrantExpirySweeper that transitions expired grants to Expired. Authorization is additionally denied when the target deployment is already terminal (covers engine-driven succeeded/failed completion outside the HTTP path).

release.share.{created,toggled,accepted,approved,revoked,used} audit actions (AuditModules.Release, resource type deployment_share_grant) provide full attribution.

Gate Types

Reconciled 2026-05-30; gate inventory corrected 2026-07-12. GateResult.GateType is a free-form string, not an enum. Four built-in gate plugins are registered into the BuiltInGateProviderRegistry and therefore into the plugin-backed promotion path (StellaOps.ReleaseOrchestrator.WebApi/Program.cs:1587-1603, registration order preserved):

1. SbomReadinessGatePlugin — runs first; blocks promotion until the SBOM is ready (see §“SBOM readiness gate”).
2. ApprovalPolicyGatePlugin — approval-policy, wrapping PolicyDrivenGateDecisionEvaluator (also performs the CVE-aware Policy.Engine call).
3. ReachabilityGatePlugin — reachability-gate, wrapping ReachabilityGateEvaluator (sprint 20260531_054).
4. AttestationGatePlugin — deploy-time attestation gate on the live chain (sprint 20260704_006 / GDB-3).

The attestation gate keeps two identifiers deliberately separate. Its Attestor list query uses type=scan-result, the indexed meta.artifact.kind; the DSSE payload still carries the full in-toto predicate URI stellaops.io/predicates/scan-result@v1. A candidate is verified only when it is subject-bound to the component digest, its Attestor log status is included, and its persisted DSSE signature keyid matches a key currently published by Scanner’s JWKS surface. Absent, pending, failed, and unknown log statuses remain invalid even for a trusted signer. The caller client id authenticates the submission transport and is not a signing-key identity. Verification results also carry a stable evidenceGapCode; callers never infer a gap from prose. A bound entry whose log status is not included is transparency_anchor_failed, while no bound entry is attestation_missing, other invalid attestations are attestation_invalid, and transport/trust-service inability is verifier_unavailable. The gate preserves that code on each component and emits aggregate evidenceGapCount plus evidenceGaps details. A transparency-anchor gap is counted in advisory mode and denies in an existing blocking environment; a merely missing entry is never relabelled as an anchoring failure.

The registration runs regardless of the UsePluginRegistry flag, so the built-in gate surface is the same either way; the flag only selects PluginBackedGateDecisionEvaluator vs the legacy CompositeGateDecisionEvaluator. Additional gate-result types — attestation, vex, reachability, security-gate — surface in policy-gate simulation and decision evidence (StellaOps.ReleaseOrchestrator.PolicyGate). The conceptual gate categories below describe the intended promotion-gate taxonomy, which is broader than the four registered built-ins. (The prior note said “only the two named built-ins are registered” — it under-counted, and contradicted this dossier’s own SBOM-readiness section.)

Plugin System (Three-Surface Model)

Note (reconciled 2026-05-30): The interface snippets below are illustrative contracts and do not match the actual type names/signatures. The real plugin capability interfaces live in StellaOps.ReleaseOrchestrator.Plugin/Capabilities/: IIntegrationConnectorCapability, IScmConnectorCapability (note Scm, not SCM), IRegistryConnectorCapability, and IStepProviderCapability. The workflow-level step contract is StellaOps.ReleaseOrchestrator.Workflow/Steps/IStepProvider and exposes Type, DisplayName, Description, ConfigSchema, Capabilities (StepCapabilities), Category (StepCategory), ExecuteAsync, and ValidateConfigAsync — there is no RollbackAsync or StepExecutionCharacteristics on it. A separate IRegistryConnector (in …Release/Registry/) handles digest resolution for release creation.

Plugins contribute through three surfaces:

1. Manifest (Static Declaration)

# plugin-manifest.yaml
name: github-integration
version: 1.0.0
provider: StellaOps.Integration.GitHub.Plugin
capabilities:
  integrations:
    - type: scm
      id: github
      displayName: GitHub
  steps:
    - type: github-status
      displayName: Update GitHub Status
  gates:
    - type: github-check
      displayName: GitHub Check Required

2. Connector Runtime (Dynamic Execution)

public interface IIntegrationConnector
{
    Task<ConnectionTestResult> TestConnectionAsync(CancellationToken ct);
    Task<HealthStatus> GetHealthAsync(CancellationToken ct);
    Task<IReadOnlyList<Resource>> DiscoverResourcesAsync(string resourceType, CancellationToken ct);
}

public interface ISCMConnector : IIntegrationConnector
{
    Task<CommitInfo> GetCommitAsync(string ref, CancellationToken ct);
    Task CreateCommitStatusAsync(string commit, CommitStatus status, CancellationToken ct);
}

public interface IRegistryConnector : IIntegrationConnector
{
    Task<string> ResolveDigestAsync(string imageRef, CancellationToken ct);
    Task<bool> VerifyDigestAsync(string imageRef, string expectedDigest, CancellationToken ct);
}

3. Step Provider (Execution Contract)

public interface IStepProvider
{
    StepExecutionCharacteristics Characteristics { get; }
    Task<StepResult> ExecuteAsync(StepContext context, CancellationToken ct);
    Task<StepResult> RollbackAsync(StepContext context, CancellationToken ct);
}

public sealed record StepExecutionCharacteristics
{
    public bool IsIdempotent { get; init; }
    public bool SupportsRollback { get; init; }
    public TimeSpan DefaultTimeout { get; init; }
    public ResourceRequirements Resources { get; init; }
}

Invariants

1. Release identity is immutable — Once created, a release’s component digests cannot be changed. Create a new release instead. If an updated artifact digest is detected for a non-draft release, Release Orchestrator returns a structured successor-version suggestion and creates the next draft version through /api/v1/release-orchestrator/releases/{id}/versions/from-artifact-update.

2. Promotions are append-only — Promotion state transitions are recorded; no edits or deletions.

3. Evidence packets are sealed — Evidence is cryptographically signed and stored immutably.

4. Digest verification at deploy time — Agents verify image digests at pull time; mismatch fails deployment.

5. Separation of duties enforced — Requester cannot be sole approver for production promotions.

6. Workflow execution is deterministic — Same inputs produce same execution order and outputs.

7. Share grants are account-bound and resource-scoped — A deployment share link carries an opaque grantId, not a credential; it elevates only the authenticated subject who accepts it, only for the one deployment (or release+environment) it targets, only within its tenant, only until expiry/revoke/terminal state, and (for Production elevation) only after a second authorized person — who must differ from the issuer — approves it. Every grant-backed privileged action is audited with the real acting subject. See ADR-027.

Error Handling

• Transient failures — Retry with exponential backoff; circuit breaker for repeated failures
• Agent disconnection — Mark agent offline; reassign pending tasks to other agents
• Deployment failure — Automatic rollback if configured; otherwise mark promotion as failed
• Gate failure — Block promotion; require manual intervention or re-evaluation

Observability

StellaOps.ReleaseOrchestrator.Deployments is the deployment lifecycle meter emitted by the Web API deployment endpoints. The broader promotion/workflow instrument names below remain design targets until their call sites register matching meters or activity sources.

Metrics

• release_orchestrator_deployment_lifecycle_total{tenant,operation,status} - Counter for deployment create, pause, resume, cancel, rollback, and retry target operations.
• release_orchestrator_deployment_lifecycle_duration_seconds{tenant,operation,status} - Histogram for the same deployment lifecycle operations.
• release_promotions_total — Counter by environment and outcome
• release_deployments_duration_seconds — Histogram of deployment times
• release_gate_evaluations_total — Counter by gate type and result
• release_agents_online — Gauge of online agents
• release_workflow_steps_duration_seconds — Histogram by step type

Traces (proposed)

• promotion.request — Span for promotion request handling
• gate.evaluate — Span for each gate evaluation
• deployment.execute — Span for deployment execution
• agent.task — Span for agent task execution

Logs

• Structured logs with correlation IDs
• Promotion ID, release ID, environment ID in all relevant logs
• Sensitive data (secrets, credentials) masked

Security Considerations

Agent Security

• mTLS authentication — Agents authenticate with CA-signed certificates
• Short-lived credentials — Task credentials expire after execution
• Capability-based authorization — Agents only receive tasks matching their capabilities
• Heartbeat monitoring — Detect and flag agent disconnections

Secrets Management

• Never stored in database — Only vault references stored
• Fetched at execution time — Secrets retrieved just-in-time for deployment
• Short-lived — Dynamic credentials with minimal TTL
• Masked in logs — Secret values never logged
• Compose push mode — Environment refs are resolved in orchestrator memory immediately before the immutable lock is generated; generated content and task diagnostics must not echo values.
• Compose pull mode — Network-backed refs fail closed until the edge assignment contract can deliver scoped references and resolve them without returning plaintext to the control plane.

Vault auth references

Agentless SSH and deployment secret resolution supports Vault-backed auth references in this form:

authref://vault/<logical-path>#<key>

At execution time Release Orchestrator resolves the reference against the configured HashiCorp Vault KV v2 backend:

${VAULT_ADDR}/v1/secret/data/<logical-path>

The named <key> is read from the returned data.data object. For example, authref://vault/partner-test#APP_SERVER_SSH_PRIVATE_KEY reads key APP_SERVER_SSH_PRIVATE_KEY from KV path secret/data/partner-test.

Runtime requirements:

• VAULT_ADDR must point at the on-prem Vault endpoint, for example https://vault.example.com:8200.
• VAULT_TOKEN must be supplied through the service runtime environment or another operator-controlled secret injection path.
• Missing VAULT_ADDR, missing VAULT_TOKEN, non-success Vault responses, malformed JSON, absent paths, and absent keys fail closed before the deployment connection is attempted.
• Secret values are returned only to the in-memory executor path and must not be emitted to structured logs, evidence packets, or error messages.

Plugin Sandbox

• Resource limits — CPU, memory, timeout limits per plugin
• Capability restrictions — Plugins declare required capabilities
• Network isolation — Optional network restrictions for plugins

Performance Characteristics

• Promotion evaluation — < 5 seconds for typical gate evaluation
• Deployment latency — Dominated by image pull time; orchestration overhead < 10 seconds
• Agent heartbeat — 30-second interval; offline detection within 90 seconds
• Workflow step timeout — Configurable; default 5 minutes per step

Implementation Roadmap

Runtime Observation (Unified — Sprint 20260512_020)

The deployment agent (StellaOps.Agent.Host) is the single host-side surface for both deployment dispatch AND runtime observation. The historical “host agent” track in src/RuntimeInstrumentation/StellaOps.Agent.{Linux,Windows}/ is preserved for reference but is no longer the deployment target — its purpose has been absorbed into the deployment agent so a single binary, provisioned once on .10/.20 with mTLS + cert rotation, performs both.

Contracts

The runtime observation DTOs (RuntimeEvent, RuntimeEventEnvelope, AdmissionDecision, RuntimeContractVersions) live in the neutral library src/ReleaseOrchestrator/__Libraries/StellaOps.Runtime.Contracts/ under namespace StellaOps.Runtime.Contracts. They were relocated out of StellaOps.Zastava.Core.Contracts during the Zastava unbundling. Schema STRING identifiers on the wire (zastava.runtime.event@v1, zastava.admission.decision@v1) are preserved for backward compatibility with deployed Scanner.WebService consumers; only the .NET namespace moved.

Downstream surfaces

Runtime evidence is dual-written to two downstream stores:

All three tables now carry a nullable release_id TEXT column (sprint 020 migrations signals/004 and findings/018) so runtime evidence can be tied directly to a deployed release without a cross-schema join. The column is nullable: legacy/v1 emitters (which have no release context) continue to write valid rows.

Capability handler

runtime.observe.container is an IAgentCapability implementation registered in CapabilityRegistry alongside winrm.execute, ssh.execute, winrm.container, ssh.container. On task execution it:

1. Reads docker inspect <containerId> against the host runtime.
2. Constructs a RuntimeEvidenceEnvelope populated from the inspect JSON + the task’s metadata-borne releaseId.
3. Validates via AgentCoreValidation.
4. Dual-writes through HttpRuntimeEvidencePublisher, which reuses the AgentRuntimeHttpHandler mTLS pipeline established by Agent.Host for the heartbeat client (cert + rotation share the same trust store).

Phase 5 honesty boundary: the capability launched in stub mode — it emitted frames: [] and the trace was minimum-viable evidence (“container observed running, digest-gate passed”). Real eBPF stack-frame capture for reachability data landed in Sprint 20260513_022 (phases A–H all DONE; archived under docs-archive/implplan/SPRINT_20260513_022_RuntimeInstrumentation_ebpf_stack_frame_capture.md): PHASE-B extended Agent.Core contracts with UserStack/KernelStack/StackCaptureMode, PHASE-C added Tetragon kprobe + stack-trace event mapping, PHASE-D added the Linux agent symbol resolver, PHASE-E activated captureMode: ebpf-frames on the Agent.Docker capability, PHASE-F wired reachability scoring in Scanner, and PHASE-G/H closed live integration on the lab Linux VM plus operator docs

• offline-kit prep. The container-inspect mode remains available as the fallback when the Tetragon bridge is disabled.

References

• Product Vision
• Architecture Overview
• Full Orchestrator Specification (archived advisory)
• Competitive Landscape
• Sprint 20260512_020 — Runtime Observation Unification (archived)
• Zastava architecture (retired; contracts relocated)