Release Orchestrator Architecture

Dated R15-RO-1 adoption and owner proof (2026-10-21 UTC). The serving local image is e114b8b9815d0922b31e8eaaed5893000377b5ef12bbd21212ee9248899abf5d from clean source ff9b51d3903eafda52c975e230cfe69caf79ede6. One reviewed image-only update preserves the full nine-file RRO1 recovery profile. Actual source/payload/profile checks and one real existing-packet re-verification pass; the subsequent chain read reports its exact Decision as verified. The selected Scan archive export is unavailable; source indicates retained local signed components that need a bounded owner retrieval contract. The remaining stages/healthy-estate, CA-14 and PG-1 gates stay open. The chain contract supplies read-only re-observation; the receipt records exact evidence and limits.

Dated RRO-1 live evidence (2026-10-21). The shared owner image built from clean 9a135ccc084f7c0e40671e270b309b8d70a1a2c4 carries the unchanged 19 reviewed inputs. Real Gateway controls distinguish ordinary inactive-tenant 403, retained 200 and invalid-mode 400/400; the complete seven-tenant owner-reference capture passes the unchanged CMP consumer. This is dated rollout/census evidence, not future VM authority, capacity qualification, a reservation or CA acceptance. Receipt.

Explicit retained inventory source contract (ROE-2, 2026-10-21). Verified against source commit de7b64acb36b0a2b5f1bfcdc1bc8a4270d248396. Only environment-list and nested target-list GETs accept X-StellaOps-Retained-Tenant-Read: true, with canonical validated global-admin identity and the actual route policy checked before DI on HTTP and Router dispatch. The separate local catalog reader verifies active/suspended registration and non-tombstone state before UUID selection; unavailable/malformed/timeout results remain faults. Request-only identity does not populate ordinary caches or overrides; ordinary UUID/cache, explicit override, Agent/background and existing deployment string reads are preserved. The manual API contract gives exact headers, routes, refusals, re-verification commands and test limits. The 139-case source cohort is not deployed/API/census/VM acceptance; the ROE-2 receipt records the evidence.

Tenant refusal source contract (ROE-1, 2026-10-21). Verified against source commit e757db3361f0f8434a44e44832aa81ad065f69ca. An answered catalog lookup that refuses a slug or returns an empty UUID produces 403 with error: tenant_required at the mapped environment/target endpoint boundary. Its completed RequestDelegate is wrapped, so direct HTTP, Router frames and handler-service activation share the same sanitized response. Missing request tenancy retains the existing RequireTenant result; directory failure/timeout and unrelated authorization exceptions remain faults. Ordinary resolution/cache and UUID pass-through are unchanged. Re-verify with pwsh ./tools/scripts/test-targeted-xunit.ps1 -Project src/ReleaseOrchestrator/__Tests/StellaOps.ReleaseOrchestrator.Integration.Tests/StellaOps.ReleaseOrchestrator.Integration.Tests.csproj -Class "*EnvironmentTenantRefusalEndpointTests" -BuildProjectReferences. This source result is not a deployed probe or retained-inactive-read capability; see the source receipt.

Source ownership, 2026-10-03. PCD-253 moved the two uncompiled Agent operations/resilience test files into src/ReleaseOrchestrator/__Tests/StellaOps.Agent.Core.Tests/. The current suite executes bootstrap installers for Linux/Windows/Docker, one-time token consumption, configuration apply/rollback/drift, Doctor composition and resilience behaviors. The adoption exposed and fixed three local defects: trend analysis requires at least one historical sample beyond its three-sample recent window; deleted state returns no value; circuit-breaker opening and expiry use the same injected clock. The obsolete Scripts test draft was frozen without reviving its excluded APIs. These are local library/test results, not deployed agent acceptance. See the per-file source disposition.

Technical architecture specification for the Release Orchestrator — Stella Ops Suite’s central release control plane for non-Kubernetes container estates.

Verified against commit 942a18727c (2026-09-26, DOC-2 of SPRINT_20260926_001). Re-verify by reading the evidence paths listed in the Evidence section.

Status: active development. Hosts: release-orchestrator (WebApi, image stellaops/release-orchestrator), release-orchestrator-agent-ca-init (one-shot agent-CA initializer, same image) and agent-core (target-resident deployment agent, built from StellaOps.Agent.Host). All three are in the dev estate (devops/compose/docker-compose.stella-services.yml) and the release bundle (devops/release/bundle/docker-compose.yml).

Implementation reality. The WebApi’s compiled closure is the Agent, Agent.Contracts, DecisionArtifact, Deployment, Environment, Federation, IntegrationHub, Persistence, Plugin, Promotion, Release, Scripts, Storage and Topology.Contracts libraries. Seven deployment target types exist (DockerHost, ComposeHost, SshHost, WinRmHost, EcsService, NomadJob, AnsibleHost). The WebApi owns environment, target, freeze-window, scripts, regional-federation, release, approval, deployment and gate-decision surfaces; Gateway routes those public operations to this host and Platform neither proxies nor hosts them. The Compliance, Evidence, EvidenceThread, Observability, Performance, PolicyGate, Progressive, Plugin.Sdk and SelfHealing libraries compile but are referenced only by their own tests, not by the WebApi; describe their behaviour as library capability, not as a running surface. There is no ReleaseOrchestrator workflow-engine library (the .Workflow assembly was deleted); /api/v1/workflows is not served by any host. The WebApi auto-migrates its own release, release_orchestrator, release_orchestrator_agent and scripts schemas in the RO database. Remaining gaps: some compatibility surfaces still need full execution-engine backing, and the bundled regional-federation host has no mutation execution provider.

Migration authority

The WebApi selects StellaOps.ReleaseOrchestrator.Persistence for its four family schemas, the shared credential-store chain for crypto, and the host-local plugin registry tables in platform (all in the RO database). Agent, Environment and Scripts libraries provide runtime services without SQL resources or their former optional migration registrations. Their predecessor SQL files are frozen unchanged under src/__Obsoleted/ReleaseOrchestrator/; explicit test-only fixtures preserve checksum, table-parity and upgrade-order coverage. Current Agent startup tests converge the consolidated owner before using its stores. The main runtime DAL remains separate in StellaOps.ReleaseOrchestrator.Storage. Its twenty predecessor SQL files are also frozen and preserved in the shared historical fixture assembly. The current migration authority keeps its SQL bytes, module and leaf identities under the canonical Persistence project name. Verified against source 9092f71860 and image source 20319246a4; the canonical persistence receipt records the SQL identities, tests and unchanged ledgers after the live restart. The earlier predecessor receipt covers the Agent, Environment and Scripts retirement.

Re-verify with the Agent, Environment and consolidated-persistence test projects, plus ReleaseOrchestratorProductionHostStartupMigrationTests in the integration suite.

Overview

The Release Orchestrator transforms Stella Ops Suite from a vulnerability scanning platform into a centralized, auditable release control plane. It sits between CI systems and runtime targets, governing promotion across environments, enforcing security and policy gates, and producing verifiable evidence for every release decision.

Ownership boundary: Platform owns the Stella Ops installation control plane (installation identity/settings, setup, tenants, and cross-service read projections). ReleaseOrchestrator exclusively owns deployment environments, targets, deployed image digests/inventory, promotions, and execution state. For RO-owned state, Platform maintains rebuildable read projections only. It does not expose an RO compatibility facade, host RO endpoints, run RO migrations, or read RO tables. Sharing one PostgreSQL server does not change database or schema ownership.

Build seam delivered (Q-10, 2026-08-25): contract-seam. The compile-time counterpart of the ownership boundary above: Platform and ReleaseOrchestrator remain DISTINCT services (one-family rejected; the RO/agent-core trust boundary stays). Platform’s sole RO project reference is now the producer-owned, zero-reference StellaOps.ReleaseOrchestrator.Topology.Contracts wire project; Platform compiles no RO implementation. Ruled direction: RO owns environment/script/deployment operations and drives deployment agents (pull or push); RO reads Platform’s non-release settings via Platform’s contract/API only. Execution: SPRINT_20260722_026 CM-6.

Owner-hosted surfaces and Platform topology contract

The public environment, scripts, and regional-federation paths remain stable at Gateway, but their target is this WebApi. Platform has no corresponding endpoint class, MVC application part, store, or proxy. STELLAOPS_FEATURE_FEDERATION and ReleaseOrchestrator:Federation:* belong to this host. When the feature is enabled, federation reads require the canonical platform:federation:read scope and mutations require platform:federation:write. The bundled single-region provider is read-only: mutation routes remain discoverable but return HTTP 503 because mutation execution is not installed or implemented; no operator-selectable execution provider exists today.

Platform’s one remaining runtime need is a tenant-scoped topology snapshot for its rebuildable read model. RO exposes GET /internal/v1/release-orchestrator/topology-snapshot, protected by release:read and authenticated tenant context. The producer-owned schema-v1 contract contains only wire DTOs and protocol constants and has no project or package references. The endpoint reads through RO service/manager boundaries, rejects any owner row outside the authenticated tenant, and emits deterministically ordered regions, environments, targets, and agents. Agent capabilities are the distinct ordinal union of built-in and extended capabilities; canonical Stale status is serialized as offline.

Platform enumerates only active tenant UUIDs from its own shared.tenants table and calls this owner endpoint directly using STELLAOPS_RELEASE_ORCHESTRATOR_URL. Each request carries a signed, tenant-bound two-minute identity envelope with only release:read. The internal route deliberately has no Gateway/Router mapping and no SQL fallback.

Verified against d3358c84319b0da835e944d70f360f7df526d9c5. Re-verify with rg -n "topology-snapshot|TopologySnapshotRead" src/ReleaseOrchestrator src/Platform, rg -n "ProjectReference Include=.*ReleaseOrchestrator" src/Platform -g '*.csproj', and rg -n "api/v1/federation" src/Router devops -g '*.json'.

Core Value Proposition

• Release orchestration — UI-driven promotion (Dev → Stage → Prod), approvals, policy gates, rollbacks
• Security decisioning as a gate — Scan on build, evaluate on release, re-evaluate on CVE updates
• OCI-digest-first releases — Immutable digest-based release identity
• Toolchain-agnostic integrations — Plug into any SCM, CI, registry, secrets system
• Auditability + standards — Evidence packets, SBOM/VEX/attestation support, deterministic replay

Design Principles

1. Digest-First Release Identity — A release is an immutable set of OCI digests, never mutable tags. Tags are resolved to digests at release creation time.

2. Pluggable Everything, Stable Core — Integrations are plugins; the core orchestration engine is stable. Plugins contribute UI screens, connector logic, step types, and agent types.

3. Evidence for Every Decision — Every deployment/promotion produces an immutable evidence record containing who, what, why, how, and when.

4. No Feature Gating — All plans include all features. Limits are only: environments, new digests/day, fair use on deployments.

5. Offline-First Operation — All core operations work in air-gapped environments. Plugins may require connectivity; core does not.

6. Immutable Generated Artifacts — Every deployment generates and stores immutable artifacts (compose lockfiles, scripts, evidence).

7. Truthful Runtime Placeholders — Production-root runtime placeholders must fail closed. Test doubles for registry connectors, event publishers, agent certificates, inventory collection, and progressive-delivery metrics live under adjacent test support with Test* names. Unconfigured agent/orchestrator and workflow-step bindings throw explicit configuration errors or return explicit failed/unavailable results instead of reporting success.

Runtime Fail-Closed Boundaries (2026-04-28)

• AgentManager.ExecuteTaskAsync is an active deployment dispatch surface. Production DI registers the IAgentTaskTransport boundary and the release_orchestrator_agent_task_transport readiness check. The default transport is UnavailableAgentTaskTransport, which is unhealthy and returns a failed TaskResult; deployment collectors must treat that as a failed task, not a successful deployment. When ReleaseOrchestrator:AgentTaskTransport:Enabled=true, the standalone WebApi registers MtlsPollingAgentTaskTransport and exposes /api/v1/release-orchestrator/agent-runtime/agents/{agentId}/tasks:poll plus /tasks/{taskId}/result. The runtime requires HTTPS configuration, mTLS client certificates bound to the registered agent certificate thumbprint, version agent-task-transport/v1, matching task ids, and a result signature verified with the mTLS certificate public key before task success is accepted. The mapped execution targets are Compose deploy (compose.up), Compose preflight/rollback, single-container Docker deploy (docker.run), SSH (ssh.execute), WinRM (winrm.execute), ECS service deploy (ecs.deploy), Nomad job deploy (nomad.deploy), and Ansible playbook deploy (deploy.ansible); unsupported deployment task types fail closed.
• Infrastructure binding connectivity checks are not satisfied by binding existence alone. InfrastructureBindingService.TestBindingAsync calls the bound IntegrationHub connector test when an IIntegrationManager is configured. It fails closed when the integration is missing, disabled, role/type mismatched, the connector test throws, or no connector-backed tester is configured. Topology readiness gates for registry, vault, and settings-store bindings consume that result.
• Policy-gate bundle simulation (StellaOps.ReleaseOrchestrator.PolicyGate, a library not referenced by the WebApi) is a preview capability only. If promotion artifact hydration is not configured, SimulateBundleAsync returns Error, AllGatesPassed=false, and a blocking gate named promotion_artifact_hydration_unavailable; it is not a policy decision record.
• In the Compliance library (not referenced by the WebApi), compliance control categories without an automated validator fail closed as failed controls with remediation guidance. They must not be counted as not applicable unless the framework map explicitly defines them as not applicable.

Current Deployment Runtime Snapshot (2026-05-11)

The Release Orchestrator runtime is no longer a stub-only deployment surface. The current WebApi can create persisted environments, targets, agents, releases, components, approvals, gate decisions, deployment jobs, and per-target execution state. The repeatable VM gate npm run test:e2e:live:deployment-vms passed 20/20 on 2026-05-09 against the local Hyper-V lab and Docker Desktop host. That run proved Docker/Compose runtime preflight, true runtime-missing failures, registry/pull/health/digest negative cases, Linux SSH and Windows WinRM script/container deployment, multi-service Compose with pre/post hook evidence, rolling/canary/A/B strategy API behavior, native target-agent ComposeHost seed/update/rollback, and a second Linux target fixture. See docs/qa/agent-targets/e2e-test-runbook.md for current command and evidence ids. Subsequent 2026-05-11 targeted Playwright runs added two-Linux rolling progression, required traffic-adapter execution, a lab Nginx data-plane cutover, and native DockerHost target-agent seed/update proof through docker.run.

Agent Transport Modes

Agent Local OCI Registry

StellaOps.Agent.Host wires the local OCI registry when either Agent:LocalRegistry:Enabled=true or Agent:LocalRegistry:OciLayoutBundlePath is set. OciLayoutBundlePath points at an extracted OCI image-layout oci/ directory from an AirGap bundle. On startup, OciLayoutBundleImportHostedService validates oci-layout, index.json, manifest descriptor resolution, and every blobs/sha256/<hex> file by recomputing SHA-256 before importing through OciImageLayoutImporter into IOciContentStore; any mismatch throws and aborts startup. In BundleOnly mode with no fetcher, the /v2/ listener serves only imported content and returns 404 on cache misses without outbound registry access. The agent registry keeps this verification inline and does not take a production dependency on StellaOps.AirGap.Bundle.

Test source verified against 039164bd8e (PCD-360, 2026-10-04). The StellaOps.Agent.Registry.Tests carrier cases use the current OfflineKitCarrierWriter to transport OCI-layout fixtures, then run the real hosted import into FileSystemOciContentStore. They retain corrupt-blob rejection, imported cache pinning and signed decision-referrer block/allow enforcement. The test fixture declares opaque carrier parts; it does not establish an OfflineKit registry-activation API. Re-run the complete project with pwsh -File tools/scripts/test-targeted-xunit.ps1 -Project src/ReleaseOrchestrator/__Agents/StellaOps.Agent.Registry.Tests/StellaOps.Agent.Registry.Tests.csproj -All -BuildProjectReferences -Restore.

In pull-through mode the registry persists fetched content under Agent:LocalRegistry:CacheRoot. The compose estates set it to /var/lib/stellaops/agent-registry-cache on the agent-core-registry-cache named volume; the code default is LocalApplicationData/StellaOps/stellaops-agent-registry-cache (LocalRegistryOptions). The agent-core runtime image (devops/docker/Dockerfile.agent-core.runtime) does not pre-create that directory, so a volume whose root is not writable by the non-root agent user makes the registry fail closed.

Agent registration endpoints live under /api/v1/release-orchestrator/agents and registration tokens under /api/v1/release-orchestrator/agents/registration-tokens. Agent task polling uses /api/v1/release-orchestrator/agent-runtime/agents/{agentId}/tasks:poll and completion uses /tasks/{taskId}/result.

The operator agent routes act within the authenticated tenant (the stellaops:tenant claim; a tenant header does not select another tenant for a non-administrator). GET /api/v1/agents and GET /api/v1/release-orchestrator/agents list only that tenant’s agents. GET, DELETE (revoke) and POST .../refresh-capabilities on /api/v1/release-orchestrator/agents/{id} read and write release.agents by tenant AND id: another tenant’s agent answers 404 agent_not_found, the same body as an unknown id, and its status, certificate and capabilities are not changed. The store repeats the tenant predicate in the status, certificate-clearing and capability updates. The certificate-authenticated agent plane (/agent-runtime/...) takes no tenant from the caller and resolves the agent by its id.

Console agent reuse is target-assignment driven: the environment target form reads registered agents and the tenant’s existing /environments/{id}/targets rows, then derives “used in” environment chips from target agentId assignments. This is intentionally separate from Platform’s /api/v2/topology/* release-control read model, which can be empty until bundle materialization produces topology projections. The Topology > Agents page falls back to the same Release Orchestrator registry view when the Platform topology projection has no agent/target rows.

Target-to-agent assignment is owned by ReleaseOrchestrator at both the topology compatibility alias POST /api/v1/targets/{id}/assign-agent and the owning PUT /api/v1/release-orchestrator/targets/{id}/agent route. Both routes use the same fail-closed eligibility check before persisting the assignment: the target must belong to the authenticated tenant, and the agent must exist in that same tenant, be Active, and advertise the executor capability selected by the deployment route (DockerHost -> Docker, ComposeHost -> Compose, SshHost -> Ssh, WinRmHost -> WinRm, EcsService -> Ecs, NomadJob -> Nomad, AnsibleHost -> Ansible). An foreign target returns non-leaking 404 target_not_found; an absent or foreign-tenant agent returns non-leaking 404 agent_not_found; a non-active agent or missing executor capability returns 409. Heartbeat freshness remains a health/execution concern rather than an enrollment precondition. A target deleted between eligibility validation and persistence still returns 404 target_not_found rather than escaping as a server error. Agent state can change between this validation and target persistence, so health checks and deployment dispatch must continue to verify the current agent and fail closed; assignment eligibility is not a liveness lease.

Target Type Runtime Matrix

ComposeHost has three explicit endpoint modes. The default useRemoteDockerEndpoint=false with no SSH configuration uses the assigned agent’s local Docker socket. SSH configuration selects DOCKER_HOST=ssh://user@host. Remote Docker TCP requires useRemoteDockerEndpoint=true and sets DOCKER_HOST=tcp://host:port; with useTls=true, the orchestrator resolves the CA/client-certificate/client-key references and the agent materializes ca.pem, cert.pem, and key.pem under the task root before setting DOCKER_TLS_VERIFY=1 and DOCKER_CERT_PATH. Partial certificate sets fail closed, and all task certificate files are deleted after preflight/deploy/rollback. NX-1 behavioral coverage uses a real mutually authenticated fake remote daemon, an actual Compose deployment, independent host-side container inspection, and credential-root deletion. NX-2 coverage sends prerequisite-first four-service Compose waves through the production Linux agent’s Docker-over-SSH path and verifies the complete suite is running.

Compose registry preflight has two explicit contracts. preloaded is cache-only: it runs docker image inspect on the deployment daemon and skips registry login and manifest inspection even when registry credentials are available. Registry mode authenticates with the task-scoped deployment credential before inspecting the manifest. Failure evidence retains a stable reason code: registry_authentication_failed, registry_credentials_missing, registry_image_missing, registry_unreachable, or registry_check_failed; the deployment evaluator converts those codes to operator-specific remediation while preserving fail-closed behavior. Plain-HTTP registries require the explicit component setting runtime.registryAllowInsecure=true; only then does the agent add Docker’s manifest-inspection --insecure opt-in. The default remains TLS-only, and the opt-in is recorded in preflight evidence.

For SSH-mode Compose, the task scope writes the private key and an initially empty known_hosts file with owner-only permissions. A task-local ssh wrapper is placed first on PATH because Docker’s ssh:// transport does not consistently honor DOCKER_SSH_COMMAND; the wrapper enforces IdentitiesOnly=yes, BatchMode=yes, StrictHostKeyChecking=accept-new, and the task-local known-hosts file. This is first-contact TOFU, not pre-pinned host-key verification. Environments that require pre-pinning must provide a separately governed host-key admission contract before changing this boundary. The wrapper, key, Docker config, and learned host key are deleted together on every terminal task path.

Ansible Delivery Engine

AnsibleHost is the first-class bridge for estates whose deployment authority remains an operator-supplied playbook. The control plane owns release identity, gates, target/agent selection, bounded configuration transport, deployment-scoped credential authority, dispatch, and evidence. It deliberately does not generate, rewrite, or vendor customer playbooks or roles. The configured PlaybookBasePath must already be visible inside the agent runtime, and DefaultPlaybookPath can be overridden per component by ansible.playbook.

Target-specific non-secret inventory and vars text belongs in AnsibleHostConfig.ManagedArtifacts. Artifact names are portable relative paths; the agent enforces count and character bounds, rejects rooted/traversal paths, materializes the files below the task temp root, resolves managed://<path> and {managedArtifactRoot}, and deletes the root on every terminal path. This replaces workstation bind mounts and direct database edits without turning target configuration into a secret store. Secret-bearing values remain references in target secret-ref fields or component ansible.environment.* configuration.

For pull-mode deployments with a minted ADR-034 capability, builtin://, vault://, openbao://, and authref:// environment values remain unresolved until the agent calls the orchestrator broker. The capability is deployment-scoped and reference-bound; the agent receives no standing secret-provider token. Target SSH, Vault-password, and become-password references are delivered through the scoped credential channel as ansible.sshPrivateKey, ansible.vaultPassword, and ansible.becomePassword. Environment values and resolved credentials are redacted from stdout, stderr, raw proof, and process errors.

The target health surface proves only that its assigned agent is active, fresh, and advertises Ansible. Acceptance still requires a live forcing function: the unchanged playbook must converge on the intended host and independent target inspection must confirm the expected digest-pinned workload. The complete setup and verification procedure is the Ansible delivery environment onboarding runbook; the payload and allow-list details remain in Execution Plugins.

Approval Policy Runtime

The current approval policy evaluator is DB-backed by default. Operator-managed rows live in release_orchestrator.approval_policies and map source environment, target environment, and optional release-name glob predicates to auto-approve, require-manual, or deny. Matching is tenant-scoped and deterministic: enabled policies are ranked by priority DESC, then predicate specificity, then creation time.

The management surface is /api/v1/release-orchestrator/approval-policies and the UI editor is reachable at /releases/policies. Read operations require release.policy.read; mutations also require release.policy.manage and emit audit actions for create, update, and delete. Delete is an audit-preserving soft disable.

The Console editor checks those exact scopes in the session; an admin scope does not imply them, as it does not at the backend. Without release.policy.read it sends no request and shows no list; without both release.policy.read and release.policy.manage the form is read-only. A handoff can carry context in the query string: sourceEnvironment, targetEnvironment and releaseName fill the promotion preview and filter the list, and policyId opens that record, including a disabled one, which is labeled as not applied. The approval detail sends policyId: its approval-policy gate row names the policy the gate applied (the row id, also named at the end of the row name) and links it to /releases/policies?policyId=<id> for a session holding release.policy.read; other sessions see the id without a link, and a row recorded under the DefaultMode fallback names no policy and has no link. The promotion preview reads the saved records the gate uses for one source, target and release: the enabled approval policies Release Orchestrator returns for those predicates, ranked in the gate order above, and the target environment’s policy binding (release.policy-bindings.read; the enabled row matched case-insensitively, as the gate selects it). A newer check or an input change cancels reads still in flight. When no policy matches or no enabled binding exists, the preview states the service-configuration fallback and shows no mode or pack, because the Console cannot read that configuration. The binding itself is changed in the Gate policies bindings panel, which the preview, each pack page and each pack card in the Gate policies list open as /ops/policy/packs?bind=<environment>&packId=<pack> (without bind when no environment was carried); a pack page and a pack card offer that handoff only to a session holding release.policy-bindings.read and release.policy-bindings.write, because the binding endpoints require read on every route. When the list was opened with ?bind=<environment>, a pack card’s links to the pack page carry it, so the pack page offers to bind that pack to that environment. The Console requests release.policy.read and release.policy.manage (Platform EnvironmentSettings Scope, the compose files and the SPA config.json) and the stella-ops-ui client allows both. Of the professional roles, the Executive Assurance Viewer and the Release Operations Steward receive release.policy.read and see the editor read-only; no professional role receives release.policy.manage, because a policy set to auto-approve removes the human approval that release:publish guards. An administrator session receives both (Authority architecture, interactive role grants).

The approval-policy gate reads these approval policy rows and resolves the policy pack it evaluates through the environment policy bindings (below). When no row matches, it applies ReleaseOrchestrator:PromotionApprovalPolicy:DefaultMode; when no binding exists, it evaluates ReleaseOrchestrator:Policy:DefaultPackId if set. The Console Platform Setup pages for promotion paths, workflows and gates, gate profiles, release templates and feed policy (/ops/platform-setup/{promotion-paths,workflows-gates,gate-profiles,release-templates,policy-bindings}) are reference pages: they read and store nothing, and no gate evaluates anything shown on them. The promotion paths, workflows and gates and gate profiles pages link the approval-policy editor and Gate policies as setup destinations; each link is shown only to a session holding that destination’s exact read grant, and the required change grants are named. The page at policy-bindings is the Feed Policy reference page and is titled so. Release Orchestrator has no named gate-profile, release-template or workflow-catalog model (the PolicyGate library that models profiles is not hosted by the WebApi), and release creation carries no template identity.

PromotionRequestProcessor persists the matched policy id, mode, reason, gate result, and required-approval override into durable release truth. Deny returns 409 and deployment remains fail-closed because the deployment truth guard refuses blocked approvals. The legacy ReleaseOrchestrator:PromotionApprovalPolicy config rules remain only as a transition fallback when ReleaseOrchestrator:PromotionApprovalPolicy:UseDbBackedStore=false or no policy repository is registered.

Promotion persistence has a strict durability order: the gate-decision row is committed first, then the resulting approval status, quorum, and complete gate results are committed before optional OCI referrer and policy-evidence work. Those optional side effects still honor request cancellation, but cancellation during them cannot strand a pending approval with empty gate results after gate truth has committed.

Approval mutation has one backend safety boundary: ApprovalSafetyGuard. The normal, batch, v2, and dashboard approve routes all call it before quorum can change. Empty gate rows, waiting, advisory/skipped incomplete gate truth, a non-passing gate, missing evidence, or an invalid/expired exception return a typed conflict without mutating the approval. The dedicated gate-exception route accepts any such transition-blocking gate result, including advisory/skipped, but only with the gate-bypass scope and a signed, time-limited operator decision. It answers 409 gate_not_blocking for a passed gate and for a gate with an active signed exception. A gate whose recorded exception has expired, or carries no active signed exception evidence, blocks approve and dispatch again (ReleaseTruthGuard.IsGateSatisfied); the route then answers 409 gate_exception_invalid_or_expired, names the recorded expiry in the message and in expiresAt (null when the expiry has not passed), and records nothing. A second exception is not granted on the same approval: to evaluate the gates again, reject or cancel the approval while it is pending, then request a new promotion. Reject remains independent so an operator can still stop an unsafe promotion. Batch approval verifies the operator decision once, preflights every selected approval, and only then persists decisions, preventing partial batch mutation.

The releases list projects current per-gate truth through ReleaseGateRollup, using the same status distinction as promotion detail. pending, waiting, running, and scanning gate results are in-flight evaluation states: their rollup status is waiting, their blocking count is zero, and Console labels them Pending evaluation. A promotion whose gates passed but still needs human quorum is separately pending/Awaiting approval. Only explicit failed, blocked, denied, rejected, or expired outcomes may produce a blocking count or blocking label.

Protected environments and signing-required environments add a stronger contract: the durable gate decision must contain a self-consistent GateDecisionBasis, a DSSE envelope bound to that canonical decision, and a policy-decision evidence packet whose status was earned through EvidenceVerificationService. Open environments retain the advisory evidence rollout and do not require signed gate-decision evidence unless global Approve signing or the per-environment signing list is enabled.

Separation of duties on approve (CCP-067)

Where the target environment sets RequireSeparationOfDuties (request field requireSeparationOfDuties of PUT /api/v1/release-orchestrator/environments/{id}/settings and of the environment create and update routes), ApprovalSafetyGuard refuses an approve by the promotion’s requester on every approve route: POST /api/v1/release-orchestrator/approvals/{id}/approve, …/approvals/batch-approve, POST /api/v1/approvals/{id}/decision (action approve) and POST /api/v1/release-orchestrator/promotions/{id}/approve. The refusal records no decision: quorum, status, action records, approvers and the decision audit chain (release_orchestrator.audit_entries) stay unchanged, and a batch that contains such a request is refused as a whole, so none of its requests is approved. The route’s HTTP audit event (.Audited) still records the refused attempt with its 403 status. Another approver’s approval then counts as before. Reject is not affected, so the requester can still reject their own request. Where the setting is off, the requester’s approval is accepted.

The check runs before gate truth and evidence, because it concerns who decides rather than the state of the request. A request records its requester as one string: the display name (promotion and developer requests) or the subject (release seal aggregate, rows written before DTC-13). The caller is the requester when any of its identity values (display name, name, preferred_username, email and subject, all from the authenticated principal) equals that string, trimmed and case-insensitive. The X-StellaOps-Actor request header is not one of them (see Actor attribution).

Limit: the requester is matched on the recorded identity string, not on a recorded subject. A requester whose display name changed after the request, and whose request recorded only that earlier name, is not recognized; two people who share a display name are treated as the same person (the second is refused). The release seal aggregate (POST /api/v1/release-orchestrator/releases/seal) does not use the approve routes: it records an approved request whose requester and approver are the same caller, also where the target environment requires separation of duties.

Decision basis requirement (CCP-055)

An approve outcome requires a decision basis. Before the outcome is fixed, the registered evaluator chain (PluginBackedGateDecisionEvaluator) classifies the aggregated gate rows with ReleaseTruthGuard.AssessApprovalBasis. An approve-capable mode (auto-approve or require-manual) is recorded as outcome approve only when all required inputs are present:

1. the Policy pack identity and version that were actually evaluated (an evaluated policy-engine row carrying packId and packVersion);
2. a findings projection for the candidate digest, carried by that Policy verdict as structured evidence (a pack without CVE rules consults no findings and does not provide one);
3. a verified attestation for every candidate digest when the deploy-time attestation gate is blocking for the target environment (advisory enforcement does not make the attestation a required input).

A missing input records outcome blocked (fail closed) with mode deny; an input that exists but whose owner read failed records outcome unknown with mode waiting. In both cases the decision policyId is decision-basis, the reason names every missing input, approval-policy rows no longer report passed, and a decision-basis gate row carries missingInputs plus the requestedMode, requestedPolicyId and requestedReason the approval policy had produced. PromotionRequestProcessor routes blocked like a deny (approval blocked, HTTP 409) and unknown like a waiting gate (approval stays pending and not passing).

The findings input is classified from the Policy verdict’s structured findingsBasis (SPRINT_20261007_013 CCP-096), carried on the policy-engine gate row as the findingsBasis detail beside evidenceNote: resolved is a findings projection, absent and not_consulted record findings_projection_absent, and unavailable or an unrecognised value records findings_projection_unavailable. The note text is not the decision input; a verdict from a Policy.Engine that does not send findingsBasis is classified by its note as before (a pack without CVE rules and the warn-posture unavailable note). An allow over a subject with no findings source (findings basis absent) therefore records blocked, while a scanned release whose findings source answered with no findings (resolved) approves.

In a blocking attestation environment, ReleaseOrchestrator:AttestationGate:BlockOnUnavailable=true turns an unreadable attestation state into a deny. With the default false the attestation row does not fail on an outage, but the basis records attestation_unavailable and the outcome is unknown, so the approval is held instead of approved. Before this rule the default let such a promotion approve. Advisory environments are unaffected.

A configured permissive rule is the only path to approve without a Policy verdict: a DB-backed approval policy or ReleaseOrchestrator:PromotionApprovalPolicy:Rules entry that carries a policy id. Its decision records the policy id on the decision, as the approval-policy gate id and in the gate name (Auto-approve policy (<id>)), and in the reason; an operator reason that does not contain the id is recorded as <mode> by rule <id>: <reason>. The rule stands in only for an absent Policy verdict, never for an unreadable Policy.Engine or a required attestation. The default mode is not a rule: without a Policy verdict it records blocked. The default-mode reason is derived from the effective mode (PromotionApprovalPolicyOptions.DefaultReasonFor); the former configurable DefaultReason text was removed, so a decision with mode auto-approve cannot carry the manual-approval default text. Consequence for a new estate: until a policy pack is bound or a named rule is configured, every promotion is blocked with policy_pack_absent and findings_projection_absent.

Recorded decisions are immutable. ReleaseTruthGuard.AssessRecordedDecision classifies a stored decision for readers without rewriting it: complete, permissive-rule, approved-without-basis (an approve whose recorded basis lacks a required input, for example decision gate-apr-012-1787402288), basis-not-recorded (an approve with no basis: rows written before the basis existed and the release seal aggregate), blocked, unknown, or not-approved, with the missing inputs by reason code. ValidateDeploymentRequest, used by the deploy endpoint, the approval dispatch processor and the seal aggregate, refuses a new deployment whose latest gate decision is approved-without-basis with reason code gate_decision_basis_absent; basis-not-recorded keeps the protected-environment rule above. The basis check runs after the approval-status, quorum, gate-row and evidence checks. A decision whose rows still include a skipped or advisory row is refused earlier with gate_evidence_incomplete, and an approval that is no longer approved with approval_not_approved. The basis check decides when every row is satisfied, for example an approve recorded against a pack without CVE rules (findings_projection_absent). Signed gate exceptions on its rows do not make an approved-without-basis decision dispatchable.

The dashboard read (GET /api/v1/release-orchestrator/dashboard) counts gate rows in gatesSummary as pass (passed), warn (warning), block (failed, denied, blocked) and unknown (unknown). Approval rows recorded before this rule keep their original statuses.

Operator-signed release decisions (Sprint 20260608_014)

Release decision signing is wired behind default-off per-decision flags under ReleaseOrchestrator:OperatorDecisionEnforcement. The flags are Approve, Reject, Promote, Deploy, and Rollback; all default to false so existing operator flows continue to work until a signing client is available. When a flag is enabled, the corresponding endpoint requires an operatorDecision body carrying an operator-decision@v1 DSSE envelope. The Release Orchestrator does not perform local signature verification: it calls the WS3 IOperatorDecisionAttestationService, which verifies through IOperatorDecisionVerifier and the configured ICryptoProviderRegistry. Fresh-auth is evaluated with the shared FreshAuthenticationGate (RequireFreshAuthentication=true, five-minute default window, optional MinimumAcr).

Production verification resolves the DSSE envelope keyid through the Authority IssuerDirectory client. Set IssuerDirectory:Client:BaseAddress and ReleaseOrchestrator:OperatorDecisionEnforcement:IssuerId before enabling a decision-enforcement flag. The resolver calls the IssuerDirectory operator-signing-key read API, maps the public material into the WS3 EnrolledKey contract, and fails closed when the client, issuer id, key row, algorithm id, or public material is unavailable. The configured client attaches the host’s existing Authority service identity, requesting only issuer-directory:read for the tenant carried by that exact key read. It never forwards the operator’s bearer or registers a second token client. Missing service credentials, a refused token, a token without the read scope, and an unavailable directory cannot supply an enrolled key. The canonical issuer namespace is operator-signing; the serving owner resolves tenant and issuer names to their canonical IDs and filters the key by all three identities.

Single-key verification reads are uncached, so the next decision sees the owner’s current revocation or refusal. The shared attestation verifier still applies purpose, authenticated subject, validity, revocation instant and signature checks; historical signatures retain their existing decision-time semantics. Service token acquisition uses the existing Authority SDK, independently of operator fresh auth. The key-read configuration and R9 operator package separate this source behavior from the required live grant and deployment.

Gate exceptions require signing unconditionally, independent of the default-off decision flags. When either required setting above is absent, an authorized POST /api/v1/approvals/{id}/gate-exception returns 503 operator_decision_signing_not_configured and names both settings; a caller without release:gate-bypass is still rejected by authorization with 403. Missing configuration never falls through to unsigned exception handling.

Approval, release, and release-activity environment filters use one comma-delimited set contract. The v1 approval aliases accept environment, the v2 approval queue accepts targetEnvironment, and release/activity list reads accept environment; each splits, trims, and matches every value case-insensitively. A value such as dev,prod-eu-west,staging is therefore a three-environment selection, never one literal environment name. Deployment lists retain the same set semantics across their environment and environments aliases. This is the server half of the Console global-context contract; endpoint filters must not compare the complete CSV string to one row.

Approve=true applies to every approval surface, including batch, v2, and the dashboard route. A narrower rollout can instead list target environments under ReleaseOrchestrator:ApprovalSafety:SigningRequiredEnvironments; those environments require both a verified operator approval envelope and verified signed gate-decision evidence. Batch clients send the envelope in the optional operatorDecision member of the batch body.

The release truth schema has nullable operator-decision columns (consolidated baseline) on both release_orchestrator.approvals and release_orchestrator.gate_decisions: decision_envelope, decision_keyid, decision_algorithm, decision_signed_at, and decision_digest. Unsigned/default-off rows keep these fields null. Signed rows persist the verified envelope metadata and the real sha256: digest of the submitted compact DSSE envelope. Approval evidence and v2 replay surfaces expose the real digest plus keyid; unsigned approvals return null / not_available and no longer synthesize sha256:decision-* or policy-decision-*.dsse placeholders.

release_orchestrator.release_evidence remains the store for release evidence bytes and release-evidence signatures. The operator-decision envelope is stored on the approval/gate-decision row because the enforcement and replay paths need per-decision truth by approval/gate id, and deploy/rollback decisions are lifecycle decisions rather than standalone release-evidence packets. The digest column preserves the compact envelope content address even though the JSONB envelope column may be rendered differently by PostgreSQL.

Digest-keyed deployment-decision artifacts (WF5)

The promotion gate emits the WF5 referrer after the durable gate-decision upsert in PromotionRequestProcessor.ProcessAsync. Emission is keyed by component image digest, so one signed deployment-decision referrer is attached per valid sha256: release component digest rather than per release. The predicate captures the gate verdict, policy id/version, tenant, masked image reference, SBOM readiness, and reachable CVE hints available from the gate results. Registry placement is derived from the component image reference host/repo when available and falls back to the internal metadata registry namespace. Referrer emission is fire-and-log: signing or OCI attachment failures are logged per component and never roll back the already-persisted promotion decision.

In connected profiles, the WebApi registers ScannerReferrerSink when Scanner:BaseUri is configured. The sink posts the deployment-decision referrer to Scanner’s POST /api/v1/oci-referrers ingest surface with X-StellaOps-TenantId, so the signed verdict lands in the Scanner OCI metadata store that MetadataRegistryGateway.ResolveDecisionAsync later reads through the /v2/<repo>/referrers/<digest> surface. When ReleaseOrchestrator:SbomReadinessGate:ServiceIdentity:Enabled=true, the Scanner sink and metadata gateway clients also attach a tenant-scoped stellaops-release-dispatch client-credentials token. The default requested scope set is scanner:read scanner:scan scanner:write, covering both deployment-decision ingest and OCI referrer read-back off the trusted bypass network. When Scanner:BaseUri is absent, the WebApi keeps the local FileSystemOciContentStore sink for offline and bundle-only profiles.

Deployed-estate deviations keep a local structured warning as the fail-visible baseline. When ReleaseOrchestrator:DeviationNotify:Enabled=true, HttpDeviationEventPublisher additionally posts the canonical generic-event envelope to Notify at /api/v1/events/releaseorchestrator. The HTTP client mints a per-tenant notify.operator client-credentials token from the shared stellaops-release-dispatch identity, carries the same tenant in X-StellaOps-TenantId, and treats transport/non-success responses as non-fatal so notification availability cannot change deployment, rollback, heartbeat, or estate re-touch state. The shipped compose profile enables this path; operators can set RO_DEVIATION_NOTIFY_ENABLED=false while retaining the local warning trail. Agent heartbeats persist drift under the agent’s canonical tenant UUID. Before an outbound Notify hop, Release Orchestrator resolves that UUID through the host’s replicated Authority tenants catalog (AddStellaOpsTenantResolverReplicaOnly) to the registered Authority tenant key and uses that one key consistently in the token request, header, body, and idempotency key. Unknown UUIDs fail closed before HTTP dispatch while retaining the local structured warning.

Risk that changes after deployment reaches the deployed estate through the estate re-touch job (EstateRetouchProcessor, ReleaseOrchestrator:EstateRetouch, hourly by default). For every currently deployed (environment, release) subject of every tenant it runs the same gate evaluation as a promotion, as a recorded re-evaluation by release-orchestrator:estate-retouch rather than a promotion request, against the current Policy answer (the gate projection the Vulnerabilities hub stream feeds), and appends a dated verdict to release_orchestrator.estate_verdicts. It never writes gate_decisions, approvals, deployments, the deployed estate or any grant: the authorizing decision keeps its original outcome and basis.

The baseline is the decision that authorized the placement, bound the way the chain read’s Decision stage binds it: the decision for the approval of the deployment that established the placement, decided no later than that deployment started (when no decision names that approval, a decision recorded without an approval id for the same release and environment within that window). A later decision for the same release and environment, for example a repeated promotion request that was denied, does not replace it. A deployment that carries no approval has no authorizing decision, and its subject reads as unbaselined once re-evaluated. The gate is evaluated with the environment as the authorizing decision names it (or, without a bound decision, as the deployment’s approval names it): the environment name the promotion recorded and the environment’s policy binding is resolved by, with its environment id. A placement keyed by the environment UUID, which the library deploy engine records on the deployment and therefore on the deployed-estate row, is compared with the same decision as one keyed by the name; the verdict and the reads stay keyed by the identifier the deployed-estate row carries.

Each verdict records its basis, assembled by the builder a promotion decision uses: the policy pack id, revision and bundle digest, the findings hash over the matched CVEs and the release component digests, with a basis hash (verdict_json.basis; the findings hash also in findings_hash). The re-evaluation request carries no components, so the builder takes the component digests from the release record; they are the release’s recorded components, not digests a gate iterated in this re-evaluation. It records the authorizing decision id (authorizingDecisionId), the environment the gate was asked for (evaluatedEnvironment) and the gate’s own reason (the Policy reason), also on a deviation. A verdict that is more severe than the authorizing decision (for example approve to warn, deny or blocked) is a deviation. It raises release.estate.finding when the regression starts (no earlier verdict, a recovery in between, or another outcome) and again when the previous verdict of the subject is at least 24 hours old (for example when the job did not run in between; hourly cycles of a standing regression announce nothing new). The notification carries the outcome pair, the Policy reason, the authorizing decision, the evaluated environment, the pack, the findings hash, the basis hash and the release component digests (componentDigests). Each regression start also records one estate_finding release-timeline event, so a regression after a recovery adds a new one; a reminder adds none. The event is written before the verdict, and its id names the verdict the regression followed: when the verdict write fails, the next cycle repeats the same idempotent insert, and when the event write fails, no verdict is stored and the next cycle starts the regression again.

The deviations read (/estate/deviations) returns the outstanding deviations: for each currently deployed subject, its latest verdict when that verdict is a deviation, newest first, with its basis. The latest verdicts of all deployed subjects come from one set-based read (IEstateVerdictStore.ListLatestForSubjectsAsync, DISTINCT ON over the environment, release and evaluation time index). A standing regression is one item however many cycles repeated it, and a recovered or replaced release is not listed; earlier verdicts stay in the table as history. The status read (/estate/status) carries the same count in deviations.count, plus deviations.subjects (deployed subjects) and, so that an absent assessment is not read as a clean estate, deviations.unevaluated (never re-evaluated), deviations.unbaselined (re-evaluated, but no authorizing decision with a pass or failure outcome was bound to compare against, so a deny is not called a regression) and deviations.unknown (re-evaluated to an outcome that is neither a pass nor a failure, such as an unavailable owner read). The re-evaluation request carries no release components, so the digest-keyed gates (scan attestation, SBOM readiness) contribute nothing to it; the Policy pack evaluation receives the release subject and answers it from its own findings projection.

The outstanding deviations reach a person through the Watch stage of the chain-completeness read, which the Console’s assurance queue renders: the chain read applies the same per-subject classification to the subjects in its scope, so a placement whose subject has an outstanding deviation reads Watch unverifiable (estate-deviation, with the outcome pair, the authorizing decision and the Policy reason), a recovered subject reads recorded-unsigned (estate-reevaluated), and an unevaluated, unbaselined or unknown subject, or an unreadable verdict store, reads unavailable, never recorded. The service-owned Doctor check check.release.environment.matches-approved runs through the existing /doctor/release-orchestrator/checks host (ops.health). It reuses the application-owned EstateRiskAssessment classification with explicit owner-tenant reads and publishes aggregate counts only. Drift, outstanding deviations and unbaselined subjects are unhealthy/Critical; failed or absent reads, unknown or unevaluated risk, incomplete observation and an empty estate are unhealthy/Warning. The old runnable Release plugin check is withdrawn. See the check article for the Platform registration settings and the activation order. Registration rides the shared stellaops-release-dispatch identity, which Authority Startup migration 040 grants platform:doctor:register. The settings ship in the Release Orchestrator compose overlay only, with registration off by default: an estate opts in with RELEASE_ORCHESTRATOR_DOCTOR_REGISTRATION_ENABLED=true after Authority has applied 040. The release bundle and docker-compose.stella-services.yml carry no Doctor__Registration key, so a release-bundle install does not register. release.estate.finding is unchanged; it reaches a person only through a Notify rule an operator imports. Source: EstateRetouchProcessor.cs, PostgresEstateVerdictStore.cs, EstateEndpoints.cs, ChainCompletenessProjection.cs, EstateRiskAssessment.cs, ReleaseEstateMatchesApprovedDoctorCheck.cs.

Deployment-decision signing can be supplied from runtime configuration with ReleaseOrchestrator:DeploymentDecision:Signing:KeyId and one of two key sources. The operator’s mounted file wins (precedence corrected by SPRINT_20260914_001 JCR-1, applying the rule recorded for the NIS2 exporter):

1. ReleaseOrchestrator:DeploymentDecision:Signing:Es256PrivateKeyPemFile (env alias STELLAOPS_DEPLOYMENT_DECISION_ES256_PRIVATE_KEY_PEM_FILE) — a path to a mounted EC (P-256) PEM. Once configured it is authoritative: a path that does not exist, cannot be read or is empty fails the host at startup naming the variable, and never falls back to the inline value. This is the primary source for the deployed stack: the base compose points it at a per-lab dev PEM generated by devops/compose/scripts/ensure-dev-certs.sh (gitignored under devops/etc/release-orchestrator/keys/, mounted read-only), mirroring the scanner scan-attestation key custody pattern — so run that script before docker compose up. A plain recreate then keeps signing without re-injecting a key from the environment. Production replaces the mounted file with a custody-managed PEM at the same container path.
2. ReleaseOrchestrator:DeploymentDecision:Signing:Es256PrivateKeyPkcs8Base64 (env alias STELLAOPS_DEPLOYMENT_DECISION_ES256_PRIVATE_KEY_PKCS8_B64) — an inline base64 PKCS#8 key, consulted only when no file path is configured.

The key id comes from ReleaseOrchestrator:DeploymentDecision:Signing:KeyId (env alias STELLAOPS_DEPLOYMENT_DECISION_KEY_ID), default deployment-decision-local-es256; the compose estates set ro-gate-decision-dev-es256. Under a regional crypto profile the host skips PEM import: the provider owns the key and only the key id is used. When a key is present the WebApi imports the P-256 private key into the configured crypto provider and sets DSSE signing defaults to ES256 for WF5 deployment-decision artifacts; the corresponding SPKI public key is exported for agent registry verification. When neither source is configured the signer degrades honestly to decision_algorithm='unsigned' with a NULL envelope — never a silent pass, and never a repo-baked key. (Until 2026-09-14 a configured-but-missing file also degraded to unsigned; that silent downgrade is exactly what the precedence rule forbids, and it now fails loudly instead.)

The live agent pull-through registry path imports those same decision referrers before it stores the subject manifest locally. On a pull-through miss, OrchestratorImageFetcher asks the orchestrator content-by-digest channel for /deployments/{id}/registry/decision-referrers/{subjectDigest}; the WebApi re-reads Scanner’s /v2/{repo}/referrers/{digest} surface, digest-verifies the referrer manifest and layer bytes, and returns the bundle over the deployment capability channel. The agent stores the referrer blobs and manifests in its local OCI cache before writing the image manifest, so local registry enforcement sees the same signed Scanner SoR decision that connected promotion emitted.

The content-by-digest endpoint itself serves each in-scope digest from the deployment’s resolved private upstream after digest verification. The orchestrator keeps no content cache: its L2 object-store tier was retired on 2026-09-23 (SPRINT_20260915_001 OSM-3), and the service refuses to start while a retired ReleaseOrchestrator:RegistryL2 or Deployment:Registry:L2 key is set. The agent’s local registry cache is the only cache on the deploy path.

OCI image indexes and Docker manifest lists are treated as first-class subjects: the pull-through fetcher stores the top-level index/list, fetches and stores the referenced child manifests and blobs, and keeps the deployment-decision referrer bound to the top-level subject digest. When a runtime pulls a child manifest, the offline verifier walks the cached parent index/list back to the signed parent decision and rejects the child transitively when the parent verdict is block or deny.

The shared WF5 deployment-decision contract (media types, predicate, serializer, fail-closed verifier) lives in StellaOps.ReleaseOrchestrator.Agent.Contracts (DecisionArtifact/); StellaOps.ReleaseOrchestrator.DecisionArtifact holds the signer, emitter and referrer sinks. The canonical predicate JSON uses alphabetically ordered camel-case fields, is signed through IDsseSigner as an application/vnd.in-toto+json DSSE payload with predicate type https://stellaops.io/attestation/v1/deployment-decision, and is attached as an OCI referrer with artifact type application/vnd.stellaops.deployment-decision.v1. Verification is fail-closed: the DSSE envelope must parse, the signature must verify against the trusted EC public key, subjectDigest must match the deployed digest, and verdict=block or verdict=deny is rejected even when the signature is valid.

Gate plugin migration (Sprint 20260527_023 GPM-003 → GPM-006)

Four built-in gate plugins are registered, in this order: SbomReadinessGatePlugin, ApprovalPolicyGatePlugin, ReachabilityGatePlugin, AttestationGatePlugin. Sprint 023 migrated approval-policy and reachability-gate to this plugin-registry path. The canonical implementations are ApprovalPolicyGatePlugin and ReachabilityGatePlugin (in WebApi/GatePlugins/); each wraps the legacy domain-service evaluator behind a BuiltInGatePluginEnvelope.ContributionKey envelope so the plugin path produces byte-identical GateDecisionDto.GateResults[] rows (pinned by PluginPathGateResultsParityTests). The IGateDecisionEvaluator is PluginBackedGateDecisionEvaluator — the only wired gate-decision path. The legacy CompositeGateDecisionEvaluator fall-through and its ReleaseOrchestrator:GateEvaluator:UsePluginRegistry rollback flag were deleted pre-release (P8-11, AGENTS.md §2.11), and the class itself followed on 2026-09-02 (P8-14) — its aggregation semantics are pinned by the Fixtures/GateDecisionGoldens/ snapshots in PluginPathGateResultsParityTests, captured while the byte-identical parity assertions were green; the domain-service evaluators (PolicyDrivenGateDecisionEvaluator, ReachabilityGateEvaluator) remain as the engines the plugin wrappers delegate to.

SCN-ID-5 keeps v2 advisory-source availability inside that same wrapped evaluator, so there is exactly one decision implementation. The strict v2 reader preserves summary.advisorySourceStatus and its optional reason exactly; only exact available is eligible to pass. A partial or wholly unavailable hub observation produces denied under blocking enforcement or a visible advisory row otherwise, and never falls back to an older clean v1 witness.

SBOM readiness gate (Sprint 20260531_054)

Component-version sync now asks Scanner whether each newly discovered digest is already tracked before enqueueing work. HttpSbomScannerClient calls Scanner’s GET /api/v1/sbom-readiness contract with the tenant header and submits POST /api/v1/scans only when the state is absent; pending, scanning, ready, and failed states are not re-submitted. The scan submit payload carries the digest-pinned image reference, component id, version id, registry integration id, and deterministic clientRequestId so a registry re-sync cannot create a scan storm.

Promotion gate evaluation has a distinct waiting state for SBOM gaps. The built-in SbomReadinessGatePlugin wraps SbomReadinessGateEvaluator and runs before approval-policy and reachability gates. If any release component digest is pending, scanning, or absent, the contribution short-circuits with mode=waiting, gate type sbom-readiness, and an operator hint such as waiting on SBOM results: digest..., status pending. A failed Scanner state is an explicit deny with the Scanner failure reason. ready digests contribute a passing SBOM-readiness gate and allow normal policy evaluation to continue.

CVE-aware policy gate (Sprint 20260512_002 R-GATE-CVE)

Release inputs are owner-served (SCN-ID-5). Policy obtains components through GET /api/v1/release-orchestrator/releases/{releaseId}/components, under the existing release:read and authenticated-tenant boundary. The owner checks release existence for every principal and returns 404 when it is absent in that tenant; developer ownership restrictions remain in place. List/single persistence reads expose the stored scalar digest, package_purl and package_ecosystem as digest, packagePurl and packageEcosystem, without rewriting component JSON or re-deriving those values in a consumer. Package metadata and component version can be explicitly null; version null is preserved, not invented from the PURL. This closes the Policy-to-RO SQL read, not the independent Scanner/Policy runtime acceptance gates.

Before the metadata-only approval-policy match runs, PolicyDrivenGateDecisionEvaluator (wrapped by ApprovalPolicyGatePlugin on the plugin path) calls Policy.Engine’s POST /api/policy/packs/{packId}/revisions/{version}/evaluate endpoint via IPolicyEngineGateClient. The pack is resolved by HttpPolicyEngineGateClient.ResolvePackAsync (see “Environment policy bindings” below): DB binding, then ReleaseOrchestrator:Policy:DefaultPackId / DefaultPackVersion. If no pack is configured, the call is skipped, the visible policy-engine-skipped row records reason code policy_pack_absent, and the approval-policy path can approve only through a named permissive rule (see “Decision basis requirement”). A deny verdict from Policy.Engine overrides any auto-approve approval-policy rule — the resulting gate decision row is outcome=deny, mode=deny, policyId={packId}@v{version}, with the policy bundle digest captured in the reason. By default the gate is fail-closed: Policy.Engine 5xx / timeout / network failures produce a deny so a denylisted CVE cannot slip through during a policy-engine outage. With ReleaseOrchestrator:Policy:FailClosed=false an outage records the skipped row with reason code policy_verdict_unavailable and the decision is unknown, not approve. ReleaseOrchestrator:Policy:Enabled=false disables the CVE-aware gate; the decision then has no Policy verdict and is blocked unless a named permissive rule applies.

CVE-to-release impact query (Sprint 20260627_005)

The reverse impact endpoint GET /api/v1/release-orchestrator/release-impact?identifier={cve-or-advisory} answers the FE vulnerability lens without introducing a separate projection table. It reads tenant-scoped release truth from persisted releases, approvals, gate decisions, approval evidence packets, and release evidence packets. Current impact is built only from the latest gate decision per (releaseId, targetEnvironment) plus approval security snapshots when no current gate decision covers that approval gate. Older matching gate decisions are returned under timeline.historical and do not contribute to currentImpact.

The response separates impacted releases, approvals, promotions, environments, gate verdicts, reachable CVE rows, evidence links, and timeline rows. All collections are sorted deterministically for UI diffing, tenant boundaries are enforced through the release-truth repository, and no-impact identifiers return HTTP 200 with empty=true and empty arrays. If only historical rows match, currentImpact remains empty while timeline.historical preserves the audit trail.

Environment policy bindings (Sprint 20260513_021)

Pack-to-environment binding is now operator-managed via the release_orchestrator.environment_policy_bindings table (Postgres, consolidated baseline 001_release_orchestrator_consolidated_baseline.sql — folded in from the pre-1.0 007_environment_policy_bindings.sql, archived under Migrations/_archived/pre_1.0/mig061/ and not embedded). The CVE-aware gate’s pack resolver walks this chain in HttpPolicyEngineGateClient.ResolvePackAsync:

1. DB binding (canonical): row in release_orchestrator.environment_policy_bindings keyed by (tenant_id, environment) with enabled = TRUE.
2. Tenant default: ReleaseOrchestrator:Policy:DefaultPackId / ReleaseOrchestrator:Policy:DefaultPackVersion.
3. Skip — the gate-call is skipped; the decision records policy_pack_absent and is blocked unless a named permissive approval-policy rule applies (see “Decision basis requirement (CCP-055)”).

The bindings are administered via tenant-scoped REST endpoints (/api/v1/release-orchestrator/environments/{environment}/policy-bindings, GET / PUT / DELETE, plus /environments/policy-bindings for list). Authorization uses the named policies release-orchestrator.policy-bindings.read (GET) and release-orchestrator.policy-bindings.write (PUT / DELETE); the JWT scope tokens that satisfy those policies are release.policy-bindings.read and release.policy-bindings.write respectively (see ReleaseOrchestratorPolicies.cs). Both scopes are in the canonical StellaOpsScopes catalog (ReleasePolicyBindingsRead, ReleasePolicyBindingsWrite) and ReleaseOrchestratorPolicies.cs binds the constants, not literals, so tokens minted by Authority satisfy them. DELETE is a soft-delete that flips enabled=false; rows remain visible via ?includeDisabled=true for audit replay.

The former static environment-to-pack config map is no longer read: ResolvePackAsync consults only the DB binding and the tenant default above.

Deployment Engine Boundary

HTTP deployment requests now enter the canonical library DeployOrchestrator.StartAsync(promotionId, options) path through LibraryDeploymentRequestStarter. The starter resolves release-truth approval rows, creates a dedicated deployment scope for the library engine, and mirrors terminal DeploymentJob state back into release_orchestrator.deployments, including target status, progress, errors, agent id, strategy evidence, and rollback status. The previous WebApiDeployOrchestrator adapter and WebApiArtifactGenerator stub were removed from the WebApi production assembly.

Deployment creation requires an approved promotion for the resolved release and target environment. If that approval is absent, the endpoint returns HTTP 409 ProblemDetails with code=approved_promotion_required, releaseId, and environment; the condition never enters the legacy fallback path, even when fallback is enabled. This is an operator-correctable state conflict, not an internal server error.

Within the deployment library, TargetExecutor owns stateful target dispatch, secret resolution, runtime preflight, health probes, task persistence, and evidence publication. Pure deployment-value rules are isolated in the internal TargetExecutionValues collaborator: digest/config parsing, masked-reference anchoring, per-bundle plugin classification, registry-context selection, result-value projection, and deterministic identifier/hash normalization. The collaborator has no service dependencies and does not change the public TargetExecutor constructor or agent task/wire contracts.

Immutable release and deployment subject projection

The cross-module canonical subject and evidence-spine contract defines the owner ids, digest passport, Web handoff parameters, lifecycle continuity, and distinct global/aggregate/record evidence-state namespaces used by Release Orchestrator, EvidenceLocker, Attestor, Estate, and the Console.

Release-artifact identity uses only the canonical OCI form sha256:<64-hex>. Component-version sync drops malformed registry values, and release/developer-approval writes reject malformed caller values instead of persisting a sha-looking identifier. Valid values are normalized to lowercase before they enter release, approval, or deployment projections.

Promotion approvals carry the canonical target environment id and component ids alongside the release id/version. Their candidateDigest is either the exact authored digest or the one unambiguous component digest; it is never derived from a release id. Deployment summary/detail JSON persists the same approval id, candidate digest, component subjects, release id, and environment id, and deployment lifecycle events repeat those identifiers for Evidence and Web consumers. Legacy records without a usable subject return legacy_release_digest_missing, legacy_release_digest_invalid, or release_digest_ambiguous; registry availability failures retain their separate currency reasons. The deploy-engine adapter uses the exact digest for a one-component release and a deterministic digest of ordered componentId=digest subjects for a multi-component manifest. These fields are additive members of the existing JSON projections, so no PostgreSQL schema migration is required.

The local deployment and bounded UI verification of 2026-07-12 are recorded in the archived RDT-2 QA run (pruned evidence docs-archive/qa/feature-checks/runs-2026-live-snapshot/2026-07-12-rdt2-immutable-subject/README.md at 3f422b9b72).

Rollback requests use WebApiDeploymentRollbackStarter, which drives the deployment pipeline rollback path directly and projects terminal rollback state through the same compatibility read model. On successful WebApi rollbacks the starter creates the canonical release.rollback.evidence.v1 JSON through RollbackEvidenceGenerator, stores it as a verified release evidence packet, and exposes the evidence id, stored content hash, proof digest, and proof status on deployment strategy evidence. The deploy engine’s automatic rollback writes the same packet through the same writer (ReleaseTruthRollbackEvidenceRecorder, the engine’s IRollbackEvidenceRecorder) when it restores its targets; the job keeps the packet reference and the terminal projection exposes it the same way. The rollback evidence also carries multi-module order proof: failed module identity, original deployment order, rollback order, dependency names, and release-script proof digests when task results include them. The projection exposes orderProofDigest and orderProofStatus next to target digest proof so replay/audit tooling can detect dependency-order proof drift. The library strategy planner covers rolling, canary, blue-green, and all-at-once execution; rolling/canary/blue-green are verified against real Testcontainers sshd targets in addition to recorder tests for batch order.

Release script graph proof now has an executable bounded contract in the Scripts library. IReleaseScriptGraphProofBuilder emits release.script.graph-proof.v1 evidence with deterministic start and update orders, module/script/version dependency references, fail-closed issues for missing dependencies or version mismatches, and a sha256: proof digest. Start scripts are not allowed to depend on update scripts; update scripts may retain cross-phase references to start scripts while update-to-update dependencies drive update ordering. See deployment/runtime-contract.md — release script graph proof.

Strategy currency — 2026-09-30

The dated source capability matrix at commit 14fe9dc8864cae2729c45b363a34a8ab370ef545 qualifies the strategy statements above. Current rolling/canary code batches target tasks; canary bake uses Task.Delay. Normal forward LibraryDeploymentRequestStarter applies optional traffic after terminal deployment, while retry/rollback DeploymentEnginePipeline has different per-batch placement. Local-file and empty-2xx webhook acknowledgements do not prove applied routing. A/B maps to BlueGreen; the retained Progressive Nginx library is unserved in the inspected host and does not establish stable first-exposure cohorts/business outcomes. These current-source capabilities are runtime unverified; historical strategy tests establish their tested boundaries only.

The planned release operation contracts require one runner contract, isolated baseline/candidate replicas, real self-hosted Nginx loaded-state/routed-request proof, continuous health windows and verified exact-baseline restoration. This is successor implementation behind PG-1, with reviewed Ansible capabilities declared separately.

Platform Themes

The themes below are a functional taxonomy used across the dossier’s secondary documents. They are not a project map: see Components for the compiled projects. WORKFL has no implementation in this module (there is no RO workflow engine; StepExecutor in the Plugin library is the only step executor).

Components

Actual layout under src/ReleaseOrchestrator/ (verified 2026-09-26). * marks projects in the WebApi’s transitive ProjectReference closure.

ReleaseOrchestrator/
├── __Apps/
│   └── StellaOps.ReleaseOrchestrator.WebApi/     # HTTP host: minimal-API endpoints, gate plugins, services
├── __Libraries/
│   ├── …Agent/ *                    # agent domain models + runtime stores (PostgresAgentStore)
│   ├── …Agent.Contracts/ *          # agent wire contracts, incl. WF5 decision media types/predicate/verifier
│   ├── …DecisionArtifact/ *         # WF5 decision signer, emitter, referrer sinks
│   ├── …Deployment/ *               # DeployOrchestrator, TargetExecutor, strategies, rollback, artifacts
│   ├── …Environment/ *              # environment/target/freeze-window models and stores
│   ├── …Federation/ *               # regional federation
│   ├── …IntegrationHub/ *           # integration connectors / hub
│   ├── …Persistence/ *              # consolidated migration authority (four schemas)
│   ├── …Plugin/ *                   # plugin infrastructure (gate/step/integration capabilities, StepExecutor)
│   ├── …Promotion/ *                # promotion, approval, decision models
│   ├── …Release/ *                  # release + release-component models, IRegistryConnector
│   ├── …Scripts/ *                  # release-script registry + graph proof
│   ├── …Storage/ *                  # runtime repositories
│   ├── …Topology.Contracts/ *       # topology-snapshot wire contract (also referenced by Platform)
│   ├── …Compliance/                 # compliance control validators (tests only)
│   ├── …Evidence/                   # evidence collector / audit exporter (tests only)
│   ├── …EvidenceThread/             # evidence-thread store (tests only)
│   ├── …Observability/              # metric/log/trace export (tests only)
│   ├── …Performance/                # batching/caching/pooling/prefetch (tests only)
│   ├── …PolicyGate/                 # policy-gate simulation models (tests + Performance)
│   ├── …Progressive/                # progressive delivery / canary (tests only)
│   ├── …Plugin.Sdk/                 # plugin SDK (tests only)
│   └── …SelfHealing/                # self-healing (tests only)
├── __Agents/
│   ├── StellaOps.Agent.Core/        # agent framework, CapabilityRegistry, OrchestratorImageFetcher
│   ├── StellaOps.Agent.Host/        # target-resident host (mTLS polling); image agent-core
│   ├── StellaOps.Agent.InProcess/   # in-process Compose/Docker/SSH/WinRM adapters (not in the WebApi)
│   ├── StellaOps.Agent.Registry/    # agent-local OCI registry/cache + offline decision enforcement
│   ├── StellaOps.Agent.Secrets/     # broker/edge secret resolution
│   ├── StellaOps.Agent.Docker/ Compose/ Ssh/ WinRM/ Ecs/ Nomad/ Ansible/
│   ├── StellaOps.Agent.DeployCompose/ DeployNative/ BuildDocker/ BuildScript/
│   └── StellaOps.Agent.BuildDocker.Reachability/  # build-context reachability producer
└── __Tests/                         # StellaOps.ReleaseOrchestrator.*.Tests, StellaOps.Agent.*.Tests,
                                     # LegacySchemaFixtures

Retired predecessors (WebApi, Agent, Environment, Persistence, Scripts) are frozen under src/__Obsoleted/ReleaseOrchestrator/. So are the directories that never had a project file: __Libraries/…Core/, …Foundation/, …ProgressiveDelivery/, …ProgressiveDelivery.Tests/, StellaOps.ReleaseOrchestrator.Api/, and __Tests/…Core.Tests/ and …TestUtilities/ (withdrawn 2026-09-27).

Data Flow

Release Orchestration Flow

CI Build → Registry Push → Webhook → Stella Scan → Create Release →
Request Promotion → Gate Evaluation → Decision Record →
Deploy via Agent → Version Sticker → Evidence Packet

Detailed Flow

1. CI pushes image to registry by digest; triggers webhook to Stella
2. Stella scans the new digest (if not already scanned); stores verdict
3. Release created bundling component digests with semantic version
4. Promotion requested to move release from source → target environment
5. Gate evaluation runs: security verdict, approval count, freeze windows, custom policies
6. Decision record produced with evidence refs and signed
7. Deployment executed via agent to target (Docker/Compose/ECS/Nomad)
8. Version sticker written to target for drift detection
9. Evidence packet sealed and stored

Key Abstractions

Reconciled 2026-05-30 against the records under src/ReleaseOrchestrator/__Libraries/. The shapes below are the actual domain records; earlier drafts used speculative field names (Slug, PromotionOrder, FreezeWindow[] FreezeWindows, ApprovalPolicy, EnvironmentState State) that do not exist in code. Freeze windows and approval policy are modelled as separate tables/records, not inline properties of Environment.

Environment

Source: StellaOps.ReleaseOrchestrator.Environment/Models/Environment.cs.

public sealed record Environment
{
 public required Guid Id { get; init; }
 public required Guid TenantId { get; init; }
 public required string Name { get; init; } // "dev", "staging", "production" (lowercase, 2-32 chars)
 public required string DisplayName { get; init; }
 public string? Description { get; init; }
 public Guid? RegionId { get; init; }
 public required int OrderIndex { get; init; } // promotion order (0 = earliest)
 public required bool IsProduction { get; init; }
 public required int RequiredApprovals { get; init; }
 public required bool RequireSeparationOfDuties { get; init; }
 public Guid? AutoPromoteFrom { get; init; }
 public required int DeploymentTimeoutSeconds { get; init; }
 public DateTimeOffset CreatedAt { get; init; }
 public DateTimeOffset UpdatedAt { get; init; }
 public Guid CreatedBy { get; init; }
 public ImmutableDictionary<string, string> BundleDefaults { get; init; } // per-env push-mode bundle defaults
 public bool RequiresHumanApproval { get; init; } // explicit human sign-off (no auto-approve)
 public ProtectionClass ProtectionClass { get; init; } // Open | Gated | Protected (Sprint 20260708_004)
 public LifecycleState LifecycleState { get; init; } // Active | Archived | Disposable (Sprint 20260717 G9)
}

// Health is a separate enum, not a state field on Environment:
public enum EnvironmentHealthStatus { Healthy = 0, Degraded = 1, Unhealthy = 2, Unknown = 3 }

// Explicit, owner-set estate lifecycle. Set ONLY by an archive/dispose/restore action — never inferred
// from a name. Governs default visibility/ranking (Archived/Disposable are demoted out of the estate
// matrix, readiness table, and pickers); it never mutates the authoritative gate mechanism fields. The
// sibling Target record carries the same field. Defaults to Active so pre-lifecycle rows keep their
// behaviour after the backfill.
public enum LifecycleState { Active = 0, Archived = 1, Disposable = 2 }

Lifecycle-state contract (Sprint 20260717 G9): lifecycle_state is persisted as a canonical lowercase token (active | archived | disposable). Storage fails closed on both sides — the enum-to-token serializer throws on an undefined value, ParseStorage throws on an unknown persisted token, and a CHECK constraint in the consolidated baseline (ck_release_environments_lifecycle_state, ck_release_targets_lifecycle_state) rejects any other value at the database. The PUT.../{environments|targets}/{id}/lifecycle endpoints (Operate scope, audited as set_environment_lifecycle / set_target_lifecycle) are the only transition path; the list endpoints accept an optional ?lifecycle= filter (all [default] | active | demoted | archived | disposable) and reject any other token with 400 invalid_lifecycle_filter.

Aggregate environment operational truth

The owning WebApi exposes a bounded, tenant-scoped read model for environment readiness and protection:

• GET /api/v1/release-orchestrator/environments/operational-truth returns all ordered environments in one HTTP response. Each item includes protection, target/readiness counts, evaluated gate reports, and component-specific problems. The response status is complete, partial, failed, or empty; an empty topology is a typed 200 result rather than an error.
• POST /api/v1/release-orchestrator/environments/readiness/validate-all evaluates every known target and returns every target result plus totalCount, succeededCount, and failedCount. One failed target does not discard successful results, and the failure names its environment and target.

The WebApi also contains a TopologyReadinessScheduler that refreshes target connection health and persists readiness reports on a cadence. It is enabled by default (ReleaseOrchestrator:TopologyReadiness:Enabled, default true; shipping it disabled left readiness empty). Every pass performs outbound probes to operator-configured targets or agent bindings and writes topology health rows. the first pass runs after InitialDelay (default 45 seconds), subsequent passes run every Interval (default 5 minutes, with a 60-second minimum), and each health/readiness step is bounded by PerTargetTimeout (default 12 seconds). A failed or timed-out connection test first persists Unreachable with a live host token; readiness is not evaluated when that failure truth cannot be stored. The connectivity_ok meta-gate requires target health to be Healthy or Degraded for every target type, including ECS, Nomad, SSH, WinRM, and Ansible targets whose Docker-specific gates are skipped.

The scheduler establishes a required ReleaseEnvironmentIdentityAccessor override for each tenant before resolving any tenant-scoped service. There is currently no distributed lease for this job; it relies on the Release Orchestrator WebApi running as exactly one instance per database, which the deploy engine requires: several WebApi replicas are not a supported shape (the engine has no job ownership, and its startup recovery settles every RollingBack job, so a second instance would settle a rollback that the first is running; see deployment/runtime-contract.md). In sealed or air-gapped installations the scheduler makes no public-Internet request of its own, but configured targets/bindings can still cross a network boundary. Disable it unless those destinations are local or explicitly permitted, and use the authenticated validate-all operation for manual evaluation. The periodic pass is the only automatic trigger in this slice; deployment- and heartbeat-triggered immediate refresh remain outside the implemented contract. Configuration is documented in appendices/config.md.

Protection repository failures are not projected as a missing policy pack or zero reviewers. The protection object becomes partial and reports protection_policy_binding_unavailable or protection_reviewer_source_unavailable. The aggregate item carries those same problems, allowing Console to distinguish unavailable truth from an actual unprotected configuration. The bounded executive operational reads GET /environments, GET /environments/operational-truth, GET /estate/status, GET /estate/drift, GET /estate/digest/{digest}, GET /estate/unproven-cells, and the Release Command Center’s per-environment GET /environments/{id}/targets and GET /environments/{id}/freeze-windows calls accept either release:read or orch:read; all eight still require the active tenant. The estate status and drift handlers filter the internal cross-tenant drift store by the active tenant’s canonical UUID before returning rows or computing counts. Agent heartbeats persist that UUID, while the compatibility estate/verdict stores continue to use the active tenant key (for example the registered default slug); the handler resolves both forms instead of comparing unlike values. Records from another tenant therefore cannot appear in the response. The digest lookup predicates its deployed-estate query on the same active tenant; other environment/estate reads remain orch:read-only, and all mutation policies are unchanged. Validate-all requires orch:operate and is audited. Source: ReleaseOrchestratorEnvironmentOperationalTruthEndpoints.cs and ReleaseOrchestratorEnvironmentEndpoints.cs; estate filtering is in EstateEndpoints.cs.

GET /api/v1/release-orchestrator/estate/chain-completeness (requires release:read and the active tenant) is the chain of custody per deployed digest: for every placement the deployed estate records, the seven stages Source, Build, Scan, Verdict, Decision, Deploy and Watch, each verified, recorded-signed, recorded-unsigned, missing, unverifiable, unavailable or not-applicable, with its observer and observation time. It composes the owner reads the custody page uses: the Evidence attestations bound to the digest (through the provenance resolver, under Release Orchestrator’s attest:read service token, at most 50 digests per read), the gate decision bound to the establishing deployment through its approval, the deployment record, the deployed estate, the estate re-evaluation verdicts (the Watch stage reads the outstanding-deviation classification of each placement’s subject) and the tenant’s agent drift. Exact-entry owner histories are read for at most 50 unique attestation entries across the request, with at most four detail reads in parallel. A newest overall pass qualifies only with a supplied bound bundle, performed passing signature and issuer checks, and actual >= expected >= 1 signatures; insufficient assurance remains recorded, newest failure is unverifiable, and unreadable or unknown coverage is unavailable. The exact bound verified decision packet earns Decision. Watch additionally requires coherent nonempty digest-matching landing and current estate proof bound to this placement, approval, selected authorizing decision and evaluated digest basis; Current uses existing outcome ranks, with no new TTL or risk threshold. The projection writes nothing and verifies no signature. Placements with a missing or unverifiable stage come first, ordered by environment protection. A developer principal sees only placements of deployments it may read. Contract: api/chain-completeness.md; source: ChainCompletenessEndpoints.cs and ChainCompletenessProjection.cs.

CCP-106 verification is against source 828e64af714918025036d95bc145302be2f79b54; the canonical chain contract records the targeted re-verification command and 105 passing source/loopback fixture cases. R15-RO-1 subsequently proves the real deployed Decision branch through owner verification and chain readback; the contract’s dated live proof distinguishes that result from the still-open supplied-bundle Evidence and healthy-estate acceptance.

For Policy release-set evidence (CCP-085), the resolver requires explicit typed membership and release-set scope; a scalar document digest does not stand for an image. The Verdict explanation states that the digest belongs to one whole-release evaluation, not an independent per-image verdict. Its recorded state can qualify only through the same exact-entry owner receipt rule; the projection itself still verifies no signature. See the Policy release-set contract.

Environment operational truth is not the source for Estate’s service x environment unproven cells: readiness gates describe target connectivity and protection, while cell proof belongs to the deployment that established the cell. GET /api/v1/release-orchestrator/estate/unproven-cells is the distinct tenant-scoped producer. PostgreSQL selects the latest completed/succeeded deployment per case-insensitive service/environment key (matching the existing Estate matrix join), computes the exact count, orders deterministically, and returns at most 50 rows (default 20) without loading tenant deployment history into the application. Each row names the service, environment, release, and deployment and carries persisted landing status, per-target non-verified verdicts, mismatch anatomy, and deployment-attestation reference. Stored mismatch plainReason, action, and action path are returned unchanged; absent cause/cure data is explicit not_recorded. The bounded rows are also enriched with authoritative ownership only: the stored release creator plus active owner grants for that release or environment, split into user and group ids. Revoked grants, other roles, and other tenants are excluded. The response marks the nested ownership assigned or unassigned; it never infers an owner from a service, release, or environment name. An attestation reference proves a record was signed/anchored, not that a later cryptographic verification passed, so the row remains unproven until such a receipt is persisted. Every row links to deployment detail and, when a canonical digest exists, the subject passport. The response never substitutes environment readiness text for deployment proof.

Release

Source: StellaOps.ReleaseOrchestrator.Release/Models/Release.cs. A release bundles an ordered array of ReleaseComponent records (not a string -> ComponentDigest dictionary), tracks lifecycle through ReleaseStatus, and carries the canonical manifest digest set on finalization. There is no Version or SourceRef field on the record.

public sealed record Release
{
 public required Guid Id { get; init; }
 public required Guid TenantId { get; init; }
 public required string Name { get; init; } // lowercase, alphanumeric with hyphens
 public required string DisplayName { get; init; }
 public string? Description { get; init; }
 public required ReleaseStatus Status { get; init; }
 public required ImmutableArray<ReleaseComponent> Components { get; init; }
 public string? ManifestDigest { get; init; } // sha256: of canonical manifest (set on finalize)
 public DateTimeOffset? FinalizedAt { get; init; }
 public Guid? FinalizedBy { get; init; }
 public string? DeprecationReason { get; init; }
 public DateTimeOffset? DeprecatedAt { get; init; }
 public DateTimeOffset CreatedAt { get; init; }
 public DateTimeOffset UpdatedAt { get; init; }
 public Guid CreatedBy { get; init; }
}

public enum ReleaseStatus { Draft = 0, Ready = 1, Promoting = 2, Deployed = 3, Deprecated = 4 }

public sealed record ReleaseComponent
{
 public required Guid Id { get; init; }
 public required Guid ComponentId { get; init; }
 public required string ComponentName { get; init; }
 public required string Digest { get; init; } // sha256:abc123...
 public string? Tag { get; init; } // tag resolved to this digest, if any
 public string? SemVer { get; init; }
 public ImmutableDictionary<string, string> Config { get; init; }
 public int OrderIndex { get; init; } // deployment order within the release
}

Promotion

Source: StellaOps.ReleaseOrchestrator.Promotion/Models/Promotion.cs. State is tracked through PromotionStatus (nine values — see below), not a five-value PromotionState. There is no Decision property; the promotion carries denormalized source/target/release names plus reason fields per terminal outcome, and a DeploymentId once deployment is triggered.

public sealed record Promotion
{
 public required Guid Id { get; init; }
 public required Guid TenantId { get; init; }
 public required Guid ReleaseId { get; init; }
 public required string ReleaseName { get; init; }
 public required Guid SourceEnvironmentId { get; init; }
 public required string SourceEnvironmentName { get; init; }
 public required Guid TargetEnvironmentId { get; init; }
 public required string TargetEnvironmentName { get; init; }
 public required PromotionStatus Status { get; init; }
 public string? Reason { get; init; }
 public string? RejectionReason { get; init; }
 public string? CancellationReason { get; init; }
 public string? FailureReason { get; init; }
 public ImmutableArray<ApprovalRecord> Approvals { get; init; }
 public ImmutableArray<GateResult> GateResults { get; init; }
 public Guid? DeploymentId { get; init; }
 public DateTimeOffset RequestedAt { get; init; }
 public DateTimeOffset? SubmittedAt { get; init; }
 public DateTimeOffset? ApprovedAt { get; init; }
 public DateTimeOffset? DeployedAt { get; init; }
 public DateTimeOffset? CompletedAt { get; init; }
 public Guid RequestedBy { get; init; }
 public string RequestedByName { get; init; }
}

public enum PromotionStatus
{
 Pending, AwaitingApproval, Approved, Deploying,
 Deployed, Rejected, Cancelled, Failed, RolledBack
}

Workflow

There is no workflow-template model in Release Orchestrator. The former StellaOps.ReleaseOrchestrator.Workflow assembly (WorkflowTemplate, WorkflowStep, WorkflowRun) was deleted as unreferenced. Step execution for plugins is StepExecutor in StellaOps.ReleaseOrchestrator.Plugin. The separate src/Workflow engine is dormant and is not called by Release Orchestrator.

Target

Source: StellaOps.ReleaseOrchestrator.Environment/Models/Target.cs. The target has a polymorphic ConnectionConfig, a HealthStatus (not a separate Health of type HealthStatus plus a TargetState — there is no TargetState enum), and an optional inventory snapshot. There is no Labels property on Target (labels live on Agent).

public sealed record Target
{
 public required Guid Id { get; init; }
 public required Guid TenantId { get; init; }
 public required Guid EnvironmentId { get; init; }
 public required string Name { get; init; }
 public required string DisplayName { get; init; }
 public required TargetType Type { get; init; }
 public required TargetConnectionConfig ConnectionConfig { get; init; }
 public Guid? AgentId { get; init; } // null for in-process / agentless dev paths
 public required HealthStatus HealthStatus { get; init; }
 public string? HealthMessage { get; init; }
 public DateTimeOffset? LastHealthCheck { get; init; }
 public DateTimeOffset? LastSyncAt { get; init; }
 public InventorySnapshot? InventorySnapshot { get; init; }
 public bool UseAgentRegistry { get; init; }
 public int AgentRegistryPort { get; init; } // default 8021
 public LifecycleState LifecycleState { get; init; } // default Active
 public DateTimeOffset CreatedAt { get; init; }
 public DateTimeOffset UpdatedAt { get; init; }
}

// Serialized as JSON strings (JsonStringEnumConverter). Note: EcsService (not "ECSService"),
// and AnsibleHost was added in the topology-migration refresh.
public enum TargetType
{
 DockerHost = 0,
 ComposeHost = 1,
 EcsService = 2,
 NomadJob = 3,
 SshHost = 4,
 WinRmHost = 5,
 AnsibleHost = 6
}

public enum HealthStatus { Unknown = 0, Healthy = 1, Degraded = 2, Unhealthy = 3, Unreachable = 4 }

Agent

Source: StellaOps.ReleaseOrchestrator.Agent/Models/Agent.cs. Capabilities are a closed AgentCapability enum plus a free-string ExtendedCapabilities set (used by Sprint 017 EPF deploy plugins such as build.docker, deploy.compose); the union of the two is the authoritative capability set for the task matcher. Status is AgentStatus (Pending/Active/Inactive/Stale/Revoked), not a three-value AgentState (Online/Offline/Degraded), and the heartbeat timestamp is nullable.

public sealed record Agent
{
 public required Guid Id { get; init; }
 public required Guid TenantId { get; init; }
 public required string Name { get; init; }
 public required string DisplayName { get; init; }
 public required string Version { get; init; }
 public string? Hostname { get; init; }
 public required AgentStatus Status { get; init; }
 public required ImmutableArray<AgentCapability> Capabilities { get; init; }
 public ImmutableArray<string> ExtendedCapabilities { get; init; } // EPF deploy-plugin capability names
 public bool RequiresRecapabilities { get; init; }
 public ImmutableDictionary<string, string> Labels { get; init; }
 public string? CertificateThumbprint { get; init; } // mTLS
 public DateTimeOffset? CertificateExpiresAt { get; init; }
 public DateTimeOffset? LastHeartbeatAt { get; init; }
 public AgentResourceStatus? LastResourceStatus { get; init; }
 public DateTimeOffset? RegisteredAt { get; init; }
 public DateTimeOffset CreatedAt { get; init; }
 public DateTimeOffset UpdatedAt { get; init; }
}

public enum AgentStatus { Pending = 0, Active = 1, Inactive = 2, Stale = 3, Revoked = 4 }

public enum AgentCapability
{
 Docker = 0, Compose = 1, Ssh = 2, WinRm = 3, VaultCheck = 4,
 ConsulCheck = 5, Ecs = 6, Nomad = 7, Ansible = 8
}

Managed placement (licence count)

The free-use limit of the LICENSE (Additional Use Grant §2) is expressed in Placements: an Installation may manage no more than twenty active Placements under that grant. This section maps the licence terms onto the Release Orchestrator data that detects them. The LICENSE wording governs; this section states how the product recognises it.

Identity. A placement is keyed by (tenant, environment id, workload name): a workload–environment relationship, never a digest. An image digest is a detection signal that identifies the installed content: a new digest, tag or version of the same component in the same environment is the same placement, and one digest installed as part of several workloads or environments counts once for each of those relationships. The release-component row id (release_components.component_id, copied into deployed_estate.component_id) is minted for each release (comp-<random>), so it is not a workload identity and the count never keys on it. Renaming a component starts a new placement; the old name stays counted until it is retired.

What brings a workload into the count. The Installation detects a workload in an environment from the image hashes it observes there:

1. a Stella Ops deployment of a release to the environment (the deployed estate);
2. a gate decision or promotion of a release for the environment, including a release that another tool deploys;
3. the creation or editing of nodes (targets) that run the workload;
4. running containers that agents report from the environment’s targets;
5. a vulnerability scan of the workload’s image or deployable artifact, or the application of an earlier scan result to it, while it is installed as part of that workload in the environment. Applying an earlier result means associating, presenting or using it as applicable scan information for that installed workload, for example showing it in the workload’s status view; keeping, exporting or viewing a report only as historical evidence does not qualify, and no new scan has to run. Scan data is read through the Scanner owner API, never from the Scanner database.

A relationship that qualifies through more than one of these signals counts once. Where a qualifying scan or reuse concerns content installed in several workload–environment relationships, each of those relationships counts.

Installed. An image or deployable artifact is installed in an environment when it forms part of a deployed workload assigned to that environment, whether the workload is running or temporarily stopped. A digest held only in a registry, cache, backup or unused local store is not installed, including after the workload has been removed. Scanning an image or artifact installed in no environment does not, by itself, create a placement.

Observations are evidence. The signals above are evidence of a relationship, not its only source: deployment records, connected inventory and the operator’s supplementary declaration establish it, and every qualifying relationship the automatic count does not reflect must be declared. Losing an observation does not by itself show that a workload was uninstalled or withdrawn from management. Which signals the recorded count reads today is stated under Implementation status.

What never adds a placement for the same workload in the same environment: replicas, restarts, redeployments, new digests, tags or versions, several files that form one native workload, repeated scans, reports, and superseded or rolled-back deployed-estate rows.

Stella Ops’ own components are never placements. They are recognised by the reserved com.stellaops.* container label, never by a name pattern. An unlabelled container with a Stella-like name is still reported as drift and still counted.

When a placement ends. Under the licence a placement ends when all management of that workload in that environment actually ends, or when the workload is removed and no other management activity keeps the relationship active. A completed scan or a stopped workload that stays deployed does not end it, and keeping historical evidence or reports does not, by itself, create, reactivate or keep it active. The product records the end through the explicit retire (unmanage) operation, because the absence of a workload from the latest deployment or observation is not by itself an end signal. Managing a retired workload again (a new deployment or gate decision) makes it count again from that time.

Recorded, declared and total. The automatic count is the recorded count. Placements that the automatic count cannot detect are entered by an administrator as declared placements, the supplementary placement inventory. A pair that is both recorded and declared counts once. While the automatic count does not cover every detection signal above, the API and the console report the recorded count together with its coverage, and never present it as the definitive licence total.

Environments shared between tenants. The count cannot tell that two environment records in different tenants describe one physical destination. Such a destination counts once per record; the operator retires the duplicate placements.

Not this count. The placementCount and brokenPlacementCount fields of GET /api/v1/release-orchestrator/estate/chain-completeness count distinct (environment, digest) pairs, the custody unit of the evidence chain: a new digest of the same workload adds one there and none here. The licence count is published only under the managedPlacements name.

Reading the count. GET /api/v1/release-orchestrator/estate/placements (stella estate placements) returns managedPlacements.recorded, breakdowns per tenant and per environment, and every placement with its detection signals, digests and newest signal time. scope=tenant (the default) counts the calling tenant under orch:read, the authority of the deployed-estate view; scope=installation (--installation) counts every tenant and also requires ops.admin. The response carries no read time and orders every list by tenant, environment id and workload, so two reads of the same estate are byte-identical. coverage lists the counted and the not-yet-counted detection signals; while any signal is not counted, definitive is false.

Implementation status. The recorded count reads the first detection signal only: the deployed estate (deployed_estate rows in status deployed). Until the retire operation exists, a workload that leaves an environment’s latest deployment also leaves the recorded count. The exclusion of Stella Ops’ own containers, the retire operation and the declarations are built in SPRINT_20261027_003. Until the declarations exist, add the supplementary inventory by hand and leave out Stella Ops’ own components.

Database Schema

Scanner event delivery ownership

Source verified against commit 163f35a2ce on 2026-10-03 for this subsection and its eventing migration/pool additions below. Re-verify from the repository root with rg -n 'ScannerEvent|EventingMaxPoolSize|AddEventingReliability' src/ReleaseOrchestrator/__Apps/StellaOps.ReleaseOrchestrator.WebApi/Program.cs src/ReleaseOrchestrator/__Apps/StellaOps.ReleaseOrchestrator.WebApi/Services/ReleaseOrchestratorPostgresPoolBudget.cs. The earlier document-wide snapshot does not cover this contract; this is source evidence, not a deployment receipt.

Release Orchestrator publishes ComponentVersionScmInfoAvailable and BuildContextReachabilityAvailable to tenant-partitioned P6 streams in its own database. PostgresScannerEventStreamStore commits the semantic domain-event row, delivery envelope and publication marker in one transaction. Startup imports retained unmarked events before serving requests, preserving their source IDs. The existing domain log remains evidence; the P6 outbox provides delivery sequence, epoch and retention state. Eventing uses a dedicated owner pool and startup migration path.

The closed StellaOps.ReleaseOrchestrator.ScannerEvents.Contracts project defines two literal HTTP partitions under /internal/v1/release-orchestrator/scanner-events/: scm-info and build-context. Each exposes GET .../events and POST .../consumer-checkpoint. Both require authenticated tenant scope and release:read; checkpoint reporting additionally requires the client selected by ReleaseOrchestrator:ScannerEvents:ConsumerClientId (default stellaops-scanner-worker). The owner derives stream and consumer identity rather than accepting caller-selected names.

Scanner owns its handler effects, inbox, checkpoints and leases. It reports only persisted progress, including renewal while caught up, so active remote consumers constrain owner retention. Invalid epochs or positions beyond the owner head are rejected. Scanner stops the affected partition on a retention gap or epoch change. This boundary replaces direct Scanner SQL against release_orchestrator.domain_events; operational configuration and recovery are in the Scanner event guide.

The scoped OpenAPI capture and generated reference cover only these four operations. See the API changelog for source provenance and the limits of the isolated owner-event runtime proof.

Migration ownership

RO domain tables are created by the migrations embedded in StellaOps.ReleaseOrchestrator.Persistence (Migrations/*.sql, applied on startup by AddStartupMigrations, schema release_orchestrator, module ReleaseOrchestrator.Consolidated). The chain is the consolidated baseline 001_release_orchestrator_consolidated_baseline.sql plus forward migrations 002–011 (retention-class comments and cleanup functions; 009 retires RO’s topology inventory projection, 010 adds the Scanner stream publication marker, and 011 restores the domain log’s retention classification and custody metadata). The Environment and Agent libraries hold only _archived/ SQL; Scripts has none. The baseline declares the family schemas release_orchestrator, release, release_orchestrator_agent and scripts, the RLS helper schema release_app, the offboarded-tenant archive schema release_orchestrator_archive, and shared (only shared.tenants, an Authority catalog RO reads). The same database also holds crypto (KEK ledger, credential-store chain), platform (plugin registry), and eventing (P6 retained delivery), migrated by shared libraries. The database is stellaops_release_orchestrator, reached only through STELLAOPS_POSTGRES_RELEASE_ORCHESTRATOR_CONNECTION.

PostgreSQL pool-budget precedence

Release Orchestrator owns seven independent long-lived PostgreSQL pools: main persistence, environment, agent, scripts, KEK, plugin registry, and eventing. Limiting only the main pool does not protect the rest of RO’s own database (stellaops_release_orchestrator, moved off the shared stellaops_platform database 2026-08-22 — see docs/architecture/database-ownership-matrix.md). The default aggregate ceiling is 28 connections (8 main, 8 environment, 4 agent, and 2 each for scripts, KEK, plugin registry, and eventing), with a 60-second idle lifetime and zero warm auxiliary connections (ReleaseOrchestratorPostgresPoolBudget).

ReleaseOrchestrator:Postgres:MaxPoolSize remains the explicit main-pool override. EnvironmentMaxPoolSize (default: the main maximum), AgentMaxPoolSize, and AuxiliaryMaxPoolSize allocate the other shares; the auxiliary share applies to KEK and plugin registry, and to scripts unless Scripts:Postgres:MaxPoolSize is set. Eventing uses ReleaseOrchestrator:Postgres:EventingMaxPoolSize, falling back to the auxiliary share. Main and auxiliary idle lifetimes use ConnectionIdleLifetimeSeconds and AuxiliaryConnectionIdleLifetimeSeconds.

These settings override larger limits carried by a shared connection string. Without a module setting, the bounded defaults above apply; the shared PostgresOptions default of 100 must never silently expand an operator budget. Application names and runtime connection defaults continue to be normalized by PostgresConnectionStringPolicy so pg_stat_activity retains per-pool attribution.

release schema (topology / agent registry)

Created by the Persistence consolidated baseline; release.agents uses the Agent-store column shape. Migration 009_release_orchestrator_retire_topology_inventory_projection.sql (RFC-14) drops RO’s nine release.topology_* inventory and watermark tables; it refuses to run if any of them holds rows. Platform keeps its own rebuildable topology projection fed by the topology-snapshot endpoint.

Tenant-owned topology tables, including release.environments, release.regions, release.targets, and release.infrastructure_bindings, must use the current tenant id on primary-key, natural-key, scope/list, and delete/update paths. Environment names and order indexes, region names and sort orders, and target names are tenant-local; lookup methods such as GetByNameAsync, GetByOrderIndexAsync, ListOrderedAsync, ListByEnvironmentAsync, and ListByAgentAsync must predicate on the active tenant instead of relying on globally unique ids or database session state alone. The infrastructure binding store also rejects creates whose model tenant differs from the active tenant, so registry/vault/settings-store bindings cannot be read or removed across tenants by guessing a binding id, scope, or integration id.

The standalone WebApi resolves that active tenant through ReleaseEnvironmentIdentityAccessor. Ordinary principals stay claim-first. For global-admin Console requests, Sprint 20260530.033 allows the canonical X-StellaOps-TenantId header to select the tenant without re-issuing the token; this is the Release Orchestrator side of ADR-023. The live Sprint 034 forcing function created dev environments in both e2e-lab and customer-dev-lab through the same admin token and verified a tenant-scoped list only returns that tenant’s row.

The release_orchestrator_agent schema holds release_orchestrator_agent.agent_tasks (durable agent task queue, cleaned by release_orchestrator_agent.cleanup_agent_tasks, migration 008).

Evidence-thread tables (release.evidence_threads and siblings) are not created by any live migration. PostgresEvidenceThreadStore in the EvidenceThread library queries them, but the WebApi does not register that store.

Topology setup compatibility aliases

ReleaseOrchestrator is the sole HTTP, mutation, and migration owner for the deployment-topology compatibility families /api/v1/regions, /api/v1/environments, /api/v1/targets, /api/v1/agents, /api/v1/infrastructure-bindings, and /api/v1/pending-deletions. These aliases moved from Concelier on 2026-07-11 without requiring Web client URL changes. They reach ReleaseOrchestrator through its Router-published (HELLO) endpoint set; the static Router rows for these families were retired on 2026-08-28 (RAR-5/RAR-7). Every relocated handler requires tenant context plus orch:read for reads or orch:operate for mutations.

The environment alias covers only list/create, readiness, rename, and deletion request shapes; the agent alias covers the exact list and deletion-request routes, and the integration compatibility route covers only deletion requests. Agent/integration rename aliases were not carried over because the environment library cannot implement those owner-specific actions. Their deletion-request aliases remain addressable but fail closed with the rest of the deletion lifecycle.

GET /api/v1/environments/{id}/readiness returns 200 with an empty list when an environment is not registered. A Router-generated 502 on this path therefore means the Router could not reach the published ReleaseOrchestrator endpoint; it is a routing/service-availability defect, not an unevaluated readiness result. Region/environment/target rename paths remain addressable but return 503 topology_rename_unavailable; the former implementation changed only the display label while reporting the immutable name as changed.

Pending deletion is fail-closed. Request and confirmation aliases return 503 topology_deletion_unavailable until cascade discovery and idempotent execution are implemented. ReleaseOrchestrator does not start the historical worker that marked a confirmed deletion completed without deleting its target. Existing pending records remain readable and cancellable under tenant scope.

release_orchestrator schema (release truth + deployment engine)

Created by the Persistence consolidated baseline; auto-migrated by the WebApi on startup.

Registry upstream credential references

release_orchestrator.registry_upstreams.credential_ref holds a reference to a runtime secret source, never the secret. UpstreamCredentialReference (Storage library) is the single contract, applied by the PUT /api/v1/registry/upstreams endpoint and again by PostgresUpstreamRegistryStore:

No pull path reads the stored credential_ref in this release; deploy-path pulls authenticate with the component’s registry-integration credential through the digest-centric content broker. The read path does not rewrite stored values; replacing a withheld value is an operator PUT (see docs/runbooks/registry/registry-bridge-cache.md §2.2).

Persisted JSON compatibility contract

Writers must serialize the full current object contract into each JSON projection; a scalar, ratio, or partial nested object is not interchangeable with that contract. The persisted shapes that previously caused whole-list failures are:

The list/detail read contract is row-scoped. If one stored JSON projection is malformed or from an incompatible older writer, canonical scalar columns recover the row identity and the API returns readError.code = stored_json_contract_mismatch; the console renders that marker and disables mutations for the affected record. Surrounding rows and surrounding deployment-log elements remain readable. Mutation paths remain strict and fail rather than rewriting an unreadable projection. The contract is pinned by ReleaseTruthRepositoryPersistenceTests, DeploymentCompatibilityStorePersistenceTests, and GateDecisionListingEndpointTests.

Audit content hashing is forward-only and epoch-aware. Hash v3 is the first storage-round-trip- stable epoch for new rows; v1/v2 rows that cannot reproduce after PostgreSQL timestamp/JSON normalization retain structural coverage and return honest partial deep-verification coverage, not a false tamper verdict. The consolidated baseline’s trigger trg_ro_audit_hash_version_immutable makes the persisted hash_version immutable after insert. The exact endpoint semantics and threat boundary are documented in Audit Trail.

scripts schema (release-script registry)

Created by the Persistence consolidated baseline (the legacy sample-script seed rows were not carried over); auto-migrated by the WebApi.

The catalog is partitioned by tenant; see Script catalog tenant partition.

Release Truth Persistence

The standalone WebApi auto-migrates the release_orchestrator schema on startup. The release-truth DDL lives in the consolidated baseline 001_release_orchestrator_consolidated_baseline.sql (folded in from the pre-1.0 003_release_truth.sql, which is archived under Migrations/_archived/pre_1.0/mig061/ and excluded from embedding; live forward migrations are 002–015). It provides the following durable tables for the compatibility endpoint cutover — a focused subset of the full release_orchestrator table list enumerated above:

Runtime rule (updated 2026-07-12): protected promote/deploy paths must not infer success from seed/demo data. Release and approval HTTP compatibility paths read and mutate durable release truth rows, and approval decisions are idempotent per actor so one approver cannot inflate quorum by repeating the same decision. Missing approval quorum, WAITING/empty/advisory/skipped gate truth, non-passing gates, missing verified evidence, unsigned/unbound protected decision evidence, or evidence hash mismatch returns a typed blocked/conflict response. The only override that can clear a gate is the dedicated GateException decision: release gate-bypass scope, a verified operator envelope, a 1-365 day expiry, durable reason/risk acknowledgement, evidence reference, and an explicit missingEvidence list are mandatory. Conditional/permanent exceptions are rejected until their conditions have a deterministic enforcement contract. Deployment requests are persisted as queued/pending deployment state and only executor completion may advance them to deployed/completed.

Weekly dashboard projection (updated 2026-07-18): authenticated release:read callers can read GET /api/v1/release-orchestrator/releases-this-week. The tenant-scoped response uses a Monday-based UTC calendar window. startInclusiveUtc is inclusive, endExclusiveUtc is the following Monday, and asOfUtc is the effective upper bound: future-dated rows inside the nominal calendar window are not reported. Shipped rows are anchored to stored deployedAt. Blocked rows are current failed releases updated in the observed window or the latest in-window approval when its stored gate result, quorum, rejection, expiry, or missing gate pass says it is blocked. Failed gate rows carry the stored gate id/name/message and a stored ruleId, policyId, or ruleName when one exists; legacy failures with no recorded reason explicitly return reasonState=not_recorded.

The response reports exact in-window totals and caps every emitted item/summary list at 50 with truncated and maxItems. Ordering is deterministic (event time descending, then release id ordinal; ownership summaries use count descending, then id ordinal). Ownership is never derived from release/service names or the promotion requester. Owner subject ids come only from persisted release createdBy and active release Owner user grants; team ids come only from active release Owner group grants. Missing ownership is explicit as state=unassigned. Owner-grant reads cover every distinct candidate in the current partial UTC week so owner/team and unassigned totals remain exact; the response lists remain hard bounded.

shippedHistory is the bounded trend contract for the same dashboard answer. It always returns eight contiguous Monday-based UTC buckets in chronological order, ending with the current week. Counts come only from persisted release deployedAt; timestamps before the oldest bucket and after asOfUtc are excluded, and the current bucket is marked partial=true. The source is stated explicitly as release.deployed_at. No blocked-history series is claimed because the current release/approval projection does not retain an immutable blocker event history; deriving one from mutable latest status would rewrite past weeks. The dashboard consumer must preserve these honesty states rather than inventing a reason, owner, team, or blocked trend.

Blocked-release explanation projection (updated 2026-10-11): authenticated release:read callers can read GET /api/v1/release-orchestrator/release-blockers. The endpoint answers the current all-release question, not the calendar-week rollup: PostgreSQL selects only the latest approval per release, returns failed releases plus approvals with blocking or advisory gate truth, computes the exact tenant total, and emits at most 50 rows ordered by stored observation time descending then release id ordinal. The application does not load the tenant’s unbounded release/approval history.

Gate exceptions are read with the approve and dispatch rule (ReleaseTruthGuard.IsGateSatisfied): a gate row with status exception is satisfied only while it is a signed, time-limited exception with recorded evidence and an expiresAt after the read instant. The stored gatesPassed flag stays true after such an exception expires, so the candidate query also selects a pending or approved approval carrying an exception row the guard refuses (approve refuses the pending one, dispatch the approved one), judged at the same instant the response reports as asOfUtc. Such an approval is a block item. Other approval states (for example deploying or deployed) are not selected for an expired exception, and when a release is listed for another reason (a failed release, or a rejected, expired, cancelled or superseded approval), an exception row of that approval is not a cause, because no approve or dispatch evaluates it.

Each item distinguishes block from warn. A gate row is a cause when ReleaseTruthGuard.IsGateSatisfied rejects it: only passed and an active signed exception are satisfied. A refused exception row keeps its stored gate id, name and status (exception); its message names the recorded expiry (Gate exception expired at <instant>) or the missing active signed evidence, with the reason code gate_exception_invalid_or_expired that approve and dispatch return. Causes carry the stored gate id, name, status, message, and stored rule identity when gate details contain ruleId, policyId, ruleName, packId, or Policy Engine cveMatches[].ruleName. The latest gate decision contributes its stored policy id/mode when present. Approval quorum, rejection, expiry, missing gate-result truth, and failed release state are named as separate stored-state causes. Cause lists cap at ten with causesTruncated; absent or malformed legacy rule detail remains ruleIdentity.state=not_recorded rather than throwing or guessing.

Remediation linkage is deliberately honest. Only an explicit stored gate-detail link (fixLink, remediationLink, fixUrl, remediationUrl, remediationUri, or actionUrl) becomes remediation.state=recorded; otherwise the response says not_recorded. Release Orchestrator does not call or search Policy again while serving this projection. Owner subjects come only from the stored release creator and active release Owner user grants; teams come only from active release Owner group grants. Missing ownership is unassigned. This projection does not store or claim digest-scoped service identity or service ownership. Stable Policy rule-to-remediation linkage remains a cross-module successor; Web must preserve the explicit absence states until that contract exists.

Developer self-service rule (updated 2026-07-12): developer principals are not authorized by broad release:write alone. Release and deployment list, detail, and release-component reads are server-filtered by the caller’s scoped ownership and environment claims; a contributor token without ownership claims receives an explicit missing_ownership_claims zero-data projection rather than tenant-wide rows labelled as personal data. Non-developer principals, including admin-family roles, are not silently converted into developer principals: their list reads remain tenant-scoped and still require the authenticated tenant plus release:read; the role is never a scope bypass. Service and repository ownership uses canonical exact identity, while image namespaces use a path-boundary match, preventing neighboring names or repositories with the same leaf from matching. Deployment read filters advertised by the Web client (serviceIds, initiating developer, source type, and current/candidate digest) are applied by the backend, and the compatibility projection persists those developer-update fields. POST /api/v1/release-orchestrator/deployments applies service/environment/digest ABAC when the caller is a developer principal or carries developer ABAC claims. Developer write decisions are made on owner records, not on request labels (updated 2026-10-09, CCP-017): the named release must be recorded in the caller’s tenant and owned under the same predicate the release read projection uses (release id or name, a recorded component id or name, a component source repository, or a component image reference); the request’s serviceId, serviceName and packageRefName must each name that release or one of its recorded components as a whole (the release id or name, a component id or name, also as a slug for a label without / or :, so Payments API names payments-api; or exactly a recorded image reference, its repository, the repository with the recorded tag or digest, or the recorded source repository), because the developer deployment projections expand a persisted label into its path leaf and colon prefix, and a label such as billing-api:x/payments-api would otherwise reach billing owners; and the target environment is resolved with the deploy engine’s precedence (environment id as a GUID, then as a name, then the name hint), so the environment claims and the maxEnvironmentType ceiling are evaluated on the environment a deploy would reach, with the environment’s recorded production flag deciding the production ceiling ahead of the name heuristic. An unresolvable target is refused. Direct deployment additionally requires directUpdateAllowed=true and an immutable sha256: package digest. The developer evaluation runs before the vulnerability floor and the protection-class gate, so a refusal carries no gate or vulnerability result of another service’s release; an unrecorded release and another service’s release are refused with the same reason, and a refusal writes no deployment record and starts no job. A released bundle is immutable for developer self-service: the candidate digest must be one the release records. Any other digest is answered with 409 release_artifact_update_requires_new_version, the same error code the component update path uses, with releaseId, candidateDigest, the actionEndpoint of POST /api/v1/release-orchestrator/releases/{releaseId}/versions/from-artifact-update and, when the request identifies the changed component, the successor-version suggestion, whose targetEnvironment is the resolved environment’s name; the developer creates the successor version and then deploys or requests approval for it. POST /api/v1/release-orchestrator/approvals/developer-requests uses the same record-bound service/environment/digest checks without requiring directUpdateAllowed. It requires releaseId (400 release_id_required otherwise) and persists an approval row with SourceType=developer_update, the recorded release name and version, the recorded component whose digest the request carries, service id/name, candidate digest, the resolved environment id (as a promotion approval records it) as the target environment id, the resolved environment label as the approval target, actor, and audit details. A digest that no release records is not turned into an approval for a derived release key; it goes through the successor version first. For a successor version, developer principals are authorized on the source release and the recorded component being changed; a requested imageRef must stay in that component’s recorded image repository, and the successor’s target environment is resolved and evaluated like the other developer paths (an unresolvable target is refused). GET /api/v1/approvals/{id}/replay returns the v2 approval replay status from the persisted decision digest and evidence packet so the approval-detail UI can verify replayability without falling back to an unimplemented compatibility route.

Approval queue cancellation (CCP-090, 2026-10-14): Source checkpoint verified against 3946e97d0baadafe129093d8c07cb2a075150424; recheck the BatchCancel HTTP cases with the canonical targeted xUnit wrapper in StellaOps.ReleaseOrchestrator.Integration.Tests. POST /api/v1/release-orchestrator/approvals/batch-cancel requires the authenticated tenant, release:read and either release:write or release:publish. A developer/contributor principal or a principal carrying developer ownership claims also needs service ownership and environment authorization claims. The owner checks the tenant’s recorded release and components and the registered target environment; a role label grants no scope or ownership bypass. The recorded target environment ID takes precedence, with the recorded target reference used only when that ID is absent. An unresolved explicit ID is not replaced with a friendly display label. Missing or foreign ownership/target authorization returns 403 approval_cancel_forbidden before any approval is changed. Unavailable owner reads fail before any write and cannot grant cancellation. Every known row in the requested batch must pass, including a terminal row, so a later foreign row cannot follow an earlier cancellation. Unknown and foreign-tenant IDs remain indistinguishable skipped IDs. After authorization, only pending or expired rows are cancelled; authorized terminal rows are skipped, duplicate IDs produce one action, and the existing actor, timestamp and comment attribution is retained. Scope-authorized unrestricted operators keep the existing queue-cleanup behavior. This is authorization of a recorded snapshot, not a database-wide transaction: the current repository writes sequentially and does not offer conditional status/version cancellation or rollback for a later persistence failure or concurrent writer. Source/HTTP qualification does not establish deployed restricted-persona acceptance; CCP-045/048 retain that adoption and live evidence gate.

The full Release Orchestrator OpenAPI/reference refresh also publishes the four retained Scanner-event routes already implemented by DCPU-2 source 0c9c5b7561c8c425ad736603e8891aaf4aa0bbb1. The full host capture omits their bearer, required-scope, tenant and consumer-client annotations. This is an OpenAPI metadata limit, not anonymous access: the serving routes retain release:read, tenant isolation and the configured consumer-client restriction for checkpoint reports. The scoped Scanner-event reference and scoped contract retain those stronger annotations unchanged. No route, role, scope or runtime grant is added by this documentation refresh.

Deployment execution reality (updated 2026-07-16): the standalone WebApi drives HTTP deploy requests through LibraryDeploymentRequestStarter into the canonical library DeployOrchestrator. POST /deployments still returns immediately after the library job and compatibility read projection are persisted, while the starter keeps a dedicated deployment scope alive until the library engine reaches terminal state. IDeploymentJobStore is backed by release_orchestrator.deployment_jobs in production WebApi DI, so deploy-engine job state survives service restarts instead of depending on the process-local in-memory store. Task-level updates use row-locked UpdateTaskAsync writes so parallel blue-green target completion cannot overwrite sibling task state. Terminal DeploymentJob state is projected back into release_orchestrator.deployments, including the real job start/completion times and per-target status/progress/error while preserving the real target ids, names, agent ids, strategy evidence, and rollback status. The current successful job wire value is succeeded; persisted legacy completed values are equivalent success aliases for target rollups, rollback eligibility, list filters, and filtered totals. A successful row promotes its canonical candidate digest to currentDigest; the read normalization applies the same rule to pre-fix successful rows while failed or incomplete rows retain their previous current digest. Progress measures finished target work independently of outcome, so a failed deployment may truthfully be 100% finished without becoming successful. Each terminal mirror also upserts one deterministic terminal event and recomputes total duration, average target duration, and target success rate; repeated mirrors do not duplicate the event or target logs. Read paths apply the same deterministic projection to legacy rows whose serialized rollups, operation flags, terminal event, metrics, or successful current-digest subject are stale, without rewriting their stored outcome. The WebApi composition root registers the library deployment strategy factory and real artifact generator (ComposeLockGenerator, VersionStickerGenerator, DeploymentManifestGenerator) against read-through release-truth adapters for IReleaseManager and IPromotionManager. Those adapters map existing string-based release, component, environment, and approval ids to stable GUIDs, pass component ImageRef into the compose writer’s repository config, and fail closed for unsupported mutation methods; release and approval HTTP endpoints remain the mutation authority. Deployment and promotion domain events are persisted through PostgresDomainEventPublisher into release_orchestrator.domain_events with tenant, aggregate, occurrence time, and JSON payload metadata. Live verification now covers library-engine rolling/canary/blue-green dispatch through real Testcontainers sshd targets, dev in-process SSH/WinRM paths, and the target-resident StellaOps.Agent.Host ComposeHost and DockerHost paths through the mTLS-polling agent-runtime contract. StellaOps.Agent.Host can also compose SSH and WinRM capabilities for out-of-process task execution, agent registration now persists through PostgresAgentStore into release.agents, and registered agents receive CA-signed client certificates from the file-backed internal agent CA; full live proof for out-of-process SSH/WinRM remains tracked in Sprint 20260506_002 alongside compose bootstrap.

Normal deployments and per-target retries use the same authoritative in-flight mirror: while the engine operation is active, changed non-terminal job/task state is sampled from IDeploymentJobStore and projected into release_orchestrator.deployments. Terminal evidence and target-log append remain on the terminal path. This keeps API and CLI watch consumers on real target status/progress instead of leaving a retried target at compatibility pending until completion.

Remediation closure in the release lineage (CCP-072)

Release Orchestrator owns the record of remediation closure (acceptance manifest owner decision 19, 2026-10-11): which later release closed an earlier release’s denied decision or gate exception, and how. The release lineage is the set of releases in one tenant that carry the same release name, the identity that latest-by-name, the version suggestion and the successor-version path (versions/from-artifact-update, invariant 1) use. When a release is promoted, its promotion decision addresses the open cases of the releases of its lineage that were created before it, in the same target environment (matched by the recorded environment name, as gate decisions are):

The successor decision is the promotion policy decision, the gate decision written when the promotion is requested. Closure is decided and recorded at that point, not when a person approves: when the environment or the policy requires manual approval, the decision has outcome approve while the approval stays pending. A later approver rejection of the successor writes no gate decision, does not change a recorded link and does not reopen the case, and the rejected successor does not become a case (its latest promotion decision is approve). The link’s successorApprovalId names the approval whose state shows whether the successor was approved by a person.

A gate exception names one gate row of the earlier release’s approval, and built-in gates write per-release gate ids: reachability-gate-<witness evidence id>, sbom-readiness:<digest> for a waiting or failed component next to the single passing row sbom-readiness, and policy-engine-<pack>-v<version>. The successor’s rows are therefore matched by gate kind, not by gate id. The kind of a built-in gate row is its gate result type (sbom-readiness, reachability-gate, policy-engine, approval-policy, attestation-gate). The kind of a row of the generic plugin projection (marked by its pluginId detail) is its gate id, which is the plugin’s gate type; its gate result type is only the plugin’s category, so another plugin of the same category does not match. A successor decision without a row of that kind records gate_not_evaluated, and one with a row of that kind that is not passed records gate_not_passed.

For each open case the decision records one link in release_orchestrator.release_remediation_links with the closure it earned:

• remediated: the case closes, and the successor’s recorded component digests differ from the earlier release’s (a new artifact).
• cleared: the case closes, but the two digest sets are equal or one of them is not recorded; the case closed through changed inputs (evidence, VEX or policy), not through an artifact the recorded components show to be new.
• not_remediated: the decision does not close the case. closureReason names why: successor_denied, successor_blocked, successor_waiting, successor_unknown, successor_not_approved, gate_not_passed or gate_not_evaluated.

The closing reasons are policy_approved (denied_decision) and gate_passed (gate_exception). A case stays open until a remediated or cleared link names it; later successors then record nothing for it. A successor whose own promotion is denied becomes a case for the next successor.

The links are written in the transaction of the successor’s decision on both decision write paths: without decision evidence sealing, and with it, where the decision, its seal intent and its links commit together. A failed write leaves none of them. A recorded link cannot change (trigger, SQLSTATE RO072); a tenant offboarding archives the rows like the gate decisions they link. The earlier decision, approval and exception are not modified, and no runtime migration seeds a row.

The release detail read (GET /api/v1/release-orchestrator/releases/{id}) carries remediationLinks: every link in which the release is the successor or the predecessor, in recording order and scoped to the caller’s tenant. An empty list means that no link is recorded. The release list read does not load the field and omits it, and release_json never stores it. Each link carries caseKind, caseKey (the earlier decision id, or the approval id, # and the gate id), targetEnvironment, predecessorReleaseId, predecessorDecisionId, predecessorApprovalId, gateId, exceptionId (the exception’s operator-decision digest, or gate-exception:<gate id>), predecessorDigests, successorReleaseId, successorDecisionId, successorApprovalId, successorOutcome, successorDigests, closure, closureReason and recordedAt (the successor decision’s time).

Limits: closure is read only from recorded gate decisions, approvals and component digests; a filtered finding list, a Console state or a Findings workflow record cannot close a case. Only promotion decisions record links. Developer approval requests, operator lifecycle decisions and the seal aggregate (POST /api/v1/release-orchestrator/releases/seal, which writes an approve gate decision before it dispatches the deployment) record none; the seal aggregate’s approve decision is still the release’s latest promotion decision, so a sealed release closes its own earlier denial without a link, like a release that is approved later by itself. A release renamed after its decision leaves its lineage for later promotions, while recorded links keep their release ids. Two concurrent promotions in one lineage can each record a closing link for the same case. The lineage is read from the tenant’s release list (the read latest-by-name uses), so that the name comparison is the service’s trimmed, case-insensitive ordinal comparison and not the database collation; the predecessors’ approvals are read by release id and environment.

Deployment Share Grants (Sprint 20260608_001)

See ADR-027 for the decision record (alternatives rejected, the seven invariants, Production 4-eyes).

An operator on the Console deployment monitor (/releases/deployments/{id}) can hand a colleague a share link so they can watch — or take over — a single live deployment. The link carries an opaque grantId (dsg_<32 hex>), not a credential: opening it does nothing privileged. The recipient must sign in through the existing OIDC flow and POST.../share/{grantId}/accept; only the accepted subject is ever elevated, scoped to that one resource, time-boxed, revocable, and audited. There is no new global scope and no Authority/identity-envelope change — delegation is enforced inside ReleaseOrchestrator against the request principal.

Capability levels

The platform never issues a half-elevated grant. An OperateDeploy grant satisfies a required View; a View grant satisfies only View.

Endpoint group

Mapped under /api/v1/release-orchestrator (ShareGrantEndpoints); the whole group requires authentication + tenant + release:read. Elevated decisions are enforced in DeploymentShareGrantService against the request principal.

Resource-scoped authorization fallback

The existing operate endpoints (/deployments/{id}/pause|resume|cancel|rollback, target retry) and initiate endpoints (/releases/{id}/deploy|promote|rollback) keep their native scope check unchanged and add a fallback via ShareGrantAccessGuard: the action is authorized when the caller holds the native scope (or is a global admin) OR has an active, accepted OperateDeploy grant for this exact resource. The grant for deployment D1 never authorizes D2; a deploy-grant for (releaseR, envStage) never authorizes (releaseR, envProd) nor another release. A privileged action authorized via a grant emits a release.share.used audit event (real acting subject + grantId + capability + route + resource ids) and atomically consumes a use; the native-scope path emits no share.used. (Implementation: the per-route elevated RequireAuthorization moved into the handler so the OR-fallback is reachable — ASP.NET authorization middleware runs before the handler. The native check is byte-identical, so the native path is not weakened.)

Lifecycle and persistence

Three tables in the release_orchestrator schema, created by the consolidated baseline 001_release_orchestrator_consolidated_baseline.sql (CHECK-constrained enum columns; JSONB scopes_granted/subjects; FK-cascade on acceptances/approvals), auto-applied on WebApi startup: deployment_share_grants, deployment_share_grant_acceptances, deployment_share_grant_approvals. Default lifetime 24h (cap 30 days), optional maxUses budget (consumed atomically), immediate revoke, auto-revoke on terminal deployment state, plus a periodic DeploymentShareGrantExpirySweeper that transitions expired grants to Expired. Authorization is additionally denied when the target deployment is already terminal (covers engine-driven succeeded/failed completion outside the HTTP path).

release.share.{created,toggled,accepted,approved,revoked,used} audit actions (AuditModules.Release, resource type deployment_share_grant) provide full attribution.

The same MapShareGrantEndpoints() call also maps the resource-access surface (ResourceAccessEndpoints.cs): /releases/{id}/access[/effective|/transfer|/{grantId}] and /environments/{id}/access… under /api/v1/release-orchestrator, backed by release_orchestrator.resource_share_grants.

Gate Types

Reconciled 2026-05-30; gate inventory corrected 2026-07-12. GateResult.GateType is a free-form string, not an enum. Four built-in gate plugins are registered into the BuiltInGateProviderRegistry and therefore into the plugin-backed promotion path (the IGateProviderRegistry factory in StellaOps.ReleaseOrchestrator.WebApi/Program.cs, registration order preserved):

1. SbomReadinessGatePlugin — sbom-readiness, runs first; blocks promotion until the SBOM is ready (see §“SBOM readiness gate”).
2. ApprovalPolicyGatePlugin — approval-policy, wrapping PolicyDrivenGateDecisionEvaluator (also performs the CVE-aware Policy.Engine call).
3. ReachabilityGatePlugin — reachability-gate, wrapping ReachabilityGateEvaluator (sprint 20260531_054).
4. AttestationGatePlugin — attestation-gate, deploy-time attestation gate on the live chain (sprint 20260704_006 / GDB-3).

The attestation gate keeps two identifiers deliberately separate. Its Attestor list query uses the artifact kind scan-result (option ScanArtifactKind), the indexed meta.artifact.kind; the DSSE payload still carries the full in-toto predicate URI stellaops.io/predicates/scan-result@v1. A candidate is verified only when it is subject-bound to the component digest, its Attestor log status is included, and its persisted DSSE signature keyid matches a key currently published by Scanner’s JWKS surface. Absent, pending, failed, and unknown log statuses remain invalid even for a trusted signer. The caller client id authenticates the submission transport and is not a signing-key identity. Verification results also carry a stable evidenceGapCode; callers never infer a gap from prose. A bound entry whose log status is not included is transparency_anchor_failed, while no bound entry is attestation_missing, other invalid attestations are attestation_invalid, and transport/trust-service inability is verifier_unavailable. The gate preserves that code on each component and emits aggregate evidenceGapCount plus evidenceGaps details. A transparency-anchor gap is counted in advisory mode and denies in an existing blocking environment; a merely missing entry is never relabelled as an anchoring failure.

The built-in gate registry feeds PluginBackedGateDecisionEvaluator, the sole IGateDecisionEvaluator (the UsePluginRegistry flag and legacy CompositeGateDecisionEvaluator fall-through were deleted pre-release, P8-11). Additional gate-result types — attestation, vex, reachability, security-gate — surface in policy-gate simulation and decision evidence (StellaOps.ReleaseOrchestrator.PolicyGate). The conceptual gate categories below describe the intended promotion-gate taxonomy, which is broader than the four registered built-ins.

Plugin System (Three-Surface Model)

Note. The snippets below are non-normative illustrations; no types with these names or signatures exist. The real plugin capability interfaces live in StellaOps.ReleaseOrchestrator.Plugin/Capabilities/: IIntegrationConnectorCapability, IScmConnectorCapability, IRegistryConnectorCapability, IStepProviderCapability (GetStepDefinitions, ExecuteStepAsync, ValidateStepConfigAsync, GetOutputSchema), IGateProviderCapability, ICiConnectorCapability, INotifyConnectorCapability, ISettingsStoreConnectorCapability, and IVaultConnectorCapability. There is no IStepProvider interface, no RollbackAsync and no StepExecutionCharacteristics. Digest resolution for release creation uses IRegistryConnector (…Release/Registry/: ResolveTagAsync, GetManifestAsync, ListTagsAsync).

Plugins contribute through three surfaces:

1. Manifest (Static Declaration)

# plugin-manifest.yaml
name: github-integration
version: 1.0.0
provider: StellaOps.Integration.GitHub.Plugin
capabilities:
 integrations:
 - type: scm
 id: github
 displayName: GitHub
 steps:
 - type: github-status
 displayName: Update GitHub Status
 gates:
 - type: github-check
 displayName: GitHub Check Required

2. Connector Runtime (Dynamic Execution)

public interface IIntegrationConnector
{
 Task<ConnectionTestResult> TestConnectionAsync(CancellationToken ct);
 Task<HealthStatus> GetHealthAsync(CancellationToken ct);
 Task<IReadOnlyList<Resource>> DiscoverResourcesAsync(string resourceType, CancellationToken ct);
}

public interface ISCMConnector : IIntegrationConnector
{
 Task<CommitInfo> GetCommitAsync(string ref, CancellationToken ct);
 Task CreateCommitStatusAsync(string commit, CommitStatus status, CancellationToken ct);
}

public interface IRegistryConnector : IIntegrationConnector
{
 Task<string> ResolveDigestAsync(string imageRef, CancellationToken ct);
 Task<bool> VerifyDigestAsync(string imageRef, string expectedDigest, CancellationToken ct);
}

3. Step Provider (Execution Contract)

public interface IStepProvider
{
 StepExecutionCharacteristics Characteristics { get; }
 Task<StepResult> ExecuteAsync(StepContext context, CancellationToken ct);
 Task<StepResult> RollbackAsync(StepContext context, CancellationToken ct);
}

public sealed record StepExecutionCharacteristics
{
 public bool IsIdempotent { get; init; }
 public bool SupportsRollback { get; init; }
 public TimeSpan DefaultTimeout { get; init; }
 public ResourceRequirements Resources { get; init; }
}

Invariants

1. Release identity is immutable — Once created, a release’s component digests cannot be changed. Create a new release instead. If an updated artifact digest is detected for a non-draft release, Release Orchestrator returns a structured successor-version suggestion and creates the next draft version through /api/v1/release-orchestrator/releases/{id}/versions/from-artifact-update.

2. Promotions are append-only — Promotion state transitions are recorded; no edits or deletions.

3. Evidence packets are sealed — Evidence is cryptographically signed and stored immutably.

4. Digest verification at deploy time — Agents verify image digests at pull time; mismatch fails deployment.

5. Separation of duties where configured — Where the target environment sets RequireSeparationOfDuties, the promotion’s requester cannot approve it on any approve route (separation_of_duties_self_approval). The release seal aggregate is not an approve route and does not apply the rule: it records an approved request whose requester and approver are the same caller (see the limit under Separation of duties on approve).

6. Gate evaluation is deterministic — The same release, environment and gate inputs produce the same ordered gate results (pinned by PluginPathGateResultsParityTests goldens).

7. Share grants are account-bound and resource-scoped — A deployment share link carries an opaque grantId, not a credential; it elevates only the authenticated subject who accepts it, only for the one deployment (or release+environment) it targets, only within its tenant, only until expiry/revoke/terminal state, and (for Production elevation) only after a second authorized person — who must differ from the issuer — approves it. Every grant-backed privileged action is audited with the real acting subject. See ADR-027.

Error Handling

• Transient failures — Retry with exponential backoff; circuit breaker for repeated failures
• Agent disconnection — An agent without a heartbeat for 2 minutes is treated as stale; task reassignment to other agents is a design target (not implemented)
• Deployment failure — Automatic rollback if configured; otherwise mark promotion as failed
• Gate failure — Block promotion; require manual intervention or re-evaluation

Observability

StellaOps.ReleaseOrchestrator.Deployments is the deployment lifecycle meter emitted by the Web API deployment endpoints. Other implemented meters: StellaOps.ReleaseOrchestrator.EvidenceSeal (releaseorchestrator.evidence_seal.{successes,failures,receipts_rejected}), StellaOps.ReleaseOrchestrator.AuditChain (releaseorchestrator.audit_chain.append_failures), StellaOps.ReleaseOrchestrator.Plugin (stellaops_step_executions_total, stellaops_gate_evaluations_total, stellaops_connector_operations_total, stellaops_step_execution_duration_ms, stellaops_gate_evaluation_duration_ms) and StellaOps.Scripts. The release_* instrument names and spans listed after the two implemented deployment instruments are design targets with no call sites.

Metrics

• release_orchestrator_deployment_lifecycle_total{tenant,operation,status} - Counter for deployment create, pause, resume, cancel, rollback, and retry target operations.
• release_orchestrator_deployment_lifecycle_duration_seconds{tenant,operation,status} - Histogram for the same deployment lifecycle operations.
• (design target) release_promotions_total — Counter by environment and outcome
• (design target) release_deployments_duration_seconds — Histogram of deployment times
• (design target) release_gate_evaluations_total — Counter by gate type and result; the implemented counterpart is stellaops_gate_evaluations_total
• (design target) release_agents_online — Gauge of online agents

Traces (proposed)

• promotion.request — Span for promotion request handling
• gate.evaluate — Span for each gate evaluation
• deployment.execute — Span for deployment execution
• agent.task — Span for agent task execution

Logs

• Structured logs with correlation IDs
• Promotion ID, release ID, environment ID in all relevant logs
• Sensitive data (secrets, credentials) masked

Security Considerations

Actor attribution

Every value the service records as the acting user is read from the claims of the authenticated principal: created_by and updated_by on approval policies, environment policy bindings and the release environment limit (create, update, upsert and disable), evidence createdBy, approval and dashboard decision actors, deployment and release-seal audit actors, the deploy engine’s user, and the actor ids of promotion and share-grant audit events. The first group records the subject (sub, then the name identifier), or system for a principal without one. Display fields such as requestedBy use the name, preferred_username or email claim and fall back to the subject.

The service does not read the X-StellaOps-Actor request header for attribution. The Router gateway removes a client-supplied value and writes the validated token sub in its place, which is the same value the signed identity envelope gives the principal, so gateway traffic is attributed as before. A caller that reaches the service directly cannot choose its recorded identity with the header. On the approval-policy, policy-binding, environment-limit, evidence and dashboard decision routes, a direct caller with a bearer token and no header is now recorded under its token subject; it was recorded as system before. The platform’s verified on-behalf-of contract is the gateway-signed X-StellaOps-OnBehalfOf-Envelope sidecar, which this service does not mount; no unsigned header substitutes for it.

Two outbound clients copy the inbound header onto their calls to other services (the integration catalog to Integrations, and the Harbor version strategy to the Platform registry lookup). They forward it and do not record it.

Script catalog tenant partition

Scripts belong to a tenant (owner ruling 2026-10-18). Every /api/v2/scripts route takes the tenant from the authenticated principal (RequireTenant(), the stellaops:tenant claim; a tenant header does not select another catalog) and the caller from the principal’s subject. The store filters every read, update and delete on the tenant AND the script id, and applies the visibility rule in the same statement:

A script of another tenant, or a private or team script of another user, answers 404 on get, versions, version content, check-compatibility, update and delete, the same as an unknown id, and is not changed; the list and its X-Total-Count include only visible scripts. Listing the versions of an unknown or invisible script answers 404 (it answered 200 with an empty list before). A principal without a subject sees only organization and public scripts and cannot create or change a script (403). A version is written only under a script of the writer’s tenant, and a new script id never takes over an existing row. The script execution path (ScriptExecutionRequest, not yet wired into a host; DPP-040) loads by the same tenant and caller.

Migration 015_release_orchestrator_scripts_tenant_partition.sql adds scripts.scripts.tenant_id. It cannot backfill: owner_id is a token subject, the subject’s tenant is recorded by Authority in a database this service cannot read, and the tenant of a subject’s other activity here is not necessarily the tenant the script was written in. A row written before 015 therefore keeps tenant_id NULL and matches no caller. The column stays nullable; the NOT VALID check ck_scripts_tenant_assigned refuses every new or changed row without a tenant, so an update of an unassigned row that does not assign one fails. To make such a row usable again, an operator assigns it to its tenant in the release-orchestrator database:

UPDATE scripts.scripts SET tenant_id = '<tenant id>' WHERE id = '<script id>' AND tenant_id IS NULL;

Once no unassigned row remains, ALTER TABLE scripts.scripts VALIDATE CONSTRAINT ck_scripts_tenant_assigned; validates the check. The tables carry no row-level security policy: as for the release-orchestrator tenant tables other than release.freeze_window_exemptions, the tenant predicate is explicit in each statement, and the capacity probe and the Doctor checks read the schema without a tenant session.

Freeze window tenant binding

Freeze windows and their exemptions belong to the tenant that created them. The /api/v1/release-orchestrator/freeze-windows routes, /freeze-window-exemptions/{exemptionId} and the per-environment freeze-window reads act within the authenticated tenant (the stellaops:tenant claim; a tenant header does not select another tenant for a non-administrator). Every statement of the freeze-window store filters on the tenant AND the id, the environment or the window:

• GET, PUT and DELETE /api/v1/release-orchestrator/freeze-windows/{id} answer 404 freeze_window_not_found for another tenant’s window, the same body as an unknown id, and the window is not changed or deleted.
• GET /api/v1/release-orchestrator/freeze-windows/{id}/exemptions answers 404 freeze_window_not_found for an unknown or another tenant’s window (it answered 200 with an empty list). Granting an exemption under such a window answers the same 404; the exemption insert itself requires a window of the writer’s tenant, so an exemption is never written under another tenant’s window.
• DELETE /api/v1/release-orchestrator/freeze-window-exemptions/{exemptionId} answers 204 when the caller’s tenant had the exemption and 404 freeze_window_exemption_not_found for an unknown or another tenant’s exemption (it answered 204 for any id).

The tenant policy on release.freeze_window_exemptions is enabled and not forced. The table is owned by the role that applies the startup migrations, which is the service’s own role, so the policy does not bind the service; the explicit predicates are the tenant boundary.

Deployment secret mode tenant binding

GET, PUT and DELETE /api/v1/release-orchestrator/admin/tenants/{tenantId}/deployment-secret-mode read and set the tenant-wide default deployment secret mode in release_orchestrator.tenant_deployment_settings. The route requires an authenticated tenant (RequireTenant()) and compares the route tenant with it, case-insensitively, before the stored mode is read or written. A different route tenant answers 403 tenant_forbidden and the other tenant’s mode is neither read nor changed. A global administrator (stellaops:admin=true) may name any tenant, which is how the console’s tenant list manages each tenant’s mode; a tenant header does not make a non-administrator act on another tenant.

Deployment cancel tenant binding

POST /api/v1/release-orchestrator/deployments/{id}/cancel reads the deployment in the caller’s tenant before it touches the authoritative real-engine job. Another tenant’s deployment answers 404 with an empty body, the same as an unknown id, and its job, its deployment projection and its promotion stay unchanged. With the real engine on, the job is then read and cancelled by the caller’s canonical tenant UUID (the tenant the engine stamped on the job), so the job store’s tenant-scoped read is a second, independent barrier. A deployment-job save never moves a job to another tenant: a job id that already belongs to another tenant refuses the save.

Agent Security

• mTLS authentication — Agents authenticate with CA-signed certificates
• Short-lived credentials — Task credentials expire after execution
• Capability-based authorization — Agents only receive tasks matching their capabilities
• Heartbeat monitoring — Detect and flag agent disconnections

Secrets Management

• Never stored in database — Only vault references stored
• Fetched at execution time — Secrets retrieved just-in-time for deployment
• Short-lived — Dynamic credentials with minimal TTL
• Masked in logs — Secret values never logged
• Compose push mode — Environment refs are resolved in orchestrator memory immediately before the immutable lock is generated; generated content and task diagnostics must not echo values.
• Compose pull mode — Network-backed refs fail closed until the edge assignment contract can deliver scoped references and resolve them without returning plaintext to the control plane.

Vault auth references

Agentless SSH and deployment secret resolution supports Vault-backed auth references in this form:

authref://vault/<logical-path>#<key>

At execution time Release Orchestrator resolves the reference through the configured secret provider (ISecretProvider, RoutingSecretProvider); authref maps to a provider of kind vault, which reads HashiCorp Vault KV v2:

<address>/v1/<mount>/data/<logical-path>      (mount defaults to "secret")

The named <key> is read from the returned data.data object. For example, authref://vault/partner-test#APP_SERVER_SSH_PRIVATE_KEY reads key APP_SERVER_SSH_PRIVATE_KEY from KV path secret/data/partner-test.

Runtime requirements:

• A vault entry under Crypto:SecretProviders:<n> (id, kind, address, auth:mode, auth:token, auth:mount). The compose estates fill it from RELEASE_ORCHESTRATOR_VAULT_ADDR, RELEASE_ORCHESTRATOR_VAULT_TOKEN and RELEASE_ORCHESTRATOR_VAULT_KV_MOUNT.
• The executor no longer reads VAULT_ADDR / VAULT_TOKEN directly. A vault://, openbao:// or authref:// reference without a matching provider, a non-success response, malformed JSON, an absent path, and an absent key fail closed before the deployment connection is attempted.
• Secret values are returned only to the in-memory executor path and must not be emitted to structured logs, evidence packets, or error messages.

Plugin Sandbox

Design target: no resource-limit or network-isolation sandbox type exists in StellaOps.ReleaseOrchestrator.Plugin. Container-host execution plugins get their sandbox from the container backend (see Execution Plugins).

• Resource limits — CPU, memory, timeout limits per plugin
• Capability restrictions — Plugins declare required capabilities
• Network isolation — Optional network restrictions for plugins

Performance Characteristics

• Agent heartbeat — 30-second interval (AgentConfiguration); an agent is treated as stale after 2 minutes without a heartbeat (AgentAwareTargetConnectionTester.HeartbeatFreshnessWindow).
• Promotion-evaluation latency, orchestration overhead and step-timeout figures are not defined or measured in src/; earlier target numbers are removed.

Implementation Roadmap

Historical phase plan; the current state is described in the sections above.

Runtime Observation (Unified — Sprint 20260512_020)

The deployment agent (StellaOps.Agent.Host) is the single host-side surface for both deployment dispatch AND runtime observation. The src/RuntimeInstrumentation/StellaOps.Agent.{Linux,Windows}/ track is NOT inactive: its Linux publisher actively evolves (Signals-only publication); it is simply no longer the deployment target — deployment dispatch was absorbed into the deployment agent so a single binary, provisioned once on .10/.20 with mTLS + cert rotation, performs both.

Contracts

The runtime observation DTOs (RuntimeEvent, RuntimeEventEnvelope, AdmissionDecision, RuntimeContractVersions) live in the neutral library src/__Libraries/StellaOps.Runtime.Contracts/ under namespace StellaOps.Runtime.Contracts. They were relocated out of StellaOps.Zastava.Core.Contracts during the Zastava unbundling. Schema STRING identifiers on the wire (zastava.runtime.event@v1, zastava.admission.decision@v1) are preserved for backward compatibility with deployed Scanner.WebService consumers. SPRINT_20260722_018 subsequently relocated the assembly from the ReleaseOrchestrator tree to this neutral home; the assembly name, namespace, DTOs and wire identifiers did not change.

Downstream surfaces

Ownership ruled (Q-7, 2026-08-25) and EXECUTED (2026-09-01, SPRINT_20260824_010 P8-16): one-owner (Signals). The agent posts the envelope once, to Signals only. The Findings.Ledger leg (endpoint, runtime_traces.release_id column + index, agent leg, and the retry-divergence machinery whose “MUST dual-write” contract modeled partial success as retryable divergence) is deleted. The measured ground for the ruling: the Findings copy had zero readers (sentinel finding_id = Guid.Empty excluded it from every finding-scoped query, frames always empty), while every real consumer (Policy VEX/EWS HasRuntimeEvidence, reachability) already reads Signals through Signals’ owner API. If Findings later needs runtime observations, it consumes a Signals event/read contract into its own projection per the DC-15 pattern.

Runtime evidence is published to one downstream store:

The Signals tables carry a nullable release_id TEXT column (Signals consolidated baseline 001_v1_signals_baseline.sql; originally sprint 020 migration 004_runtime_evidence_release_id.sql, now archived) so runtime evidence can be tied directly to a deployed release without a cross-schema join. The column is nullable: legacy/v1 emitters (which have no release context) continue to write valid rows. The ingest dedups on the agent-stamped eventId (Q-7 idempotency: the event marker is embedded in the contexts JSONB entry and the upsert’s DO UPDATE arm is containment-guarded against it).

Capability handler

runtime.observe.container is an IAgentCapability implementation registered in CapabilityRegistry (catalog id runtime.observe.container, alias runtime-observe) alongside winrm.execute, winrm.container, ssh.execute, ssh.upload, ssh.download and ssh.tunnel. On task execution it:

1. Reads docker inspect <containerId> against the host runtime.
2. Constructs a RuntimeEvidenceEnvelope populated from the inspect JSON + the task’s metadata-borne releaseId.
3. Publishes once, to Signals (/api/v1/runtime/observations), through HttpRuntimeEvidencePublisher (the body carries the envelope’s eventId; Signals deduplicates on it). The agent posts to Signals directly at Agent:RuntimeEvidence:SignalsBaseAddress; the route is not exposed at the gateway. Since PCD-865 (owner ruling 2026-10-18) the publisher authenticates with an Authority client-credentials token (signals:write aoc:verify, minted for the agent’s tenant) from Agent:RuntimeEvidence:Authority:{Url,TokenEndpoint,ClientId,Tenant} plus the secret from a mounted file (ClientSecretFile, which wins) or the environment (Agent__RuntimeEvidence__Authority__ClientSecret); Signals takes the observation’s tenant from that token and refuses a body tenant that differs. The client is a confidential Authority client registered per tenant by the operator (the Linux host agent’s runtime-facts pattern, PCD-45). The agent refuses to start while a Signals base address is configured without that client. Per-agent values belong in env/agent-core.local.env.

Phase 5 honesty boundary: the capability launched in stub mode — it emitted frames: [] and the trace was minimum-viable evidence (“container observed running, digest-gate passed”). Real eBPF stack-frame capture for reachability data landed in Sprint 20260513_022 (phases A–H all DONE; archived under docs-archive/implplan/SPRINT_20260513_022_RuntimeInstrumentation_ebpf_stack_frame_capture.md): PHASE-B extended Agent.Core contracts with UserStack/KernelStack/StackCaptureMode, PHASE-C added Tetragon kprobe + stack-trace event mapping, PHASE-D added the Linux agent symbol resolver, PHASE-E activated captureMode: ebpf-frames on the Agent.Docker capability, PHASE-F wired reachability scoring in Scanner, and PHASE-G/H closed live integration on the lab Linux VM plus operator docs

• offline-kit prep. The container-inspect mode remains available as the fallback when the Tetragon bridge is disabled.

References

• Product Vision
• Architecture Overview
• Full Orchestrator Specification (archived advisory)
• Competitive Landscape
• Sprint 20260512_020 — Runtime Observation Unification (archived)
• Zastava architecture (retired; contracts relocated)

Evidence

Paths are repo-relative; WebApi/ is src/ReleaseOrchestrator/__Apps/StellaOps.ReleaseOrchestrator.WebApi/.