Region-aware, per-plugin signing & OCI release (operator guide)

Audience: operators who sign, release, deploy, and rotate Stella Ops runtime-plugin bundles, and platform engineers wiring a host to verify them. Authority: the binding contract is ADR-030. The signing-root lifecycle is ADR-028, reused verbatim. The OCI registry primitives are ADR-024. Sprint: docs/implplan/SPRINT_20260610_004_Platform_region_aware_plugin_signing_release.md. Source of truth: src/ always wins over this doc. The load-bearing files are listed in the ADR References section; verify command flags against src/Cli/StellaOps.Cli/Commands/CommandFactory.cs and the deploy payload against src/ReleaseOrchestrator/__Agents/StellaOps.Agent.DeployNative/DeployPluginPullPayload.cs.

1. What changed, and why

Before ADR-030, plugin signing was hardwired: every runtime bundle (SignedRuntimePluginManifest, runtime-bundle.v1) carried a single detached <assembly>.dll.sig verified against one hardcoded RSA cosign.pub, and every plugin shipped baked into a service image. There was no algorithm/keyId in the manifest, no region awareness, and no way to release a plugin independently.

Three operator requirements drove the change:

1. Region-aware signing. Plugins are signed with the active regional cryptography (world/fips/gost/sm/eidas/kcmvp), routed through the same ICryptoProviderRegistry.ResolveSigner path every other signing surface uses.
2. Per-plugin release. Each plugin is signed and released individually — no whole-service rebuild to ship one plugin, and a safe rotation story.
3. OCI release artifacts. Signed bundles live as digest-pinned OCI artifacts in the internal Zot registry (registry.stella-ops.local:5000), not baked into images. Deploy pulls by digest and verifies the region signature before extraction.

The signature envelope is runtime-bundle.v2: the manifest gains a multi-signature array signatures: [{ algorithm, keyId, provider, profile, signaturePath }]. A legacy v1 bundle normalizes to a synthetic single-entry v2 (RS256 / legacy-cosign), so existing cosign / offline-dev-RSA bundles keep verifying unchanged.

2. The per-region signing model

2.1 Algorithm per profile

The canonical signing algorithm token is fixed per compliance profile (PluginSigningProfileAlgorithms, in src/__Libraries/StellaOps.Cryptography/PluginSigningCapabilities.cs):

Each signed bundle advertises a crypto:signing:<algo> capability (PluginSigningCapabilities.ForAlgorithm) alongside its functional capabilities. The verifier maps the manifest token to the crypto-registry verification token before resolving the provider — the only divergence is GOST (GOST3410-2012-256 in the manifest → GOST12-256 in the registry).

2.2 Key custody — sample/dev keys vs production HSM/manager

This is the most important operational distinction:

• Dev / world / default profile uses sample keys through DefaultCryptoProvider. These keys are not secrets to protect — they exist so a developer or air-gap demo can sign and verify with no HSM. Never deploy a production region on a sample key.
• Production regional profiles (fips/gost/sm/eidas/kcmvp) hold the private signing key in the regional manager — an HSM, a PKCS#11 token, a remote signing service, or CryptoPro for GOST. The CLI never touches private key material: ICryptoProviderRegistry.ResolveSigner(Signing, algo, keyRef) hands the bytes to the manager, which signs inside the boundary and returns the detached signature. Only the public key (PEM) is recorded as the signing-root material.
• The signing-root set held by the platform (ADR-028 VerificationRootRegistry) is public material only: public keys and fingerprints. Secrets are never placed in env_settings; symmetric secrets (e.g. VexHub HMAC trust roots) are AEAD-sealed in the encrypted credential store, never in the signing-root set.

The SM_SOFT_ALLOWED caveat (SM2 only)

SM2 signing with the soft provider is gated behind the SM_SOFT_ALLOWED=1 environment variable; verification is ungated (a deployed host always verifies SM2 signatures). This is a guard against accidentally producing production SM signatures with the soft (non-HSM) key:

• stella plugin sign --profile sm (and release --profile sm) errors clearly when SM_SOFT_ALLOWED is unset, on the signing host.
• A production SM region signs through the SM HSM/PKCS#11 bundle — SM_SOFT_ALLOWED is irrelevant there because the soft path is not used.
• Deploy-time and runtime SM2 verification never requires the flag.

3. Per-plugin sign / release flow (CLI)

All three subcommands live under stella plugin (src/Cli/StellaOps.Cli/Commands/CommandFactory.cs, BuildPluginCommand). Handlers: src/Cli/StellaOps.Cli/Commands/Plugin/PluginSignReleaseCommandHandlers.cs.

3.1 stella plugin sign — region-sign a bundle in place

Resolves a signer for the active profile via ICryptoProviderRegistry, writes the detached <assembly>.sig, and emits a runtime-bundle.v2 manifest.json binding (algorithm, keyId, provider, profile).

stella plugin sign <bundle-dir> \
  --profile sm \
  --assembly MyPlugin.dll \
  --key-id sm-key-1 \
  [--id <plugin-id>] [--module <module>] \
  [--capability crypto:signing:SM2] [--entry-type <fqtn>] \
  [--provider <hint>] [--algorithm <override>]

• <bundle-dir> (positional) — the bundle directory; the plugin id defaults to its folder name.
• --assembly (required) — the bundle-relative assembly path that is signed.
• --key-id (required) — the signing-root key id resolved through the crypto registry.
• --profile / -p — the active compliance profile (defaults to the resolved CryptoComplianceService.ActiveProfile).

For --profile sm, set SM_SOFT_ALLOWED=1 on the signing host (dev/soft only) or the command fails with a clear error.

3.2 stella plugin release — sign → publish OCI artifact → register the root → emit digest

SM_SOFT_ALLOWED=1 stella plugin release <bundle-dir> \
  --profile sm \
  --assembly MyPlugin.dll \
  --key-id sm-key-1 \
  --registry registry.stella-ops.local:5000/stellaops/plugins \
  [--tag <tag>] [--public-material <PEM> | --public-material-file <path>] \
  [--grace-deadline <ISO-8601>] \
  [--report-progress --plugins-total <N> --plugins-done <n>] \
  [--no-register] [--force-push] [--allow-insecure-registry]

release chains:

1. Sign (as plugin sign) → produces the v2 manifest + detached sig.
2. Publish the bundle as a digest-pinned OCI artifact via PluginBundleOciPublisher → OciArtifactPusher.PushAsync. The bundle directory is packed into one deterministic tar layer (sorted entries, fixed mtimes) so re-releasing an unchanged bundle yields a byte-identical layer and a stable digest (SkipIfTagExists makes re-release a no-op). Artifact type: application/vnd.stellaops.plugin.bundle.v1; layer media type: application/vnd.stellaops.plugin.bundle.tar.v1 (src/Scanner/__Libraries/StellaOps.Scanner.Storage.Oci/OciMediaTypes.cs).
3. Register / rotate the region signing-root via the ADR-028 verification-keys endpoint (skip with --no-register). When registering introduces a new root over an existing one, the existing root moves to retiring (dual-trust grace).
4. Post re-envelope progress (--report-progress --plugins-total N --plugins-done n) so a multi-plugin rotation shows n/N re-signed.
5. Emit the artifact digest — the value pinned into ReleaseManifest.Plugins[].

The artifact is tagged by its manifest digest unless --tag is given. --public-material[-file] records the non-secret public key (PEM) as the signing-root material when registering.

3.3 stella plugin signing-root — drive the signing-root lifecycle

Reuses the ADR-028 lifecycle (same IPlatformVerificationKeyClient + consequences primitive as crypto verification-key), pinned to the signing-root material type:

stella plugin signing-root introduce  --profile sm --key-id sm-key-1 \
   --public-material-file sm-key-1.pub \
   [--grace-deadline <ISO-8601>] \
   --confirm --i-understand-old-material-will-be-invalidated

stella plugin signing-root invalidate --profile sm --key-id sm-key-1 \
   [--retain-archived] \
   --confirm --i-understand-old-material-will-be-invalidated

stella plugin signing-root status     --profile sm

• introduce adds a new root and moves any current root to retiring (dual-trust). Both verify during the grace window.
• invalidate revokes a root; --retain-archived (default true) keeps the old public key as an archived (verify-only) root for immutable/historical material. Both introduce and invalidate require the typed consequences acknowledgment (--confirm and --i-understand-old-material-will-be-invalidated); there is no silent root change from the CLI.
• status shows the active/retiring/archived roots and re-envelope progress for the profile.

4. Verification at the trust boundary (fail-closed)

RegionAwarePluginSignatureVerifier (src/__Libraries/StellaOps.Plugin/Security/RegionAwarePluginSignatureVerifier.cs) resolves, per signature entry, a verifier + public key by (algorithm, keyId) against the active profile’s signing-root set (active + retiring + archived) supplied by an IPluginSigningRootSource, and verifies the detached signature through the crypto registry’s ephemeral verifier (managed crypto, no native cosign binary).

Acceptance rule:

• Admitted when any entry validates against an active or retiring root (dual-trust grace).
• An archived root is verify-only — it does not admit fresh mounted material on its own.
• A revoked root is never served by the source; an entry matching only revoked/archived roots fails closed.
• The synthetic legacy-cosign entry verifies against the configured file trust root (cosign.pub) via OfflineDevRsaSha256PluginVerifier — the legacy path, so v1 bundles keep admitting.

Fail-closed is preserved end-to-end: an empty/unreachable signing-root set, no-matching-keyId, an unsupported algorithm, or a signature mismatch all reject. AllowUnsigned is never enabled in production.

IPluginSigningRootSource has two implementations:

• PlatformBootPullPluginSigningRootSource— pulls the ADR-028 signing-root JWKS-set endpoint at boot (the same internal verification-token channel EvidenceLocker/Attestor use). This is the in-cluster path.
• LocalDirectoryPluginSigningRootSource— reads a local trust-root dir (<root>/<profile>/<keyId>.pub) for the CLI / air-gap / offline path.

Centralized verifier selection (no drift)

Every host that builds a mounted-plugin verifier from configuration (Authority, Excititor WebService/Worker, Concelier, Scheduler, Notify) routes through one shared factory — PluginSignatureVerifierSelector.CreateVerifier(...) (src/__Libraries/StellaOps.Plugin/Security/PluginSignatureVerifierSelector.cs) — which owns the none | cosign | offline-dev-rsa-sha256 | region-aware branch. Selecting region-aware requires a RegionAwarePluginVerifierContext (crypto registry + signing-root source + active profile); an absent context for that provider is a fail-fast configuration error, never a silent downgrade.

5. The OCI-artifact deploy mechanism (and why it beats image-baking)

5.1 Why not bake plugins into a service image

• Per-plugin independence. Releasing one plugin no longer forces a whole service-image rebuild — you sign and push one bundle.
• A real per-plugin signature gate at the trust boundary. Deploy verifies the region signature before extraction; a tampered or wrong-region bundle never reaches the plugin volume.
• Air-gap friendliness. The bundle is content-addressed by digest in the internal Zot registry; no Rekor/Fulcio, no external network. Incremental release is a digest pull, not a multi-GB image.

5.2 The deploy path: deploy.plugin-pull

DeployPluginPullExecutionPlugin (capability deploy.plugin-pull, src/ReleaseOrchestrator/__Agents/StellaOps.Agent.DeployNative/DeployPluginPullExecutionPlugin.cs) takes one DeployPluginPullPayload and:

1. Fetch by digest — reference is registry/repository@sha256:... (digest-pinned; a tag is never trusted at deploy).
2. Verify the layer digest against the manifest.
3. Verify the region signature against the trust root for profile, fail-closed, reusing RegionAwarePluginSignatureVerifier.
4. Unpack to staging, then sha256-idempotent atomic swap into <destinationRoot>/<pluginId> (keeping <destination>.previous for rollback when backupPreviousVersion is true — the default).
5. Optionally restart the containers in restartContainers.

The release-orchestrator deploy-plan builder emits deploy.plugin-pull steps from ReleaseManifest.Plugins[] (PluginComponent{PluginId, Profile, Registry, Repository, Digest, Algorithm, KeyId, Order}). The legacy deploy.artifact-extract path (image-embedded plugins) is unchanged for backward compatibility.

Payload (DeployPluginPullPayload):

6. Regional crypto bundles (the VexHub-unblocking dependency)

ADR-030 Phase 2 ships the base-profile regional crypto bundles the platform mounts at /app/plugins/crypto/base:

• com.stellaops.crypto.sm, com.stellaops.crypto.fips, com.stellaops.crypto.gost (dir name == manifest id). The FIPS bundle was previously missing and is now shipped.
• Each emits a runtime-bundle.v2 manifest advertising both credential-aead:<algo> (for the credential-store seal path) and crypto:signing:*, with explicit entryTypes so RegionalCryptoMountedPluginLoader’s single-entry-type discovery stays unambiguous.
• Produced by Publish-RegionalCryptoBaseBundles in devops/build/package-runtime-plugins.ps1; signed with the offline-dev RSA signer and recorded as a legacy-cosign/RS256 v2 entry so the existing cosign.pub trust root verifies them with no verifier swap on the AEAD seal path.

This is why ADR-030 unblocks the VexHub D1 trust-root seal: before these bundles existed, the regional crypto loader threw DirectoryNotFoundException and no AEAD provider could be resolved. With the bundles mounted, POST /platform/verification-keys/world/introduce (materialType VexHmacTrustRoot) seals the secret via the regional crypto catalog (AES-256-GCM) and indexes the fingerprint only in env_settings.

7. End-to-end rotation flow (introduce → re-sign → invalidate)

1. Introduce a new signing root: stella plugin signing-root introduce --profile sm --key-id sm-key-2 ... — sm-key-1 moves to retiring. Both verify (dual-trust).
2. Re-sign per plugin under the new key: stella plugin release <bundle> --profile sm --key-id sm-key-2 --report-progress --plugins-total N --plugins-done n. Plugins not yet re-signed still admit under the retiring sm-key-1.
3. Dual-trust holds for the grace window — both re-signed (sm-key-2) and not-yet-re-signed (sm-key-1) plugins admit; signing-root status shows n/N.
4. Invalidate the old root once re-enveloping is complete (or you accept the residual — warn-and-allow, ADR-028 §Resolved decision 4): stella plugin signing-root invalidate --profile sm --key-id sm-key-1 --confirm --i-understand-old-material-will-be-invalidated. From here, a plugin still signed only under sm-key-1 fails admission cleanly (fail-closed); a plugin re-signed under sm-key-2 passes.

8. Cross-references

• ADR-030 — the binding contract (v2 envelope, fail-closed verification, OCI-over-image-bake, ADR-028 reuse).
• ADR-028 — the dual-trust introduce → re-envelope → consequences-gated invalidate lifecycle reused for the signing root.
• ADR-024 — internal Zot registry + OciArtifactPusher.
• Cryptography & compliance defaults — first-run crypto setup and the active-profile model.
• Regional crypto route matrix — which algorithm each profile routes to.
• Sprint: docs/implplan/SPRINT_20260610_004_Platform_region_aware_plugin_signing_release.md.