StellaOps Release Process

This document describes the release process for StellaOps suite and module releases.

It has two layers:

1. The release-governance model (this top section) — who decides to cut a release, when, and on what evidence. This is the authoritative answer to “how do we do releases” and establishes that a release is an out-of-sprint operational act, not a sprint deliverable.
2. The release mechanics (everything from “Overview” downward) — the CI/CD pipelines, artifact set, evidence pack, rollback, and hotfix procedures the release owner executes once the go decision is made.

Release governance — releases are out-of-sprint

Releases are NOT sprints

A release is an operational act performed by the release owner, not a unit of engineering work tracked in a sprint. This is a binding rule:

• Sprints deliver engineering, QA, and docs. A sprint’s job is to land a feature, fix a defect, harden a service, write tests, or update documentation — and to converge its Delivery Tracker to all-DONE with evidence. See CLAUDE.md §2.1–§2.2 for sprint discipline.
• A sprint is never “do the release.” No sprint task may be “cut release vX.Y”, “obtain release sign-off”, or “ship to prod”. A sprint that depends on a release happening would be un-archivable by construction, because the release is not under the sprint’s control — it is a downstream decision the release owner makes after consuming the sprint’s outputs.
• Capstone / consolidation sprints produce inputs to the release decision, not the decision itself. A capstone sprint (e.g. an operator-trust catalog consolidation) delivers the consolidated rollup, an advisory readiness assessment, and the operator handoff package. Those are sprint deliverables. The actual go/no-go is taken outside the sprint and does not gate the sprint’s archival.

Rationale: tying the release to a sprint creates a false dependency. The release owner must be free to hold, defer, or proceed based on the current state of the evidence and business context — not be forced to “complete” a release to close a sprint, and not have a sprint blocked indefinitely waiting on a business decision.

Release-readiness inputs (these INFORM the decision)

The release owner consults the following before deciding. None of them is the release; they are the evidence base:

The decision-gate is an ADVISORY readiness assessment

The capstone consolidation produces a decision-gate verdict — RECOMMEND-RELEASE / CONDITIONAL / HOLD (sometimes phrased PASS / CONDITIONAL / HOLD). This is an advisory readiness assessment authored by QA, evaluated against published thresholds (Tier-1 PASS-rate, overall PASS-rate, auth/integrity floor, no-regression, etc.).

Critically:

• The assessment is an input to the release decision, NOT the release itself and NOT a sprint blocker. A HOLD verdict is a delivered output of the capstone — the capstone’s job was to produce an honest assessment, and it did.
• It does not block the capstone sprint’s archival. Once the assessment is produced (whatever its verdict), that sprint task is DONE.
• QA recommends; the release owner decides. A HOLD is a recommendation to hold, not a veto. The release owner (PM/Product) may proceed with documented risk acceptance, or hold, on the basis of the assessment plus business context.

Who decides, and when

• Release owner: PM / Product makes the go/no-go. (Security Maintainer co-signs the tag per docs/GOVERNANCE.md §4 once go is taken.)
• Cadence: on-demand, not sprint-bound. There is no “release sprint” on the calendar. The decision is triggered by business need or a readiness milestone — e.g. the capstone assessment reaching RECOMMEND-RELEASE, a security hotfix (out-of-band per GOVERNANCE.md §4), or a scheduled channel cut (edge/stable/lts — see Release Channels below). The release owner picks the moment; the engineering sprints simply make the suite releasable.
• The go decision is recorded by the release owner (e.g. in the release notes and/or the evidence pack), not as a sprint task transition.

Carried gaps — accepted/deferred at release time, out-of-sprint

Known gaps in the gap registry are not blockers that a sprint must close before the release. They are dispositioned by the release owner at release time:

• Each carried-forward gap (e.g. GAP-T6-KMS-RESTORE, GAP-T7-DISK-QUOTA) is listed in the capstone FINAL_REPORT.md §“Carried-forward gaps” and registered in README_CROSS_TIER_ROLLUP.md.
• The release owner explicitly accepts (ship with a documented known-issue) or defers (route to a future engineering sprint) each gap. This acceptance is a release-time act, recorded in the release notes’ known-issues section.
• Gap acceptance is out-of-sprint: the consolidation sprint’s job is to report the gaps honestly (carrying PARTIAL/HOLD states as their true current value); it is not the consolidation’s job to make every gap PASS before it can archive.

How the decision turns into a release

Once the release owner takes the go decision, the release is executed using the mechanics below (no sprint required):

1. Versioning — pick the version + channel + codename per VERSIONING.md and codenames.md.
2. All-services rebuild with gitSha baking — rebuild every service so the baked gitSha lands in /buildinfo.json across the fleet (closes the GAP-T1-BUILDINFO-ROLLOUT class). Use the suite build pipeline (.gitea/workflows/release-suite.yml) / build-all-services.sh with --no-cache when shared libs changed.
3. Artifacts / SBOM / attestation — the suite pipeline emits container images, CLI binaries, the release manifest, and checksums; the evidence pipeline (.gitea/workflows/release-evidence-pack.yml) emits SBOMs, SLSA provenance, cosign signatures, and Rekor proofs. See Suite Release Process and Release Evidence Pack below.
4. Release notes — author docs/releases/release-notes-<topic>.md (or vX.Y.Z-release-notes.md) following the existing convention; record the go decision, the accepted/deferred carried gaps, and the readiness assessment reference.
5. Air-gap kit assembly — package the Offline Kit and validate per the Release Checklist (offline-first; no Helm/Kubernetes artifacts).
6. Tag + sign — co-signed by a Security Maintainer per GOVERNANCE.md §4.

The remainder of this document is the mechanics reference for those steps. It is intentionally not duplicated above — the governance section tells you whether and when to release; the sections below tell you how.

Overview

StellaOps uses automated CI/CD pipelines for releases:

Deployment Packaging Scope

Stella Ops does not support Kubernetes or Helm deployments. Release packaging must not publish Helm charts, Kubernetes manifests, kubectl rollout steps, or Helm rollback procedures. Supported release packaging targets Docker Compose, Offline Kit, signed release manifests, and the release evidence pack.

Historical note: release digest tooling remains useful for non-Kubernetes packaging because it validates digest-pinned release-manifest authority from local CI artifacts. It is not Helm support.

TODO(release-digests/schema/model/provider): update this release guide whenever the required image digest inventory, release/evidence schema, default local AI model, or crypto/evidence provider pack changes.

Module Release Process

Prerequisites

• [ ] All tests passing on main branch
• [ ] CHANGELOG.md updated with changes
• [ ] Version bumped in module’s version.txt (if applicable)
• [ ] Breaking changes documented

Steps

Option A: Tag-based Release

# Create and push tag
git tag module-authority-v1.2.3
git push origin module-authority-v1.2.3

The pipeline will automatically:

1. Parse module name and version from tag
2. Build the module
3. Publish NuGet package to Gitea registry
4. Build and push container image (if applicable)

Option B: Manual Dispatch

1. Navigate to Actions > Module Publish
2. Click Run workflow
3. Select:
   • Module: e.g., Authority
   • Version: e.g., 1.2.3
   • Publish NuGet: true
   • Publish Container: true
4. Click Run

Artifacts Published

Suite Release Process

Prerequisites

• [ ] All module versions finalized
• [ ] Integration tests passing
• [ ] Security scan completed
• [ ] CHANGELOG.md updated
• [ ] Compatibility matrix documented
• [ ] Codename selected (see codenames.md)

Pre-Release Checklist

- [ ] All P1 issues resolved
- [ ] Performance benchmarks meet SLOs
- [ ] Documentation updated
- [ ] Migration guide prepared
- [ ] Release notes drafted
- [ ] Release notes state that Git tags, release manifests, artifacts, checksums, and evidence packs are signed before publication
- [ ] Plugin sandbox posture changes are called out, including that Trusted plugins now run out-of-process through `OutOfProcessOverlay` and do not use AppArmor/SELinux/Job Object confinement by default
- [ ] RuntimeInstrumentation release notes state that the module ships as Linux/Windows host agents, not Kubernetes charts, Helm charts, DaemonSets, or Kubernetes API watchers
- [ ] RuntimeInstrumentation optional sensor pins and local AI/model pins are listed as operator-replaceable release-note items when they change
- [ ] QTSP/QSCD provider-pack details are recorded in the release notes when the release claims eIDAS qualified signing/sealing evidence
- [ ] Security advisory review complete
- [ ] Air-gap bundle tested
- [ ] Compose and Offline Kit packaging validated
- [ ] No Kubernetes or Helm packaging artifact is present in the release

Steps

Option A: Tag-based Release

# Create and push tag
git tag suite-2026.04-nova
git push origin suite-2026.04-nova

Option B: Manual Dispatch

1. Navigate to Actions > Suite Release
2. Click Run workflow
3. Fill in:
   • Version: e.g., 2026.04
   • Codename: e.g., Nova
   • Channel: edge, stable, or lts
   • Skip tests: false (default)
   • Dry run: false for actual release
4. Click Run

Pipeline Stages

validate → test-gate → build-modules → build-containers
                    ↘              ↓
                     build-cli → package-bundles → release-manifest → create-release → summary

1. Validate - Check version format, resolve inputs
2. Test Gate - Run unit, architecture, and contract tests
3. Build Modules - Build all 9 modules (matrix)
4. Build Containers - Push container images (9 modules)
5. Build CLI - Build for 5 platforms
6. Package Bundles - Package Compose/Offline Kit assets and evidence inputs
7. Release Manifest - Generate suite-{version}.yaml
8. Create Release - Create Gitea release with artifacts
9. Summary - Generate summary report

Artifacts Published

Release Channels

Edge

• Pre-release builds
• May contain experimental features
• Not recommended for production
• Triggered by: channel: edge or tag without -stable/-lts

Stable

• Production-ready releases
• Thoroughly tested
• 9 months of support (feature releases)
• Triggered by: channel: stable

LTS (Long Term Support)

• April releases only (XX.04)
• 5 years of security updates
• 3 years of standard support
• Triggered by: channel: lts

Rollback Procedures

Compose Rollback

# Pull previous version if needed
docker pull git.stella-ops.org/stella-ops.org/authority:2025.10

# Apply the previous release Compose profile
docker compose --env-file devops/compose/env/prod.env -f devops/compose/docker-compose.prod.yaml up -d

Kubernetes and Helm rollback commands are intentionally absent because Stella Ops does not support those deployment targets.

Database Rollback

1. Stop all services
2. Restore database from backup
3. Deploy previous version
4. Verify data integrity

Important: Always test rollback procedures in staging before production.

Hotfix Process

For critical security fixes:

1. Create hotfix branch from release tag

   git checkout -b hotfix/2026.04.1 suite-2026.04
   

2. Apply fix and test

3. Tag hotfix release

   git tag suite-2026.04.1
   git push origin suite-2026.04.1
   

4. Cherry-pick fix to main branch

Release Evidence Pack

Every release includes a Release Evidence Pack for customer verification and compliance.

Evidence Pack Contents

Generation Workflow

The evidence pack is generated by .gitea/workflows/release-evidence-pack.yml:

1. Verify Test Gates - Ensures all test workflows passed
2. Generate Checksums - Create SHA256SUMS and SHA512SUMS
3. Sign Artifacts - Sign with cosign (keyless or key-based)
4. Generate SBOMs - Create CycloneDX SBOMs per artifact
5. Generate Provenance - Create SLSA v1.0 statements
6. Collect Rekor Proofs - Fetch inclusion proofs from Rekor
7. Build Pack - Assemble final evidence pack bundle
8. Self-Verify - Run verify.sh to validate the pack

Release signing is mandatory for publication. QTSP/QSCD-backed signing is a separate compliance provider-pack decision: use it only when the release makes an eIDAS qualified signature/seal or BaselineT/LT/LTA evidence claim, and record the selected provider-pack details in the release notes.

Manual Trigger

# Trigger evidence pack generation for a release
gh workflow run release-evidence-pack.yml \
  -f version=2.5.0 \
  -f release_tag=v2.5.0

Verification

Customers can verify releases offline:

tar -xzf stella-release-2.5.0-evidence-pack.tgz
cd stella-release-2.5.0-evidence-pack
./verify.sh --verbose

See Release Evidence Pack for detailed documentation.

Reproducible Builds

All release builds are reproducible using SOURCE_DATE_EPOCH:

# Set from git commit timestamp
export SOURCE_DATE_EPOCH=$(git show -s --format=%ct HEAD)

# Build with deterministic settings
dotnet build -c Release /p:Deterministic=true /p:ContinuousIntegrationBuild=true

The CI verifies reproducibility by building twice and comparing checksums.

See Reproducible Builds for details.

Post-Release Tasks

• [ ] Verify artifacts in registry
• [ ] Generate and publish Release Evidence Pack
• [ ] Verify evidence pack passes self-verification
• [ ] Update documentation site
• [ ] Send release announcement
• [ ] Update compatibility matrix
• [ ] Monitor for issues (24-48 hours)
• [ ] Update roadmap

Troubleshooting

Build Failures

1. Check test results in artifacts
2. Review workflow logs
3. Verify secrets are configured (GITEA_TOKEN)

Push Failures

1. Verify registry authentication
2. Check network connectivity
3. Ensure no conflicting tags exist

Common Issues

Related Documentation

• Versioning Strategy
• Codename Registry
• CI/CD Workflows