Agent Operations Quick Start

This guide is the operator reference for deploying, configuring, and maintaining Stella Ops agents at scale: bootstrap, configuration and drift, local health diagnostics, certificate management, updates, and fleet monitoring. For the current target-host procedure, start with the canonical Agent Host Bootstrap playbook.

Implementation status (verify against src/ — code wins over this doc). The shipping CLI has a real, wired AgentCommandGroup: stella agent list, stella agent show, stella agent token create, and stella agent token list call /api/v1/release-orchestrator/agents.... The former sample-backed stella agent bootstrap command was removed and must not be used.
The separate config, local-doctor, certificate, and agent-update handlers described later in this guide remain draft/unwired or simulated; see their per-section notes.
What is actually wired and real today:
stella agent list/show/token create/token list— Release Orchestrator deployment-agent observation and one-time registration-token operations.
stella self update— DSSE-verified offline product update flow (UpdateCommands.CreateSelfCommand, registered at CommandFactory.cs:96).
Server-side fleet health — the Doctor agent plugin (StellaOps.Doctor.Plugin.Agent, plugin id stellaops.doctor.agent) and the StellaOps.Agent.Core AgentDoctor health checks. See Fleet Monitoring and Health Check Categories.
Treat only the four commands named above as current runtime commands. Code in src/Cli/StellaOps.Cli/Commands/AgentCommandGroup.cs remains the source of truth.

Zero-Touch Bootstrap

Bootstrap uses the real Release Orchestrator registration-token and registration endpoints. The supported scripts create and immediately consume the one-time token, write the issued mTLS material to ignored local paths, and can start the compose agent. For central-agent and per-host commands, including OCI save/scp/load/run, use the Agent Host Bootstrap playbook.

Generate Bootstrap Token

# Low-level token command for an external installer or manual API registration.
# The plaintext token is returned once; list responses mask it.
stella agent token create \
  --name prod-agent-01 \
  --capability docker \
  --capability compose

# Preferred repository-provided bootstrap: mint + consume a token, persist the
# issued mTLS bundle locally, but leave process launch to the operator.
cd devops/compose
./scripts/agent-bootstrap.sh --no-compose-up --strict-orchestrator-certificate

Custom Capabilities

stella agent token create \
  --name prod-agent-01 \
  --display-name "Production agent 01" \
  --capability docker \
  --capability compose \
  --valid-for-hours 1

Configuration Management

Draft / not wired; mock output. agent config and agent apply exist as factory methods (ConfigCommands) but are not registered in CommandFactory.cs. The handlers print a mock configuration and a simulated drift report / dry-run diff; nothing is read from or written to a live agent. Note the live handler emits connection.jobengineUrl (not orchestratorUrl as shown in the sample YAML below).

View Current Configuration

# Show current config in YAML format
stella agent config

# Show as JSON
stella agent config --format json

Detect Configuration Drift

# Check for drift between current and desired state
stella agent config --diff

Apply New Configuration

# agent-config.yaml
identity:
  agentId: agent-abc123
  agentName: prod-agent-01
  environment: production

connection:
  # Live handler reads/emits `jobengineUrl`; agents connect to the JobEngine.
  jobengineUrl: https://jobengine.example.com
  heartbeatInterval: 30s

capabilities:
  docker: true
  scripts: true
  compose: true

resources:
  maxConcurrentTasks: 10
  workDirectory: /var/lib/stella-agent

security:
  certificate:
    source: AutoProvision

# Validate without applying
stella agent apply -f agent-config.yaml --dry-run

# Apply configuration
stella agent apply -f agent-config.yaml

Agent Health Diagnostics (Doctor)

Draft / not wired; mock output. agent doctor exists as a factory method (DoctorCommands) but is not registered in CommandFactory.cs, and the handler renders a fixed set of simulated diagnostic rows and simulated fixes. The underlying check engine is real, however: StellaOps.Agent.Core defines the AgentDoctor and the IAgentHealthCheck implementations behind the five categories listed under Health Check Categories. Flags below (--category/-c, --fix/-f, --format, --agent-id/-a) match the factory definition.

Run Local Diagnostics

# Run all health checks
stella agent doctor

# Filter by category
stella agent doctor --category security
stella agent doctor --category network
stella agent doctor --category runtime
stella agent doctor --category resources
stella agent doctor --category configuration

Apply Automated Fixes

# Run diagnostics and apply fixes
stella agent doctor --fix

Output Formats

# Table output (default)
stella agent doctor

# JSON output for scripting
stella agent doctor --format json

# YAML output
stella agent doctor --format yaml

Certificate Management

Draft / not wired; mock output. agent cert-status and agent renew-cert exist as factory methods (CertificateCommands) but are not registered in CommandFactory.cs. Both handlers print simulated certificate details (fixed thumbprint, 45-day validity) and do not contact a CA.

Check Certificate Status

stella agent cert-status

Renew Certificate

# Renew if nearing expiry
stella agent renew-cert

# Force renewal
stella agent renew-cert --force

Agent Updates

What is actually wired: the update logic lives in UpdateCommands, but only the self group is registered in the CLI (UpdateCommands.CreateSelfCommand at CommandFactory.cs:96). Run it as stella self update. The agent update / agent rollback factory methods exist but are not registered (stella agent update does not resolve today). Substitute stella self update in the examples below.
self update is a real, offline DSSE-verified flow: it locates a local product update manifest (product-update-manifest.dsse.json in the Offline Update Kit, or via --manifest), verifies its signature against a trust root (--trust-root / STELLAOPS_PRODUCT_UPDATE_TRUST_ROOT), enforces CRA lifecycle (end-of-support) policy, and prints resolved image digests and fixed CVEs before applying. The package download/apply steps themselves are still stubbed (simulated drain/apply) in the current handler.
Maintenance-window gate is not live. The handler hardcodes inMaintenanceWindow = true (UpdateCommands.cs:214), so the “Sat-Sun 02:00-06:00 UTC” / --force check below is currently dead code — updates never block on the window, and --force is effectively a no-op against it. The window message is wired but unreachable. --force is still accepted as a flag.

Check for Updates

stella self update --check

Apply Updates

# Update to latest (resolves available version from the manifest, else 1.3.0)
stella self update

# Update to a specific version
stella self update --version 1.3.0

# Accepted flag intended to force updates outside the Sat-Sun 02:00-06:00 UTC
# maintenance window. NOTE: the window check is currently dead code
# (UpdateCommands.cs:214 hardcodes inMaintenanceWindow=true), so this has no
# effect against the window today.
stella self update --force

# Offline: verify a local DSSE manifest and print the plan without writing binaries
stella self update --manifest ./offline-update-kit/product-update-manifest.dsse.json \
  --trust-root ./offline-update-kit/product-update-trust-root.pem --dry-run

# Allow a manifest whose support train is past end-of-support
stella self update --manifest <path> --trust-root <path> --allow-eol

Rollback

# Rollback to previous version
# (factory method `agent rollback` is defined but NOT registered in the CLI today)
stella agent rollback

Health Check Categories

The five categories below are the real HealthCheckCategory enum values in StellaOps.Agent.Core (Doctor/IAgentHealthCheck.cs: Security, Network, Runtime, Resources, Configuration). The Checks column lists the actual IAgentHealthCheck implementations in Doctor/Checks/AgentHealthChecks.cs — there are exactly five check classes in this assembly today:

Category	Checks (class in `AgentHealthChecks.cs`)
Security	Certificate expiry (`CertificateExpiryCheck`)
Network	(no check implemented in this assembly today — the `Network` enum value exists but ships unused locally)
Runtime	Docker daemon connectivity (`DockerConnectivityCheck`)
Resources	Disk space (`DiskSpaceCheck`), memory usage (`MemoryUsageCheck`)
Configuration	Configuration drift (`ConfigurationDriftCheck`)

Reconciliation note. A prior version of this table listed certificate validity, DNS resolution, task-queue depth, and CPU usage. Those checks do not exist as IAgentHealthCheck implementations in StellaOps.Agent.Core. Several of them do exist on the server-side Doctor agent plugin — e.g. AgentCertificateValidityCheck, AgentResourceUtilizationCheck, TaskQueueBacklogCheck (see Fleet Monitoring) — which is a separate fleet-monitoring surface, not the local agent’s self-diagnostics.

Troubleshooting

Common Issues

Certificate Expired

stella agent renew-cert --force

Docker Not Accessible

# Check Docker socket
ls -la /var/run/docker.sock

# Add agent to docker group
sudo usermod -aG docker stella-agent
sudo systemctl restart stella-agent

Disk Space Low

# Clean up Docker resources
docker system prune -af --volumes

# Check agent work directory
du -sh /var/lib/stella-agent

Connection Issues

# Agents connect to the JobEngine (connection.jobengineUrl). Check DNS:
nslookup jobengine.example.com

# Check port
telnet jobengine.example.com 443

# Check firewall
sudo iptables -L -n | grep 443

Fleet Monitoring

The server-side Doctor agent plugin (StellaOps.Doctor.Plugin.Agent, plugin id stellaops.doctor.agent, category Infrastructure) monitors the agent fleet. Each of the checks below maps to a real check class under StellaOps.Doctor.Plugin.Agent/Checks/, tagged agent:

Heartbeat Freshness (AgentHeartbeatFreshnessCheck): Alerts on stale heartbeats
Certificate Expiry (AgentCertificateExpiryCheck): Warns before fleet certificates expire
Version Consistency (AgentVersionConsistencyCheck): Detects version skew across agents
Capacity (AgentCapacityCheck, TaskQueueBacklogCheck): Monitors task queue and agent load
Failed Task Rate (FailedTaskRateCheck): Alerts on high failure rates

Additional checks in this plugin include AgentCertificateValidityCheck, AgentResourceUtilizationCheck, AgentClusterHealthCheck, AgentClusterQuorumCheck, and StaleAgentCheck.

Run via the Doctor CLI. There is no --plugin option; the doctor run command filters by --check, --category (-c), or --tag (-t).

Reconciliation note (code wins). cli-routes.json declares an intent to deprecate doctor run in favour of admin doctor run (Sprint 014, “Doctor consolidated under admin”, removeIn: 3.0). That consolidation is not wired today. The doctor command group is registered directly at the root (CommandFactory.cs:212, root.Add(DoctorCommandGroup.BuildDoctorCommand(...))), and AdminCommandGroup registers no doctor subcommand. The deprecation router only attaches an alias when the canonical (new) path already exists in the command tree (TryRegisterDeprecatedAlias → commandLookup.TryGetValue), so the doctor run → admin doctor run route is silently skipped. Net runtime reality: stella doctor run is the live, canonical command (no deprecation warning) and stella admin doctor run does not resolve. Use stella doctor run. (Some in-code remediation hints still print the not-yet-wired admin/run forms; prefer the command below.)

# Run all agent-fleet checks (they share the `agent` tag)
stella doctor run --tag agent

# Run a single check by id. `--check` takes the check ID, e.g.:
stella doctor run --check check.agent.heartbeat.freshness