Agent-Based Deployment

Agent-based deployment using Docker and Compose agents for executing tasks on targets.

Status: Planned (not yet implemented) Source: Architecture Advisory Section 10.3 Related Modules: Agents Module, Deploy Orchestrator Sprints: 108_002 Docker Agent, 108_003 Compose Agent

Overview

Agent-based deployment uses lightweight agents installed on target hosts to execute deployment tasks. Agents communicate with the orchestrator over mTLS and receive tasks through heartbeat polling or WebSocket streams.

Agent Task Protocol

Task Payload Structure

// Task assignment (Core -> Agent)
interface AgentTask {
  id: UUID;
  type: TaskType;
  targetId: UUID;
  payload: TaskPayload;
  credentials: EncryptedCredentials;
  timeout: number;
  priority: TaskPriority;
  idempotencyKey: string;
  assignedAt: DateTime;
  expiresAt: DateTime;
}

type TaskType =
  | "deploy"
  | "rollback"
  | "health-check"
  | "inspect"
  | "execute-command"
  | "upload-files"
  | "write-sticker"
  | "read-sticker";

interface DeployTaskPayload {
  image: string;
  digest: string;
  config: DeployConfig;
  artifacts: ArtifactReference[];
  previousDigest?: string;
  hooks: {
    preDeploy?: HookConfig;
    postDeploy?: HookConfig;
  };
}

Task Result Structure

// Task result (Agent -> Core)
interface TaskResult {
  taskId: UUID;
  success: boolean;
  startedAt: DateTime;
  completedAt: DateTime;

  // Success details
  outputs?: Record<string, any>;
  artifacts?: ArtifactReference[];

  // Failure details
  error?: string;
  errorType?: string;
  retriable?: boolean;

  // Logs
  logs: string;

  // Metrics
  metrics: {
    pullDurationMs?: number;
    deployDurationMs?: number;
    healthCheckDurationMs?: number;
  };
}

Per-target command output on the deployment record

Sprint 20260613_001. Source of truth: SshExecuteTask.cs, SshOutputRedactor.cs, DeploymentCompatibilityModels.cs, DeploymentCompatibilityStateFactory.cs.

Command-running agent capabilities (SSH today) capture the remote command’s stdout, stderr, and exit code into AgentTaskResult.Outputs. That output now flows all the way to the deployment record and is returned by GET /api/v1/release-orchestrator/deployments/{id} on each target, for both success and failure:

// DeploymentDto.targets[]
{
  "id": "…", "name": "vm11", "type": "SshHost", "status": "completed",
  "commandOutput": {
    "exitCode": 0,
    "stdout": "Pulling core:dev\nContainer 9ab created\nStarted OK",
    "stderr": "",
    "truncated": false        // true when the head/tail bound elided output
  },
  "noOp": false,              // RO-SSHLOG-004 — see below
  "warning": null
}

Redaction guarantee (RO-SSHLOG-003). Output is redacted at the point of capture in the agent, before it is logged, persisted, or returned — so an unredacted secret never crosses a persistence or wire boundary. SshOutputRedactor masks (a) the task’s own known credential/env secret values (the agent injects export <k>='<v>' env pairs and plain:<secret> credential refs and knows those values) and (b) secret-shaped tokens as a backstop (plain:…, Authorization: Bearer …, password=…, GitLab/GitHub PATs). Normal output is preserved verbatim — no over-redaction. Because the redacted output rides inside TaskResult.ResultData, it round-trips unchanged through the existing AgentTaskResultSignature (the signature already covers ResultData); no signature change was required.

Bounding. Output is truncated to a configurable head + tail with a visible …[N chars elided]… marker (Agent:Ssh:MaxOutputHeadBytes default 8192, Agent:Ssh:MaxOutputTailBytes default 4096) so a chatty command cannot flood agent logs or bloat the persisted deployment row. The agent-side log body can be toggled with Agent:Ssh:LogCommandOutput (default true); the byte-count + exit code summary is always logged.

Hollow-success / no-op visibility (RO-SSHLOG-004). A target that completed without executing any command / producing any output (the “green-but-did-nothing” hazard) is flagged noOp: true with an explanatory warning. Per the back-compat decision the target stays completed/succeeded(no new terminal state that existing callers do not handle); the noOp flag + warning make the no-op explicit instead of silent.

Docker Agent Implementation

The Docker agent deploys single containers to Docker hosts with digest verification.

Docker Agent Capabilities

• Pull images with digest verification
• Create and start containers
• Stop and remove containers
• Health check monitoring
• Version sticker management
• Rollback to previous container

Deploy Task Flow

class DockerAgent implements TargetExecutor {
  private docker: Docker;

  async deploy(task: DeployTaskPayload): Promise<DeployResult> {
    const { image, digest, config, previousDigest } = task;
    const containerName = config.containerName;

    // 1. Pull image and verify digest
    this.log(`Pulling image ${image}@${digest}`);
    await this.docker.pull(image, { digest });

    const pulledDigest = await this.getImageDigest(image);
    if (pulledDigest !== digest) {
      throw new DigestMismatchError(
        `Expected digest ${digest}, got ${pulledDigest}. Possible tampering detected.`
      );
    }

    // 2. Run pre-deploy hook
    if (task.hooks?.preDeploy) {
      await this.runHook(task.hooks.preDeploy, "pre-deploy");
    }

    // 3. Stop and rename existing container
    const existingContainer = await this.findContainer(containerName);
    if (existingContainer) {
      this.log(`Stopping existing container ${containerName}`);
      await existingContainer.stop({ t: 10 });
      await existingContainer.rename(`${containerName}-previous-${Date.now()}`);
    }

    // 4. Create new container
    this.log(`Creating container ${containerName} from ${image}@${digest}`);
    const container = await this.docker.createContainer({
      name: containerName,
      Image: `${image}@${digest}`,  // Always use digest, not tag
      Env: this.buildEnvVars(config.environment),
      HostConfig: {
        PortBindings: this.buildPortBindings(config.ports),
        Binds: this.buildBindMounts(config.volumes),
        RestartPolicy: { Name: config.restartPolicy || "unless-stopped" },
        Memory: config.memoryLimit,
        CpuQuota: config.cpuLimit,
      },
      Labels: {
        "stella.release.id": config.releaseId,
        "stella.release.name": config.releaseName,
        "stella.digest": digest,
        "stella.deployed.at": new Date().toISOString(),
      },
    });

    // 5. Start container
    this.log(`Starting container ${containerName}`);
    await container.start();

    // 6. Wait for container to be healthy (if health check configured)
    if (config.healthCheck) {
      this.log(`Waiting for container health check`);
      const healthy = await this.waitForHealthy(container, config.healthCheck.timeout);
      if (!healthy) {
        // Rollback to previous container
        await this.rollbackContainer(containerName, existingContainer);
        throw new HealthCheckFailedError(`Container ${containerName} failed health check`);
      }
    }

    // 7. Run post-deploy hook
    if (task.hooks?.postDeploy) {
      await this.runHook(task.hooks.postDeploy, "post-deploy");
    }

    // 8. Cleanup previous container
    if (existingContainer && config.cleanupPrevious !== false) {
      this.log(`Removing previous container`);
      await existingContainer.remove({ force: true });
    }

    return {
      success: true,
      containerId: container.id,
      previousDigest: previousDigest,
      logs: this.getLogs(),
      durationMs: this.getDuration(),
    };
  }
}

Rollback Implementation

async rollback(task: RollbackTaskPayload): Promise<DeployResult> {
  const { containerName, targetDigest } = task;

  // Find previous container or use specified digest
  if (targetDigest) {
    // Deploy specific digest
    return this.deploy({
      ...task,
      digest: targetDigest,
    });
  }

  // Find and restore previous container
  const previousContainer = await this.findContainer(`${containerName}-previous-*`);
  if (!previousContainer) {
    throw new RollbackError(`No previous container found for ${containerName}`);
  }

  // Stop current, rename, start previous
  const currentContainer = await this.findContainer(containerName);
  if (currentContainer) {
    await currentContainer.stop({ t: 10 });
    await currentContainer.rename(`${containerName}-failed-${Date.now()}`);
  }

  await previousContainer.rename(containerName);
  await previousContainer.start();

  return {
    success: true,
    containerId: previousContainer.id,
    logs: this.getLogs(),
    durationMs: this.getDuration(),
  };
}

Version Sticker Management

async writeSticker(sticker: VersionSticker): Promise<void> {
  const stickerPath = this.config.stickerPath || "/var/stella/version.json";
  const stickerContent = JSON.stringify(sticker, null, 2);

  // Write to host filesystem or container volume
  if (this.config.stickerLocation === "volume") {
    // Write to shared volume
    await this.docker.run("alpine", [
      "sh", "-c",
      `echo '${stickerContent}' > ${stickerPath}`
    ], {
      HostConfig: {
        Binds: [`${this.config.stickerVolume}:/var/stella`]
      }
    });
  } else {
    // Write directly to host
    fs.writeFileSync(stickerPath, stickerContent);
  }
}

Compose Agent Implementation

The Compose agent deploys multi-container applications defined in Docker Compose files.

Compose Agent Capabilities

• Pull images for all services
• Verify digests for all services
• Deploy using compose lock files
• Health check all services
• Rollback to previous deployment
• Version sticker management

Deploy Task Flow

class ComposeAgent implements TargetExecutor {
  async deploy(task: DeployTaskPayload): Promise<DeployResult> {
    const { artifacts, config } = task;
    const deployDir = config.deploymentDirectory;

    // 1. Write compose lock file
    const composeLock = artifacts.find(a => a.type === "compose_lock");
    const composeContent = await this.fetchArtifact(composeLock);

    const composePath = path.join(deployDir, "compose.stella.lock.yml");
    await fs.writeFile(composePath, composeContent);

    // 2. Write any additional config files
    for (const artifact of artifacts.filter(a => a.type === "config")) {
      const content = await this.fetchArtifact(artifact);
      await fs.writeFile(path.join(deployDir, artifact.name), content);
    }

    // 3. Run pre-deploy hook
    if (task.hooks?.preDeploy) {
      await this.runHook(task.hooks.preDeploy, deployDir);
    }

    // 4. Pull images
    this.log("Pulling images...");
    const pullResult = await this.runCompose(deployDir, ["pull"]);
    if (!pullResult.success) {
      throw new Error(`Failed to pull images: ${pullResult.stderr}`);
    }

    // 5. Verify digests
    await this.verifyDigests(composePath, config.expectedDigests);

    // 6. Deploy
    this.log("Deploying services...");
    const upResult = await this.runCompose(deployDir, [
      "up", "-d",
      "--remove-orphans",
      "--force-recreate"
    ]);

    if (!upResult.success) {
      throw new Error(`Failed to deploy: ${upResult.stderr}`);
    }

    // 7. Wait for services to be healthy
    if (config.healthCheck) {
      this.log("Waiting for services to be healthy...");
      const healthy = await this.waitForServicesHealthy(
        deployDir,
        config.healthCheck.timeout
      );

      if (!healthy) {
        // Rollback
        await this.rollbackToBackup(deployDir);
        throw new HealthCheckFailedError("Services failed health check");
      }
    }

    // 8. Run post-deploy hook
    if (task.hooks?.postDeploy) {
      await this.runHook(task.hooks.postDeploy, deployDir);
    }

    // 9. Write version sticker
    await this.writeSticker(config.sticker, deployDir);

    return {
      success: true,
      logs: this.getLogs(),
      durationMs: this.getDuration(),
    };
  }
}

Digest Verification

private async verifyDigests(
  composePath: string,
  expectedDigests: Record<string, string>
): Promise<void> {
  const composeContent = yaml.parse(await fs.readFile(composePath, "utf-8"));

  for (const [service, expectedDigest] of Object.entries(expectedDigests)) {
    const serviceConfig = composeContent.services[service];
    if (!serviceConfig) {
      throw new Error(`Service ${service} not found in compose file`);
    }

    const image = serviceConfig.image;
    if (!image.includes("@sha256:")) {
      throw new Error(`Service ${service} image not pinned to digest: ${image}`);
    }

    const actualDigest = image.split("@")[1];
    if (actualDigest !== expectedDigest) {
      throw new DigestMismatchError(
        `Service ${service}: expected ${expectedDigest}, got ${actualDigest}`
      );
    }
  }
}

Security Considerations

1. Digest Verification: All deployments verify image digests before execution
2. Credential Encryption: Credentials are encrypted in transit and at rest
3. mTLS Communication: All agent-server communication uses mutual TLS
4. Hook Sandboxing: Pre/post-deploy hooks run in isolated environments
5. Audit Logging: All deployment actions are logged with actor context

See Also

• Agents Module
• Agent Security
• Deployment Orchestrator
• Agentless Deployment