Metrics Specification

Prometheus-compatible metrics exposed by the Stella Ops Release Orchestrator.

Audience: SREs and platform operators building dashboards, SLOs, and alerts for the Release Orchestrator. Status: Planned — this is the reference metrics catalog; treat the names here as the canonical contract. Related: Alerting · Logging · Tracing · Operations Overview

Overview

The Release Orchestrator exposes Prometheus-compatible metrics for monitoring deployment health, performance, and operational status. Metrics are grouped by domain (releases, promotions, deployments, agents, workflows, targets, integrations, gates) plus cross-cutting API and evidence series. All metric names are prefixed stella_. The same metrics back the alert rules in Alerting and the SLIs defined below.

Core Metrics

Release Metrics

MetricTypeDescriptionLabels
stella_releases_totalcounterTotal releases createdtenant, status
stella_releases_activegaugeCurrently active releasestenant, status
stella_release_components_counthistogramComponents per releasetenant

Promotion Metrics

MetricTypeDescriptionLabels
stella_promotions_totalcounterTotal promotionstenant, env, status
stella_promotions_in_progressgaugePromotions currently in progresstenant, env
stella_promotion_duration_secondshistogramTime from request to completiontenant, env, status
stella_approval_pending_countgaugePending approvalstenant, env
stella_approval_duration_secondshistogramTime to approvetenant, env

Deployment Metrics

MetricTypeDescriptionLabels
stella_deployments_totalcounterTotal deploymentstenant, env, strategy, status
stella_deployment_duration_secondshistogramDeployment durationtenant, env, strategy
stella_deployment_tasks_totalcounterTotal deployment taskstenant, status
stella_deployment_task_duration_secondshistogramTask durationtarget_type
stella_rollbacks_totalcounterTotal rollbackstenant, env, reason

Agent Metrics

MetricTypeDescriptionLabels
stella_agents_connectedgaugeConnected agentstenant
stella_agents_by_statusgaugeAgents by statustenant, status
stella_agent_tasks_totalcounterTasks executed by agentsagent, type, status
stella_agent_task_duration_secondshistogramAgent task durationagent, type
stella_agent_heartbeat_age_secondsgaugeSeconds since last heartbeatagent
stella_agent_resource_cpu_percentgaugeAgent CPU usageagent
stella_agent_resource_memory_percentgaugeAgent memory usageagent

Workflow Metrics

MetricTypeDescriptionLabels
stella_workflow_runs_totalcounterWorkflow executionstenant, template, status
stella_workflow_runs_activegaugeCurrently running workflowstenant, template
stella_workflow_duration_secondshistogramWorkflow durationtemplate, status
stella_workflow_step_duration_secondshistogramStep execution timestep_type, status
stella_workflow_step_retries_totalcounterStep retry countstep_type

Target Metrics

MetricTypeDescriptionLabels
stella_targets_totalgaugeTotal targetstenant, env, type
stella_targets_by_healthgaugeTargets by health statustenant, env, health
stella_target_drift_detectedgaugeTargets with drifttenant, env

Integration Metrics

MetricTypeDescriptionLabels
stella_integrations_totalgaugeConfigured integrationstenant, type
stella_integration_healthgaugeIntegration health (1=healthy)tenant, integration
stella_integration_requests_totalcounterRequests to integrationsintegration, operation, status
stella_integration_latency_secondshistogramIntegration request latencyintegration, operation

Gate Metrics

MetricTypeDescriptionLabels
stella_gate_evaluations_totalcounterGate evaluationstenant, gate_type, result
stella_gate_evaluation_duration_secondshistogramGate evaluation timegate_type
stella_gate_blocks_totalcounterBlocked promotions by gatetenant, gate_type, env

API Metrics

MetricTypeDescriptionLabels
stella_http_requests_totalcounterHTTP requestsmethod, path, status
stella_http_request_duration_secondshistogramRequest latencymethod, path
stella_http_requests_in_flightgaugeActive requestsmethod
stella_http_request_size_byteshistogramRequest sizemethod, path
stella_http_response_size_byteshistogramResponse sizemethod, path

Evidence Metrics

MetricTypeDescriptionLabels
stella_evidence_packets_totalcounterEvidence packets generatedtenant, type
stella_evidence_packet_size_byteshistogramEvidence packet sizetype
stella_evidence_verification_totalcounterEvidence verificationsresult

Prometheus Configuration

# prometheus.yml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'stella-orchestrator'
    static_configs:
      - targets: ['stella-orchestrator:9090']
    metrics_path: /metrics
    scheme: https
    tls_config:
      ca_file: /etc/prometheus/ca.crt

  - job_name: 'stella-agents'
    kubernetes_sd_configs:
      - role: pod
        selectors:
          - role: pod
            label: "app.kubernetes.io/name=stella-agent"
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_label_agent_id]
        target_label: agent_id

Histogram Buckets

Duration Buckets (seconds)

# Short operations (API calls, gate evaluations)
short_duration_buckets: [0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]

# Medium operations (workflow steps)
medium_duration_buckets: [0.1, 0.5, 1, 2.5, 5, 10, 30, 60, 120, 300]

# Long operations (deployments)
long_duration_buckets: [1, 5, 10, 30, 60, 120, 300, 600, 1200, 3600]

Size Buckets (bytes)

# Request/response sizes
size_buckets: [100, 1000, 10000, 100000, 1000000, 10000000]

# Evidence packet sizes
evidence_buckets: [1000, 10000, 100000, 500000, 1000000, 5000000]

SLI Definitions

Availability SLI

# API availability (99.9% target)
sum(rate(stella_http_requests_total{status!~"5.."}[5m]))
/
sum(rate(stella_http_requests_total[5m]))

Latency SLI

# API latency P99 < 500ms
histogram_quantile(0.99,
  sum(rate(stella_http_request_duration_seconds_bucket[5m])) by (le)
)

Deployment Success SLI

# Deployment success rate (99% target)
sum(rate(stella_deployments_total{status="succeeded"}[24h]))
/
sum(rate(stella_deployments_total[24h]))

Alert Rules

groups:
  - name: stella-orchestrator
    rules:
      - alert: HighDeploymentFailureRate
        expr: |
          sum(rate(stella_deployments_total{status="failed"}[1h]))
          /
          sum(rate(stella_deployments_total[1h])) > 0.1
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: High deployment failure rate
          description: More than 10% of deployments failing in the last hour

      - alert: AgentOffline
        expr: stella_agent_heartbeat_age_seconds > 120
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: Agent {{ $labels.agent }} offline
          description: Agent has not sent heartbeat for > 2 minutes

      - alert: PendingApprovalsStale
        expr: |
          stella_approval_pending_count > 0
          and
          time() - stella_promotion_request_timestamp > 3600
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: Stale pending approvals
          description: Approvals pending for more than 1 hour

      - alert: IntegrationUnhealthy
        expr: stella_integration_health == 0
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: Integration {{ $labels.integration }} unhealthy
          description: Integration health check failing

      - alert: HighAPILatency
        expr: |
          histogram_quantile(0.99,
            sum(rate(stella_http_request_duration_seconds_bucket[5m])) by (le, path)
          ) > 1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: High API latency on {{ $labels.path }}
          description: P99 latency exceeds 1 second

Grafana Dashboards

Main Dashboard Panels

  1. Deployment Pipeline Overview

    • Promotions per environment (time series)
    • Success/failure rates (gauge)
    • Active deployments (stat)
  2. Agent Health

    • Connected agents (stat)
    • Agent status distribution (pie chart)
    • Heartbeat age (table)
  3. Gate Performance

    • Gate evaluation counts (bar chart)
    • Block rate by gate type (time series)
    • Evaluation latency (heatmap)
  4. API Performance

    • Request rate (time series)
    • Error rate (time series)
    • Latency distribution (heatmap)

References