checkId: check.evidencelocker.retrieval plugin: stellaops.doctor.evidencelocker severity: fail tags: [evidence, attestation, retrieval, core]

Attestation Retrieval

This stella doctor check confirms that attestation artifacts can be fetched from the Evidence Locker within acceptable latency, so operators know that release approvals, audit queries, and compliance exports will not stall on slow or failing retrieval.

What It Checks

Verifies that attestation artifacts can be retrieved from the evidence locker. The check supports two modes depending on the deployment:

HTTP mode (when IHttpClientFactory is available): Sends a GET request to {endpoint}/v1/attestations/sample with a 5-second timeout and measures response latency.

Local file mode (fallback): Checks the local evidence locker path at EvidenceLocker:Path, verifies the attestations/ subdirectory exists, and attempts to read a sample attestation JSON file.

ConditionResult
Endpoint not configuredSkip
HTTP request times out (> 5000ms)Fail
HTTP error status codeFail
Connection errorFail
HTTP success but latency > 500msWarn
Local attestations directory missingWarn
HTTP success with latency <= 500msPass
Local file read successfulPass

Evidence collected: Endpoint, StatusCode, LatencyMs, Threshold, Path, SampleAttestation, ContentLength.

The check only runs when EvidenceLocker:Endpoint or Services:EvidenceLocker is configured.

Why It Matters

Attestation retrieval is a core operation used throughout the release pipeline. Release approvals, audit queries, compliance reports, and evidence exports all depend on being able to retrieve attestation artifacts from the evidence locker. If retrieval is slow or failing, release approvals may time out, audit queries will fail, and compliance reports cannot be generated. Latency above 500ms indicates performance degradation that will compound when retrieving multiple attestations during a release or audit.

Common Causes

How to Fix

Docker Compose

# Check evidence locker service status
docker compose ps evidence-locker

# Test evidence retrieval
docker compose exec evidence-locker stella evidence status

# Test authentication
docker compose exec evidence-locker stella evidence auth-test

# Check service logs for errors
docker compose logs evidence-locker --since 5m

# If local mode, verify the evidence path and permissions
docker compose exec evidence-locker ls -la /data/evidence/attestations/

# Initialize evidence locker if needed
docker compose exec evidence-locker stella evidence init

# Set endpoint configuration
# EvidenceLocker__Endpoint=http://evidence-locker:5080

Bare Metal / systemd

# Check service status
sudo systemctl status stellaops-evidence-locker

# Test evidence retrieval
stella evidence status

# Test connectivity
stella evidence ping

# Check attestations directory
ls -la /var/lib/stellaops/evidence/attestations/

# Initialize if empty
stella evidence init

# Check disk I/O
iostat -x 1 5

# In appsettings.json:
# "EvidenceLocker": { "Endpoint": "http://localhost:5080" }

sudo systemctl restart stellaops-evidence-locker

Kubernetes / Helm

# Check evidence locker pod status
kubectl get pods -l app=stellaops-evidence-locker

# Check pod logs
kubectl logs deploy/stellaops-evidence-locker --since=5m

# Test retrieval from within cluster
kubectl exec deploy/stellaops-evidence-locker -- stella evidence status

# Check persistent volume
kubectl describe pvc stellaops-evidence-data

# Check for resource constraints
kubectl top pod -l app=stellaops-evidence-locker

# values.yaml
evidenceLocker:
  endpoint: http://stellaops-evidence-locker:5080
  resources:
    requests:
      memory: 256Mi
      cpu: 100m
    limits:
      memory: 512Mi
      cpu: 500m

Verification

stella doctor run --check check.evidencelocker.retrieval