checkId: check.evidencelocker.index plugin: stellaops.doctor.evidencelocker severity: warn tags: [evidence, index, consistency]

Evidence Index Consistency

This stella doctor check confirms that the Evidence Locker’s lookup index matches the artifacts actually stored on disk, so audit queries, compliance exports, and release-verification lookups return complete and accurate results.

What It Checks

Verifies that the evidence index is consistent with the artifacts stored on disk. The check operates on the local evidence locker path (EvidenceLocker:Path) and performs:

  1. Index existence: looks for index.json or an index/ directory at the locker root.
  2. Artifact counting: counts .json files across five artifact directories: attestations/, sboms/, vex/, verdicts/, provenance/.
  3. Cross-reference validation: for each entry in index.json, verifies the referenced artifact file exists on disk. Records any artifacts that are indexed but missing from disk.
  4. Drift detection: compares the total indexed count against the total disk artifact count. Flags a warning if drift exceeds 10% of total artifacts.
ConditionResult
Evidence locker path not configured or missingSkip
Index file and index directory both missingWarn
Artifacts indexed but missing from diskFail
Index count drifts > 10% from disk countWarn
Index consistent with disk artifactsPass

Evidence collected: IndexedCount, DiskArtifactCount, MissingFromDisk, MissingSamples, Drift, per-directory counts (attestationsCount, sbomsCount, vexCount, verdictsCount, provenanceCount).

The check only runs when EvidenceLocker:Path is configured and the directory exists.

Why It Matters

The evidence index provides fast lookup for attestations, SBOMs, VEX documents, and provenance records. An inconsistent index means queries may return stale references to deleted artifacts (causing retrieval errors) or miss artifacts that exist on disk (causing incomplete audit reports). Index drift accumulates over time and degrades the reliability of evidence searches, compliance exports, and release verification lookups.

Common Causes

How to Fix

Docker Compose

# Check index status
docker compose exec evidence-locker ls -la /data/evidence/index.json

# Rebuild evidence index
docker compose exec evidence-locker stella evidence index rebuild

# Fix orphaned index entries
docker compose exec evidence-locker stella evidence index rebuild --fix-orphans

# Verify evidence integrity after rebuild
docker compose exec evidence-locker stella evidence verify --all

# Refresh index (less aggressive than rebuild)
docker compose exec evidence-locker stella evidence index refresh

# Check disk health
docker compose exec evidence-locker df -h /data/evidence

Bare Metal / systemd

# Check index file
ls -la /var/lib/stellaops/evidence/index.json

# Rebuild evidence index
stella evidence index rebuild

# Fix orphaned entries
stella evidence index rebuild --fix-orphans

# Refresh index
stella evidence index refresh

# Check for disk errors
sudo fsck -n /dev/sda1

# Verify evidence integrity
stella evidence verify --all

sudo systemctl restart stellaops-evidence-locker

Kubernetes / Helm

# Check index in pod
kubectl exec deploy/stellaops-evidence-locker -- ls -la /data/evidence/index.json

# Rebuild index
kubectl exec deploy/stellaops-evidence-locker -- stella evidence index rebuild --fix-orphans

# Verify evidence
kubectl exec deploy/stellaops-evidence-locker -- stella evidence verify --all

# Check persistent volume health
kubectl describe pvc stellaops-evidence-data

# values.yaml - enable background indexer
evidenceLocker:
  indexer:
    enabled: true
    intervalMinutes: 15
    repairOnDrift: true

Verification

stella doctor run --check check.evidencelocker.index