checkId: check.attestation.transparency.consistency plugin: stellaops.doctor.attestor severity: fail tags: [attestation, transparency, security, air-gap]

Transparency Log Consistency

This Stella Ops Doctor check detects transparency-log rollback or tampering by comparing the locally stored Rekor checkpoint against the remote log – the primary defense for the attestation trust model. It is part of the attestation diagnostics surfaced by stella doctor; see the Doctor overview for how checks are run and reported.

What It Checks

Verifies that locally stored transparency log checkpoints are consistent with the remote Rekor log. This is a critical security check that detects log rollback or tampering.

The check only runs if a checkpoint path is configured (Attestor:Transparency:CheckpointPath or Transparency:CheckpointPath) or a checkpoint file exists at the default path ({AppData}/stellaops/transparency/checkpoint.json).

Steps performed:

  1. Read stored checkpoint – parses the local checkpoint.json file containing TreeSize, RootHash, UpdatedAt, and LogId.

    • If the file does not exist: Skip (checkpoint will be created on first verification run).
    • If the JSON is invalid: Fail with remediation to remove the corrupted file and re-sync.
    • If the file is empty/null: Fail.

  2. Fetch remote log state – HTTP GET to {rekorUrl}/api/v1/log (10-second timeout). Parses treeSize and rootHash from the response.

    • If Rekor is unreachable: Skip (cannot verify consistency without remote state).

  3. Compare tree sizes:

    • If remote tree size < stored tree size: Fail with “possible fork/rollback” (the log should only grow, never shrink). This is a CRITICAL security finding.
    • If tree sizes match but root hashes differ: Fail with “possible tampering” (same size but different content). This is a CRITICAL security finding.
    • If remote tree size >= stored tree size and hashes are consistent: Pass with entries-behind count and checkpoint age.

Evidence collected: CheckpointPath, Exists, StoredTreeSize, RemoteTreeSize, StoredRootHash, RemoteRootHash, EntriesBehind, CheckpointAge, ConsistencyVerified, Error.

Air-Gap Behavior

When the deployment is sealed by AirGap policy, or STELLAOPS_DOCTOR_OFFLINE_PROFILE=true forces Doctor offline mode, this check returns Skip with airGapSealed=true before contacting remote Rekor. Use an offline-kit Rekor mirror or signed transparency checkpoint bundle to validate consistency without egress. Do not treat the skip as a pass for compliance evidence unless the offline bundle has been imported and verified separately.

Why It Matters

The transparency log is the tamper-evident backbone of the attestation system. If an attacker modifies or rolls back the log, they could hide revoked signatures, alter attestation records, or forge provenance data. This check is the primary defense against such attacks. A root hash mismatch at the same tree size is one of the strongest indicators of log tampering and should trigger an immediate security investigation.

Common Causes

For log rollback (remote < stored):

For root hash mismatch:

For corrupted checkpoint file:

How to Fix

Docker Compose

# Check stored checkpoint
docker compose -f devops/compose/docker-compose.stella-ops.yml exec attestor \
  cat /app/data/transparency/checkpoint.json | jq .

# Verify you are connecting to the correct Rekor instance
docker compose -f devops/compose/docker-compose.stella-ops.yml exec attestor \
  curl -s https://rekor.sigstore.dev/api/v1/log | jq .

# If corrupted checkpoint, remove and re-sync
docker compose -f devops/compose/docker-compose.stella-ops.yml exec attestor \
  rm /app/data/transparency/checkpoint.json

docker compose -f devops/compose/docker-compose.stella-ops.yml exec attestor \
  stella attestor transparency sync

Bare Metal / systemd

For corrupted checkpoint:

# Back up the corrupted checkpoint first
cp /path/to/checkpoint.json /path/to/checkpoint.json.bak

# Remove corrupted checkpoint
rm /path/to/checkpoint.json

# Trigger re-sync
stella attestor transparency sync

For log rollback or hash mismatch (CRITICAL):

# CRITICAL: This may indicate a security incident. Do not dismiss without investigation.

# Get current root hash from Rekor
curl -s https://rekor.sigstore.dev/api/v1/log | jq .rootHash

# Compare with stored checkpoint
stella attestor transparency checkpoint show

# Verify you are connecting to the correct Rekor instance
curl -s https://rekor.sigstore.dev/api/v1/log | jq .

# Check stored checkpoint
cat /path/to/checkpoint.json | jq .

# If using wrong log instance, reset checkpoint (DESTRUCTIVE -- only after confirming wrong instance)
rm /path/to/checkpoint.json
stella attestor transparency sync

# If mismatch persists with correct log, escalate to security team

Kubernetes / Helm

# Check stored checkpoint
kubectl exec -it deploy/stellaops-attestor -n stellaops -- \
  cat /app/data/transparency/checkpoint.json | jq .

# Verify Rekor connectivity
kubectl exec -it deploy/stellaops-attestor -n stellaops -- \
  curl -s https://rekor.sigstore.dev/api/v1/log | jq .

# If corrupted, remove checkpoint and re-sync
kubectl exec -it deploy/stellaops-attestor -n stellaops -- \
  rm /app/data/transparency/checkpoint.json

kubectl exec -it deploy/stellaops-attestor -n stellaops -- \
  stella attestor transparency sync

# Check checkpoint persistence (PVC)
kubectl get pvc -l app.kubernetes.io/component=attestor -n stellaops

# Set checkpoint path in Helm values
# attestor:
#   transparency:
#     checkpointPath: "/app/data/transparency/checkpoint.json"

Root hash mismatches or log rollbacks should be treated as potential security incidents. Do not reset the checkpoint without first investigating whether the remote log was actually compromised.

Verification

stella doctor run --check check.attestation.transparency.consistency