Rekor Verification Technical Design

Document ID: DOCS-ATTEST-REKOR-001 Version: 2.0 Last Updated: 2026-01-13 Status: Draft

Current implementation note (2026-07-17): the PostgreSQL queue schema, adapter, and hosted retry worker exist, but AttestorSubmissionService does not enqueue failed submissions and the queue/worker integration tests remain behind the undefined STELLAOPS_EXPERIMENTAL_REKOR_QUEUE symbol. Treat the queue flow diagrams and retry-on-failure error table below as target design, not live submission behavior, until enqueue plus restart recovery are behaviorally proved.


1. OVERVIEW

This document provides the comprehensive technical design for Rekor transparency log verification in StellaOps. It covers four key capabilities:

  1. Merkle Proof Verification - Cryptographic verification of inclusion proofs
  2. Durable Retry Queue - Reliable submission with failure recovery
  3. Time Skew Validation - Replay protection via timestamp validation
  4. Tile-Based Verification (v2) - Support for Rekor v2 Sunlight format
SprintPriorityDescription
SPRINT_3000_0001_0001P0Merkle Proof Verification
SPRINT_3000_0001_0002P1Rekor Retry Queue & Metrics
SPRINT_3000_0001_0003P2Time Skew Validation
SPRINT_3000_0001_0004P1Rekor v2 Tile-Based Verification

2. ARCHITECTURE CONTEXT

Current State

┌─────────────────────────────────────────────────────────────────────┐
│                        Attestor Module                               │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ┌─────────────────────┐    ┌─────────────────────┐                 │
│  │ AttestorSubmission  │───►│   IRekorClient      │                 │
│  │ Service             │    │   (HttpRekorClient) │                 │
│  └─────────────────────┘    └──────────┬──────────┘                 │
│           │                            │                             │
│           ▼                            ▼                             │
│  ┌─────────────────────┐    ┌─────────────────────┐                 │
│  │ IAttestorEntry      │    │   Rekor API         │                 │
│  │ Repository          │    │   (External)        │                 │
│  └─────────────────────┘    └─────────────────────┘                 │
│                                                                      │
│  Current Limitations:                                                │
│  ✗ Stores proofs but doesn't verify them cryptographically          │
│  ✗ Failed submissions are lost (no retry)                           │
│  ✗ No integrated_time validation                                    │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

Target State

┌─────────────────────────────────────────────────────────────────────┐
│                        Attestor Module (Enhanced)                    │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ┌─────────────────────┐    ┌─────────────────────┐                 │
│  │ AttestorSubmission  │───►│   IRekorClient      │                 │
│  │ Service             │    │   + VerifyInclusion │◄──┐             │
│  └─────────┬───────────┘    └──────────┬──────────┘   │             │
│            │                           │               │             │
│            │ (on failure)              │               │             │
│            ▼                           ▼               │             │
│  ┌─────────────────────┐    ┌─────────────────────┐   │             │
│  │ IRekorSubmission    │    │ MerkleProofVerifier │   │             │
│  │ Queue (PostgreSQL)  │    │ CheckpointVerifier  │   │             │
│  └─────────┬───────────┘    └─────────────────────┘   │             │
│            │                                           │             │
│            ▼                                           │             │
│  ┌─────────────────────┐    ┌─────────────────────┐   │             │
│  │ RekorRetryWorker    │───►│   Rekor API         │───┘             │
│  │ (Background)        │    │   (External)        │                 │
│  └─────────────────────┘    └─────────────────────┘                 │
│                                                                      │
│  ┌─────────────────────┐    ┌─────────────────────┐                 │
│  │ AttestorVerification│───►│ ITimeSkewValidator  │                 │
│  │ Service             │    │ (integrated_time)   │                 │
│  └─────────────────────┘    └─────────────────────┘                 │
│                                                                      │
│  Enhancements:                                                       │
│  ✓ Cryptographic Merkle proof verification                          │
│  ✓ Durable retry queue with exponential backoff                     │
│  ✓ Time skew detection and alerting                                 │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

3. COMPONENT DESIGN

3.1 Merkle Proof Verification

3.1.1 Algorithm

Rekor uses RFC 6962 (Certificate Transparency) Merkle tree structure:

                    Root Hash
                   /         \
                  /           \
            Hash(0,1)       Hash(2,3)
            /     \         /     \
         H(0)   H(1)     H(2)   H(3)
          │       │        │       │
        Leaf0   Leaf1    Leaf2   Leaf3

Leaf Hash Computation:

leafHash = SHA256(0x00 || RFC6962_Entry)

Interior Node Computation:

interiorHash = SHA256(0x01 || leftChild || rightChild)

Inclusion Proof Verification:

Given:

def verify_inclusion(leaf_hash, leaf_index, tree_size, proof_path, expected_root):
    current_hash = leaf_hash
    current_index = leaf_index
    remaining_size = tree_size

    for sibling_hash in proof_path:
        if current_index % 2 == 1:
            # Current node is right child
            current_hash = sha256(0x01 || sibling_hash || current_hash)
        else:
            # Current node is left child
            current_hash = sha256(0x01 || current_hash || sibling_hash)

        current_index = current_index // 2
        remaining_size = (remaining_size + 1) // 2

    return current_hash == expected_root

3.1.2 Checkpoint Verification

Rekor checkpoints are signed using the log’s private key:

Checkpoint Format:
─────────────────
rekor.sigstore.dev - 1234567
<tree_size>
<root_hash_base64>

— rekor.sigstore.dev <signature>

Verification steps:

  1. Parse checkpoint text format
  2. Extract signature and public key hint
  3. Verify Ed25519/ECDSA signature over checkpoint body
  4. Extract root hash and tree size

3.1.3 Implementation Classes

/// <summary>
/// RFC 6962 Merkle proof verification.
/// </summary>
public static class MerkleProofVerifier
{
    private static readonly byte LeafPrefix = 0x00;
    private static readonly byte NodePrefix = 0x01;

    public static bool VerifyInclusion(
        byte[] leafHash,
        long leafIndex,
        long treeSize,
        IReadOnlyList<byte[]> proofPath,
        byte[] expectedRoot)
    {
        ArgumentNullException.ThrowIfNull(leafHash);
        ArgumentNullException.ThrowIfNull(proofPath);
        ArgumentNullException.ThrowIfNull(expectedRoot);

        if (leafHash.Length != 32 || expectedRoot.Length != 32)
            throw new ArgumentException("Hash must be 32 bytes (SHA-256)");

        if (leafIndex < 0 || leafIndex >= treeSize)
            throw new ArgumentOutOfRangeException(nameof(leafIndex));

        var currentHash = leafHash;
        var currentIndex = leafIndex;
        var currentSize = treeSize;

        foreach (var siblingHash in proofPath)
        {
            if (siblingHash.Length != 32)
                throw new ArgumentException("Sibling hash must be 32 bytes");

            if (currentIndex % 2 == 1)
            {
                // Current is right child
                currentHash = HashInterior(siblingHash, currentHash);
            }
            else
            {
                // Current is left child
                currentHash = HashInterior(currentHash, siblingHash);
            }

            currentIndex /= 2;
            currentSize = (currentSize + 1) / 2;
        }

        return currentHash.AsSpan().SequenceEqual(expectedRoot);
    }

    private static byte[] HashInterior(byte[] left, byte[] right)
    {
        Span<byte> buffer = stackalloc byte[1 + 32 + 32];
        buffer[0] = NodePrefix;
        left.CopyTo(buffer.Slice(1, 32));
        right.CopyTo(buffer.Slice(33, 32));
        return SHA256.HashData(buffer);
    }

    public static byte[] ComputeLeafHash(byte[] entryData)
    {
        Span<byte> buffer = stackalloc byte[1 + entryData.Length];
        buffer[0] = LeafPrefix;
        entryData.CopyTo(buffer.Slice(1));
        return SHA256.HashData(buffer);
    }
}

3.2 Durable Retry Queue

3.2.1 State Machine

                         PENDING
                            │
                            │ Worker picks up
                            ▼
                       SUBMITTING
                      /          \
                     /            \
               (success)        (failure)
                  /                \
                 ▼                  ▼
            SUBMITTED            RETRYING
                                    │
                                    │ (after backoff delay)
                                    ▼
                               SUBMITTING
                                    │
                                    │ (max attempts exceeded)
                                    ▼
                              DEAD_LETTER

3.2.2 Exponential Backoff

public static TimeSpan CalculateBackoff(int attemptCount, RekorQueueOptions options)
{
    var delayMs = options.InitialDelayMs * Math.Pow(options.BackoffMultiplier, attemptCount - 1);
    var cappedDelayMs = Math.Min(delayMs, options.MaxDelayMs);

    // Add jitter (±10%)
    var jitter = Random.Shared.NextDouble() * 0.2 - 0.1;
    var finalDelayMs = cappedDelayMs * (1 + jitter);

    return TimeSpan.FromMilliseconds(finalDelayMs);
}

Default backoff sequence:

AttemptBase DelayWith Jitter
11s0.9s - 1.1s
22s1.8s - 2.2s
34s3.6s - 4.4s
48s7.2s - 8.8s
516s14.4s - 17.6s

3.2.3 Queue Table Design

CREATE TABLE attestor_rekor_queue (
    id                  UUID PRIMARY KEY,
    tenant_id           TEXT NOT NULL,
    bundle_sha256       TEXT NOT NULL UNIQUE,  -- Idempotency key
    dsse_payload        BYTEA NOT NULL,
    backend             TEXT NOT NULL,
    status              TEXT NOT NULL,
    attempt_count       INT NOT NULL DEFAULT 0,
    max_attempts        INT NOT NULL,
    last_attempt_at     TIMESTAMPTZ,
    last_error          TEXT,
    next_retry_at       TIMESTAMPTZ,
    rekor_uuid          TEXT,                  -- Set on success
    rekor_log_index     BIGINT,                -- Set on success
    created_at          TIMESTAMPTZ NOT NULL,
    updated_at          TIMESTAMPTZ NOT NULL
);

-- Efficient dequeue query
CREATE INDEX idx_rekor_queue_dequeue
    ON attestor_rekor_queue (next_retry_at, status)
    WHERE status IN ('pending', 'retrying');

3.2.4 Dequeue Query

-- Atomic dequeue with row locking
WITH eligible AS (
    SELECT id
    FROM attestor_rekor_queue
    WHERE status IN ('pending', 'retrying')
      AND (next_retry_at IS NULL OR next_retry_at <= NOW())
    ORDER BY next_retry_at NULLS FIRST, created_at
    LIMIT :batch_size
    FOR UPDATE SKIP LOCKED
)
UPDATE attestor_rekor_queue q
SET status = 'submitting',
    updated_at = NOW()
FROM eligible e
WHERE q.id = e.id
RETURNING q.*;

3.3 Time Skew Validation

3.3.1 Threat Model

AttackDescriptionDetection
Backdated EntryAttacker inserts entry with old timestampLarge positive skew
Future TimestampAttacker pre-dates entryNegative skew (future)
Log ManipulationAttacker modifies existing entriesTimestamp inconsistency

3.3.2 Threshold Design

                    Time Skew Detection Zones
    ─────────────────────────────────────────────────────────────────
    │ REJECT │   WARN   │       OK       │   WARN   │ REJECT │
    │ FUTURE │  FUTURE  │                │   PAST   │  PAST  │
    ─────────────────────────────────────────────────────────────────
    ◄───────────────────────┼───────────────────────►
         -60s     -5m       NOW      +5m      +1h
                                    (local time)

    Default Thresholds:
    • Future tolerance: 60 seconds (beyond = reject)
    • Warn threshold: 5 minutes
    • Reject threshold: 1 hour

3.3.3 Validation Flow

public TimeSkewResult Validate(DateTimeOffset integratedTime, DateTimeOffset localTime)
{
    var skew = integratedTime - localTime;

    // Future timestamps are highly suspicious
    if (skew > TimeSpan.Zero)
    {
        if (skew > _options.FutureTolerance)
        {
            return Rejected($"Future timestamp by {skew}");
        }
        return Ok(skew);  // Within future tolerance
    }

    // Past timestamps (normal case - Rekor time is in the past)
    var absSkew = skew.Duration();

    if (absSkew >= _options.RejectThreshold)
    {
        return Rejected($"Time skew {absSkew} exceeds reject threshold");
    }

    if (absSkew >= _options.WarnThreshold)
    {
        return Warning($"Time skew {absSkew} exceeds warn threshold");
    }

    return Ok(skew);
}

3.4 Tile-Based Verification (Rekor v2)

Rekor v2 introduces a tile-based log structure following the Sunlight/C2SP tlog-tiles specification. This enables offline-capable verification and more efficient proof computation.

3.4.1 Architecture Overview

In tile-based logs, the Merkle tree is stored in fixed-size chunks (tiles) of 256 entries each:

                    Tile Structure (256 entries/tile)
    ───────────────────────────────────────────────────────────
                           Level 2 (root)
                              [Tile]
                             /      \
                    Level 1 (intermediate)
                   [Tile 0]  [Tile 1]  ...
                   /      \
              Level 0 (leaves)
    [Tile 0] [Tile 1] [Tile 2] [Tile 3] ...

    Each tile contains up to 256 hashes (32 bytes each = 8KB max)

3.4.2 Log Version Configuration

StellaOps supports automatic selection and explicit version selection:

public enum RekorLogVersion
{
    Auto = 0,   // Auto-selects v2 tiles
    V2 = 2      // Tile-based Sunlight format
}

Version Selection Logic:

VersionResult
V2Always use tile proofs
AutoAlways use tile proofs

3.4.3 Checkpoint Format

V2 checkpoints follow the c2sp.org/tlog-tiles format:

rekor.sigstore.dev - 2605736670972794746
<tree_size>
<root_hash_base64>

- rekor.sigstore.dev <signature_base64>

Checkpoint Components:

3.4.4 Tile Path Calculation

Tiles are fetched via URL paths following the scheme:

GET {tile_base_url}/tile/{level}/{index:03d}[.p/{partial_width}]

Examples:
- /tile/0/000        # Level 0, tile 0 (entries 0-255)
- /tile/0/001        # Level 0, tile 1 (entries 256-511)
- /tile/1/000        # Level 1, tile 0 (intermediate hashes)
- /tile/0/042.p/128  # Partial tile with 128 entries

3.4.5 Implementation Classes

IRekorTileClient Interface:

public interface IRekorTileClient
{
    Task<RekorTileCheckpoint?> GetCheckpointAsync(
        RekorBackend backend,
        CancellationToken cancellationToken = default);

    Task<RekorTileData?> GetTileAsync(
        RekorBackend backend,
        int level,
        long index,
        CancellationToken cancellationToken = default);

    Task<RekorTileEntry?> GetEntryAsync(
        RekorBackend backend,
        long logIndex,
        CancellationToken cancellationToken = default);

    Task<RekorTileInclusionProof?> ComputeInclusionProofAsync(
        RekorBackend backend,
        long logIndex,
        long treeSize,
        CancellationToken cancellationToken = default);
}

RekorTileData Model:

public sealed class RekorTileData
{
    public required int Level { get; init; }
    public required long Index { get; init; }
    public required int Width { get; init; }  // Number of hashes (max 256)
    public required byte[] Hashes { get; init; }  // Width * 32 bytes

    public byte[] GetHash(int position)
    {
        if (position < 0 || position >= Width)
            throw new ArgumentOutOfRangeException(nameof(position));

        var result = new byte[32];
        Array.Copy(Hashes, position * 32, result, 0, 32);
        return result;
    }
}

3.4.6 Proof Computation Algorithm

Computing an inclusion proof from tiles:

def compute_inclusion_proof(log_index, tree_size, tile_client):
    """Compute inclusion proof by fetching necessary tiles."""
    proof_path = []
    level = 0
    index = log_index
    size = tree_size

    while size > 1:
        tile_index = index // 256
        position_in_tile = index % 256

        # Determine sibling position
        if index % 2 == 1:
            sibling_pos = position_in_tile - 1
        else:
            sibling_pos = position_in_tile + 1 if position_in_tile + 1 < size else None

        if sibling_pos is not None:
            tile = tile_client.get_tile(level, tile_index)
            proof_path.append(tile.get_hash(sibling_pos))

        index = index // 2
        size = (size + 1) // 2
        level += 1

    return proof_path

3.4.7 Configuration

attestor:
  rekor:
    primary:
      url: https://rekor.sigstore.dev
      # Version: Auto or V2
      version: Auto
      # Custom tile base URL (optional, defaults to {url}/tile/)
      tile_base_url: ""
      # Log ID for multi-log environments (hex-encoded SHA-256)
      log_id: "c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d"

Environment Variables:

# Rekor v2 Configuration
REKOR_SERVER_URL=https://rekor.sigstore.dev
REKOR_VERSION=Auto           # Auto or V2
REKOR_TILE_BASE_URL=         # Optional custom tile endpoint
REKOR_LOG_ID=c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d

3.4.8 Offline Verification Benefits

Tile-based verification enables true offline capability:

  1. Pre-fetch tiles: Download all necessary tiles during online phase
  2. Bundle checkpoint: Include signed checkpoint with offline kit
  3. Local proof computation: Compute proofs entirely from local tile data
  4. No API dependency: Verification works without Rekor connectivity
┌─────────────────────────────────────────────────────────────┐
│                    Offline Verification                      │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌─────────────┐     ┌──────────────┐     ┌──────────────┐  │
│  │ Checkpoint  │────►│ Tile Cache   │────►│ Proof        │  │
│  │ (signed)    │     │ (local)      │     │ Verifier     │  │
│  └─────────────┘     └──────────────┘     └──────────────┘  │
│                                                              │
│  Advantages:                                                 │
│  - No network round-trips for proof fetching                │
│  - Deterministic verification (same tiles = same proof)     │
│  - Caching efficiency (tiles are immutable)                 │
│  - Air-gap compatible                                        │
│                                                              │
└─────────────────────────────────────────────────────────────┘

4. DATA FLOW

4.1 Submission Flow (with Queue)

┌─────────┐      ┌──────────────────┐      ┌───────────┐
│ Client  │─────►│ SubmissionService │─────►│ RekorAPI  │
└─────────┘      └────────┬─────────┘      └─────┬─────┘
                          │                       │
                          │ (success)             │
                          ▼                       │
                 ┌─────────────────┐              │
                 │ Store Entry     │◄─────────────┘
                 │ (status=ok)     │
                 └─────────────────┘

                          │ (failure)
                          ▼
                 ┌─────────────────┐
                 │ Enqueue         │
                 │ (status=pending)│
                 └────────┬────────┘
                          │
                          ▼
                 ┌─────────────────┐      ┌───────────┐
                 │ RetryWorker     │─────►│ RekorAPI  │
                 │ (background)    │      └─────┬─────┘
                 └─────────────────┘            │
                          ▲                     │
                          │                     │
                          └─────────────────────┘
                              (retry loop)

4.2 Verification Flow (with Proof Verification)

┌─────────┐      ┌───────────────────┐      ┌─────────────────┐
│ Client  │─────►│ VerificationSvc   │─────►│ EntryRepository │
└─────────┘      └────────┬──────────┘      └────────┬────────┘
                          │                          │
                          │◄─────────────────────────┘
                          │      (AttestorEntry)
                          ▼
                 ┌─────────────────────┐
                 │ 1. TimeSkewValidator│
                 │    (integrated_time)│
                 └────────┬────────────┘
                          │
                          ▼
                 ┌─────────────────────┐
                 │ 2. MerkleProof      │
                 │    Verifier         │
                 └────────┬────────────┘
                          │
                          ▼
                 ┌─────────────────────┐
                 │ 3. Checkpoint       │
                 │    Verifier         │
                 └────────┬────────────┘
                          │
                          ▼
                 ┌─────────────────────┐
                 │ 4. Aggregate Result │
                 │    (VerificationRpt)│
                 └─────────────────────┘

5. CONFIGURATION REFERENCE

# attestor.yaml

attestor:
  rekor:
    primary:
      url: https://rekor.sigstore.dev
      proof_timeout_ms: 15000
      poll_interval_ms: 250
      max_attempts: 60

    mirror:
      enabled: false
      url: https://rekor-mirror.internal

    verification:
      public_key_path: /etc/stellaops/rekor-pub.pem
      # Or inline:
      # public_key_base64: LS0tLS1CRUdJTi...
      allow_offline_without_signature: false
      max_checkpoint_age_minutes: 60

    queue:
      enabled: true
      max_attempts: 5
      initial_delay_ms: 1000
      max_delay_ms: 60000
      backoff_multiplier: 2.0
      batch_size: 10
      poll_interval_ms: 5000
      dead_letter_retention_days: 30

    time_skew:
      enabled: true
      warn_threshold_seconds: 300      # 5 minutes
      reject_threshold_seconds: 3600   # 1 hour
      future_tolerance_seconds: 60     # 1 minute
      reject_future_timestamps: true
      skip_in_offline_mode: true

6. METRICS REFERENCE

MetricTypeLabelsDescription
attestor.inclusion_verify_totalCounterresultInclusion proof verifications
attestor.inclusion_verify_latency_secondsHistogramVerification latency
attestor.checkpoint_verify_totalCounterresultCheckpoint signature verifications
attestor.rekor_queue_depthGaugeCurrent pending + retrying items
attestor.rekor_retry_attempts_totalCounterbackend, attemptRetry attempts
attestor.rekor_submission_status_totalCounterstatus, backendSubmission outcomes
attestor.rekor_queue_wait_secondsHistogramTime in queue before submission
attestor.time_skew_detected_totalCounterseverity, actionTime skew detections
attestor.time_skew_secondsHistogramObserved skew distribution

7. ERROR HANDLING

7.1 Error Taxonomy

Error CodeDescriptionRetry?Action
rekor_unavailableRekor API not reachableYesQueue for retry
rekor_conflictDuplicate entry (409)NoRetrieve existing entry
rekor_rate_limitedRate limit exceeded (429)YesBackoff and retry
rekor_internal_errorServer error (5xx)YesQueue for retry
proof_invalidMerkle proof verification failedNoReject, log alert
checkpoint_signature_invalidCheckpoint signature failedNoReject, log alert
time_skew_rejectedTime skew exceeds thresholdNoReject, log warning

7.2 Structured Logging

{
  "timestamp": "2025-12-14T10:30:00Z",
  "level": "Warning",
  "message": "Rekor submission failed, queuing for retry",
  "error_code": "rekor_unavailable",
  "bundle_sha256": "abc123...",
  "backend": "primary",
  "attempt_count": 1,
  "next_retry_at": "2025-12-14T10:30:02Z",
  "error_message": "Connection refused"
}

8. SECURITY CONSIDERATIONS

8.1 Key Management

8.2 Trust Model

                    Trust Hierarchy
                    ───────────────

                    ┌─────────────┐
                    │ Rekor Root  │
                    │ Public Key  │
                    └──────┬──────┘
                           │ signs
                           ▼
                    ┌─────────────┐
                    │ Checkpoint  │
                    │ (root hash) │
                    └──────┬──────┘
                           │ commits to
                           ▼
                    ┌─────────────┐
                    │ Merkle Tree │
                    │ (entries)   │
                    └──────┬──────┘
                           │ includes
                           ▼
                    ┌─────────────┐
                    │ Attestation │
                    │ (DSSE)      │
                    └─────────────┘

8.3 Offline Security

In air-gapped environments:


9. TESTING STRATEGY

9.1 Test Categories

CategoryCoverage TargetTools
Unit>90%xUnit, Moq
Integration>80%Testcontainers (PostgreSQL)
ContractAll public APIsSnapshot testing
PerformanceLatency P99BenchmarkDotNet

9.2 Golden Fixtures

Obtain from public Sigstore Rekor instance:

# Get a real Rekor entry for testing
rekor-cli get --uuid 24296fb24b8ad77a... --format json > fixtures/rekor-entry-valid.json

# Get checkpoint
curl https://rekor.sigstore.dev/api/v1/log > fixtures/rekor-checkpoint.json

# Get public key
curl https://rekor.sigstore.dev/api/v1/log/publicKey > fixtures/rekor-pubkey.pem

9A. PERIODIC VERIFICATION (Background Job)

Sprint Reference: SPRINT_20260117_001_ATTESTOR_periodic_rekor_verification

9A.1 Overview

The Periodic Verification system provides continuous validation of previously logged Rekor entries. This addresses the gap where entries are logged but never re-verified, enabling detection of:

9A.2 Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                   Periodic Verification Job                         │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ┌─────────────────────┐    ┌─────────────────────┐                 │
│  │ RekorVerification   │───►│ IRekorVerification  │                 │
│  │ Job (Scheduler)     │    │ Service             │                 │
│  └─────────┬───────────┘    └──────────┬──────────┘                 │
│            │                           │                             │
│            │ batch query               │ verify                      │
│            ▼                           ▼                             │
│  ┌─────────────────────┐    ┌─────────────────────┐                 │
│  │ IRekorEntry         │    │ RekorVerification   │                 │
│  │ Repository          │    │ Metrics             │                 │
│  └─────────────────────┘    └──────────┬──────────┘                 │
│                                        │                             │
│                                        ▼                             │
│                             ┌─────────────────────┐                 │
│                             │ IRekorVerification  │                 │
│                             │ StatusProvider      │                 │
│                             └─────────────────────┘                 │
└─────────────────────────────────────────────────────────────────────┘

9A.3 Configuration

attestor:
  rekor:
    verification:
      enabled: true
      intervalMinutes: 60           # Run every hour
      batchSize: 100                # Entries per batch
      sampleRate: 0.1               # 10% sampling for large deployments
      maxTimeSkewSeconds: 300       # 5 minute tolerance
      alertOnRootInconsistency: true

9A.4 Verification Checks

CheckDescriptionFailure Severity
SignatureVerify entry signature against stored public keyCritical
Inclusion ProofRFC 6962 Merkle inclusion proof verificationCritical
Time SkewValidate integrated_time within toleranceWarning
Root ConsistencyCompare stored tree root with remoteCritical

9A.5 Metrics (OpenTelemetry)

# Meter: StellaOps.Attestor.RekorVerification

attestor.rekor.verification.runs              # Counter
attestor.rekor.verification.entries.verified  # Counter
attestor.rekor.verification.entries.failed    # Counter
attestor.rekor.verification.entries.skipped   # Counter
attestor.rekor.verification.time_skew_violations # Counter
attestor.rekor.verification.signature_failures   # Counter
attestor.rekor.verification.inclusion_proof_failures # Counter
attestor.rekor.verification.root_consistency_checks  # Counter
attestor.rekor.verification.entry_duration    # Histogram
attestor.rekor.verification.batch_duration    # Histogram

9A.6 Health Check Integration

The RekorVerificationHealthCheck integrates with the Doctor diagnostic system:

Check ID: check.attestation.rekor.verification.job

Status Levels:
- Healthy: Last run within expected window, failure rate < 1%
- Degraded: Failure rate 1-5%, or last run overdue
- Unhealthy: Failure rate > 5%, root inconsistency detected, or job not running

9A.7 Alerting

ConditionAlert LevelAction
Root inconsistencyP1 CriticalImmediate investigation required
Signature failure rate > 5%P2 HighReview key material
Job not running > 3x intervalP3 MediumCheck scheduler
Time skew violations > 10%P3 MediumCheck NTP sync

9A.8 Offline Verification

When network access to Rekor is unavailable, the system falls back to stored inclusion proofs:

  1. Read stored inclusion_proof from database
  2. Verify Merkle path locally against stored root
  3. Log verification as “offline” mode
  4. Schedule online re-verification when connectivity returns

10. MIGRATION GUIDE

10.1 Database Migrations

Run in order:

  1. 00X_rekor_submission_queue.sql - Queue table
  2. Update AttestorEntry schema if stored in PostgreSQL

10.2 Configuration Migration

# Before (existing)
attestor:
  rekor:
    primary:
      url: https://rekor.sigstore.dev

# After (add new sections)
attestor:
  rekor:
    primary:
      url: https://rekor.sigstore.dev
    verification:
      public_key_path: /etc/stellaops/rekor-pub.pem
    queue:
      enabled: true
    time_skew:
      enabled: true

10.3 Rollback Plan


11. REFERENCES