Offline Parity Verification

Audience: air-gap operators, release engineers, and QA automation owners who must prove that disconnected scans are trustworthy.

This guide defines the methodology for verifying that the Stella Ops scanner produces identical results in offline / air-gapped environments and in connected deployments. Parity verification ensures that release-gate decisions made in a sealed enclave are equivalent to those made with full network access.

1. Parity verification objectives

1.1 Core Guarantees

1.2 What Parity Does NOT Cover

• Feed freshness: Offline feeds may be hours/days behind live feeds (by design)
• Network-only enrichment: EPSS lookups, live KEV checks (graceful degradation applies)
• Transparency log submission: Rekor entries created only when connected

2. Test methodology

2.1 Environment Configuration

Connected Environment

environment:
  mode: connected
  network: enabled
  feeds:
    sources: [osv, ghsa, nvd]
    refresh: live
  rekor: enabled
  epss: enabled
  timestamp_source: ntp

Offline Environment

environment:
  mode: offline
  network: disabled
  feeds:
    sources: [local-bundle]
    refresh: none
  rekor: offline-snapshot
  epss: bundled-cache
  timestamp_source: frozen
  timestamp_value: "2025-12-14T00:00:00Z"

2.2 Test Procedure

PARITY VERIFICATION PROCEDURE v1.0
══════════════════════════════════

PHASE 1: BUNDLE CAPTURE (Connected Environment)
─────────────────────────────────────────────────
1. Capture current feed state:
   - Record feed version/digest
   - Snapshot EPSS scores (top 1000 CVEs)
   - Record KEV list state

2. Run connected scan:
   stellaops scan --image <test-image> \
     --format json \
     --output connected-scan.json \
     --receipt connected-receipt.json

3. Export offline bundle:
   stellaops offline bundle export \
     --feeds-snapshot \
     --epss-cache \
     --output parity-bundle-$(date +%Y%m%d).tar.zst

PHASE 2: OFFLINE SCAN (Air-Gapped Environment)
───────────────────────────────────────────────
1. Import bundle:
   stellaops offline bundle import parity-bundle-*.tar.zst

2. Freeze clock to bundle timestamp:
   export STELLAOPS_DETERMINISM_TIMESTAMP="2025-12-14T00:00:00Z"

3. Run offline scan:
   stellaops scan --image <test-image> \
     --format json \
     --output offline-scan.json \
     --receipt offline-receipt.json \
     --offline-mode

PHASE 3: PARITY COMPARISON
──────────────────────────
1. Compare findings digests:
   diff <(jq -S '.findings | sort_by(.id)' connected-scan.json) \
        <(jq -S '.findings | sort_by(.id)' offline-scan.json)

2. Compare policy decisions:
   diff <(jq -S '.policyDecision' connected-scan.json) \
        <(jq -S '.policyDecision' offline-scan.json)

3. Compare receipt input hashes:
   jq '.inputHash' connected-receipt.json
   jq '.inputHash' offline-receipt.json
   # MUST be identical if same bundle used

PHASE 4: RECORD RESULTS
───────────────────────
1. Generate parity report:
   stellaops parity report \
     --connected connected-scan.json \
     --offline offline-scan.json \
     --output parity-report-$(date +%Y%m%d).json

2.3 Test Image Matrix

Run parity tests against this representative image set:

3. Comparison criteria

3.1 Bitwise Comparison

Compare canonical JSON outputs after normalization:

# Canonical comparison script
canonical_compare() {
    local connected="$1"
    local offline="$2"

    # Normalize both outputs
    jq -S . "$connected" > /tmp/connected-canonical.json
    jq -S . "$offline" > /tmp/offline-canonical.json

    # Compute hashes
    CONNECTED_HASH=$(sha256sum /tmp/connected-canonical.json | cut -d' ' -f1)
    OFFLINE_HASH=$(sha256sum /tmp/offline-canonical.json | cut -d' ' -f1)

    if [[ "$CONNECTED_HASH" == "$OFFLINE_HASH" ]]; then
        echo "PASS: Bitwise identical"
        return 0
    else
        echo "FAIL: Hash mismatch"
        echo "  Connected: $CONNECTED_HASH"
        echo "  Offline:   $OFFLINE_HASH"
        diff --color /tmp/connected-canonical.json /tmp/offline-canonical.json
        return 1
    fi
}

3.2 Semantic Comparison

When bitwise comparison fails, perform semantic comparison:

3.3 Fields Excluded from Comparison

These fields are expected to differ and are excluded from parity checks:

{
  "excludedFields": [
    "$.metadata.scanId",
    "$.metadata.timestamp",
    "$.metadata.hostname",
    "$.metadata.environment.network",
    "$.attestations[*].rekorEntry",
    "$.metadata.epssEnrichedAt"
  ]
}

3.4 Graceful Degradation Fields

Fields that may be absent in offline mode (acceptable):

4. Automated parity CI

4.1 CI Workflow

# .gitea/workflows/offline-parity.yml
name: Offline Parity Verification

on:
  schedule:
    - cron: '0 3 * * 1'  # Weekly Monday 3am
  workflow_dispatch:

jobs:
  parity-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup .NET
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: '10.0.x'

      - name: Set determinism environment
        run: |
          echo "TZ=UTC" >> $GITHUB_ENV
          echo "LC_ALL=C" >> $GITHUB_ENV
          echo "STELLAOPS_DETERMINISM_SEED=42" >> $GITHUB_ENV

      - name: Capture connected baseline
        run: scripts/parity/capture-connected.sh

      - name: Export offline bundle
        run: scripts/parity/export-bundle.sh

      - name: Run offline scan (sandboxed)
        run: |
          docker run --network none \
            -v $(pwd)/bundle:/bundle:ro \
            -v $(pwd)/results:/results \
            stellaops/scanner:latest \
            scan --offline-mode --bundle /bundle

      - name: Compare parity
        run: scripts/parity/compare-parity.sh

      - name: Upload parity report
        uses: actions/upload-artifact@v4
        with:
          name: parity-report
          path: results/parity-report-*.json

4.2 Parity Test Script

#!/bin/bash
# scripts/parity/compare-parity.sh

set -euo pipefail

CONNECTED_DIR="results/connected"
OFFLINE_DIR="results/offline"
REPORT_FILE="results/parity-report-$(date +%Y%m%d).json"

declare -a IMAGES=(
    "alpine:3.19"
    "debian:12-slim"
    "node:20-alpine"
    "python:3.12"
    "mcr.microsoft.com/dotnet/aspnet:8.0"
    "postgres:16-alpine"
)

TOTAL=0
PASSED=0
FAILED=0
RESULTS=()

for image in "${IMAGES[@]}"; do
    TOTAL=$((TOTAL + 1))
    image_hash=$(echo "$image" | sha256sum | cut -c1-12)

    connected_file="${CONNECTED_DIR}/${image_hash}-scan.json"
    offline_file="${OFFLINE_DIR}/${image_hash}-scan.json"

    # Compare findings
    connected_findings=$(jq -S '.findings | sort_by(.id) | map(del(.metadata.timestamp))' "$connected_file")
    offline_findings=$(jq -S '.findings | sort_by(.id) | map(del(.metadata.timestamp))' "$offline_file")

    connected_hash=$(echo "$connected_findings" | sha256sum | cut -d' ' -f1)
    offline_hash=$(echo "$offline_findings" | sha256sum | cut -d' ' -f1)

    if [[ "$connected_hash" == "$offline_hash" ]]; then
        PASSED=$((PASSED + 1))
        status="PASS"
    else
        FAILED=$((FAILED + 1))
        status="FAIL"
    fi

    RESULTS+=("{\"image\":\"$image\",\"status\":\"$status\",\"connectedHash\":\"$connected_hash\",\"offlineHash\":\"$offline_hash\"}")
done

# Generate report
cat > "$REPORT_FILE" <<EOF
{
  "reportDate": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
  "bundleVersion": "$(cat bundle/version.txt)",
  "summary": {
    "total": $TOTAL,
    "passed": $PASSED,
    "failed": $FAILED,
    "parityRate": $(echo "scale=4; $PASSED / $TOTAL" | bc)
  },
  "results": [$(IFS=,; echo "${RESULTS[*]}")]
}
EOF

echo "Parity Report: $PASSED/$TOTAL passed ($(echo "scale=2; $PASSED * 100 / $TOTAL" | bc)%)"

if [[ $FAILED -gt 0 ]]; then
    echo "PARITY VERIFICATION FAILED"
    exit 1
fi

5. Parity results

5.1 Latest Verification Results

5.2 Historical Parity Tracking

-- Query for parity trend analysis
SELECT
    date_trunc('week', report_date) AS week,
    AVG(parity_rate) AS avg_parity,
    MIN(parity_rate) AS min_parity,
    COUNT(*) AS test_runs
FROM parity_reports
WHERE report_date >= NOW() - INTERVAL '90 days'
GROUP BY 1
ORDER BY 1 DESC;

5.3 Parity Database Schema

CREATE TABLE scanner.parity_reports (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    report_date TIMESTAMPTZ NOT NULL,
    bundle_version TEXT NOT NULL,
    bundle_digest TEXT NOT NULL,
    total_images INT NOT NULL,
    passed_images INT NOT NULL,
    failed_images INT NOT NULL,
    parity_rate NUMERIC(5,4) NOT NULL,
    results JSONB NOT NULL,
    ci_run_id TEXT,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE INDEX idx_parity_reports_date ON scanner.parity_reports(report_date DESC);
CREATE INDEX idx_parity_reports_bundle ON scanner.parity_reports(bundle_version);

6. Known limitations

6.1 Acceptable Differences

6.2 Known Edge Cases

1. Race conditions during bundle capture: If feeds update during bundle export, connected scan may include newer data than bundle. Mitigation: Capture bundle first, then run connected scan.

2. Clock drift: Offline environments with drifted clocks may compute different freshness scores. Mitigation: Always use frozen timestamps from bundle.

3. Locale differences: String sorting may differ across locales. Mitigation: Force LC_ALL=C in both environments.

4. Floating point rounding: CVSS v4 MacroVector interpolation may have micro-differences. Mitigation: Use integer basis points throughout.

6.3 Out of Scope

The following are intentionally NOT covered by parity verification:

• Real-time threat intelligence (requires network)
• Live vulnerability disclosure (requires network)
• Transparency log inclusion proofs (requires Rekor)
• OIDC/Fulcio certificate chains (requires network)

7. Troubleshooting

7.1 Common Parity Failures

7.2 Debug Commands

# Compare feed versions
stellaops feeds version --connected
stellaops feeds version --offline --bundle ./bundle

# Compare policy digests
stellaops policy digest --connected
stellaops policy digest --offline --bundle ./bundle

# Detailed diff of findings
stellaops parity diff \
  --connected connected-scan.json \
  --offline offline-scan.json \
  --verbose

8. Metrics and monitoring

8.1 Prometheus Metrics

# Parity verification metrics
parity_test_total{status="pass|fail"}
parity_test_duration_seconds (histogram)
parity_bundle_age_seconds (gauge)
parity_findings_diff_count (gauge)

8.2 Alerting Rules

groups:
  - name: offline-parity
    rules:
      - alert: ParityTestFailed
        expr: parity_test_total{status="fail"} > 0
        for: 0m
        labels:
          severity: critical
        annotations:
          summary: "Offline parity test failed"

      - alert: ParityRateDegraded
        expr: |
          (sum(parity_test_total{status="pass"}) /
           sum(parity_test_total)) < 0.95
        for: 1h
        labels:
          severity: warning
        annotations:
          summary: "Parity rate below 95%"

9. References

• Airgap Overview — modes, lifecycle, and governance for sealed deployments.
• Offline Bundle Format — bundle layout, hashes, and DSSE metadata.
• Staleness and Time — time anchors and freshness thresholds.
• Offline Update Kit (OUK)
• Determinism CI Harness
• Performance Baselines