Export Center Mirror Bundles

Mirror bundles package StellaOps evidence, policy overlays, and indexes for air-gapped or bandwidth-constrained environments. They are produced by the mirror:full and mirror:delta profiles described in Epic 10 (Export Center) and implemented across Sprints 35-37 (EXPORT-SVC-35-004, EXPORT-SVC-37-001, EXPORT-SVC-37-002). This guide details bundle layouts, delta mechanics, encryption workflow, import procedures, and operational best practices.

Export Center workers are being wired while this document is written. Treat the content as the target contract for adapter development and update specifics as the implementation lands.

1. Bundle overview

Both profiles respect AOC boundaries: raw ingestion data remains untouched, and policy outputs live under their own directory with explicit provenance.

2. Filesystem layout

Directory structure inside the extracted bundle:

mirror/
  manifest.yaml
  export.json
  provenance.json
  README.md
  indexes/
    advisories.index.json
    vex.index.json
    sbom.index.json
    findings.index.json
  data/
    raw/
      advisories/*.jsonl.zst
      vex/*.jsonl.zst
      sboms/<subject>/sbom.json
    normalized/
      advisories/*.jsonl.zst
      vex/*.jsonl.zst
    policy/
      snapshot.json
      evaluations.jsonl.zst
    consensus/
      vex_consensus.jsonl.zst
  signatures/
    export.sig
    manifest.sig

manifest.yaml summarises profile metadata, selectors, counts, sizes, and SHA-256 digests. export.json and provenance.json mirror the JSON profile manifests and are signed using the configured KMS key.

Example manifest.yaml:

profile: mirror:full
runId: run-20251029-01
tenant: acme
selectors:
  products:
    - registry.example.com/app:*
  timeWindow:
    from: 2025-10-01T00:00:00Z
    to: 2025-10-29T00:00:00Z
counts:
  advisories: 15234
  vex: 3045
  sboms: 872
  policyEvaluations: 19876
artifacts:
  - path: data/raw/advisories/a0.jsonl.zst
    sha256: 9f4b...
    bytes: 7340021
encryption:
  mode: age
  strict: false
  recipients:
    - age1tenantkey...

3. Delta mechanics

Delta bundles reference a previous full or delta run via baseExportId and baseManifestDigest. They contain:

delta/
  changed/
    data/raw/advisories/*.jsonl.zst
    ...
  removed/
    advisories.jsonl        # list of advisory IDs removed
    vex.jsonl
    sboms.jsonl
  manifest.diff.json        # summary of counts, hashes, base export metadata

• Base lookup: The worker verifies that the base export is reachable (download path or OCI reference). If missing, the run fails with ERR_EXPORT_BASE_MISSING.
• Change detection: Uses deterministic hashing of normalized records to compute additions/updates. Indexes are regenerated only for affected subjects.
• Application order: Consumers apply deltas sequentially. A resetBaseline=true flag instructs them to drop cached state and apply the bundle as a full refresh.
• Tombstones required: Every removal must emit a tombstone entry plus the removed list; deltas without tombstones fail verification (verify-export-kit.sh).
• Integrity headers: Each delta bundle exports Digest, X-Stella-Signature, and X-Stella-Immutability derived from the OCI annotation io.stellaops.export.manifest-digest. Consumers must validate before applying.

Example manifest.diff.json (delta):

{
  "baseExportId": "run-20251025-01",
  "baseManifestDigest": "sha256:aa11...",
  "resetBaseline": false,
  "added": {
    "advisories": 43,
    "vex": 12,
    "sboms": 5
  },
  "changed": {
    "advisories": 18,
    "vex": 7
  },
  "removed": {
    "advisories": 2,
    "vex": 0,
    "sboms": 0
  }
}

4. Encryption workflow

Mirror bundles support optional encryption of the data/ subtree:

• Algorithm: Age (X25519) or AES-GCM (256-bit) based on profile configuration.
• Key wrapping: Keys fetched from Authority-managed KMS through Export Center. Wrapped data keys stored in provenance.json under encryption.recipients[].
• Metadata: manifest.yaml records encryption.mode, recipients, and encryptedPaths.
• Strict mode: strict=true encrypts everything except manifest.yaml and export.json. Default (false) leaves manifests unencrypted to simplify discovery.
• Verification: CLI (stella export verify) and Offline Kit scripts perform signature checks prior to decryption.

Operators must distribute recipient keys out of band. Export Center does not transmit private keys.

5. Import workflow

5.1 Offline Kit

Offline Kit bundles reference the latest full mirror export plus the last N deltas. Administrators run:

./offline-kit/bin/mirror import /path/to/mirror-20251029-full.tar.zst
./offline-kit/bin/mirror import /path/to/mirror-20251030-delta.tar.zst

The tool verifies signatures, applies deltas, and updates the mirror index served by the local gateway.

5.2 Custom automation

1. Download bundle (stella export download) and verify signatures (stella export verify).
2. Extract archive into a staging directory.
3. For encrypted bundles, decrypt using the provided age/AES key.
4. Sync mirror/data onto the target mirror store (object storage, NFS, etc.).
5. Republish indexes or reload services that depend on the mirror.

Delta consumers must track appliedExportIds to ensure ordering.

Sequence diagram of download/import:

sequenceDiagram
  participant CLI as stella CLI
  participant Mirror as Mirror Store
  participant Verify as Verification Tool
  CLI->>CLI: stella export download run-20251029-01
  CLI->>Verify: stella export verify run-20251029-01
  CLI->>Mirror: mirror import mirror-20251029-full.tar.zst
  CLI->>Mirror: mirror import mirror-20251030-delta.tar.zst
  Mirror-->>CLI: import complete (run-20251030-02)

6. Operational guidance

• Retention: Keep at least one full bundle plus the deltas required for disaster recovery. Configure ExportCenter:Retention:Mirror to prune older bundles automatically.
• Storage footprint: Full bundles can exceed tens of gigabytes. Plan object storage or NAS capacity accordingly and enable compression (compression.codec=zstd).
• Scheduling: For high-churn environments, run daily full exports and hourly deltas. Record cadence in manifest.yaml (schedule.cron).
• Incident recovery: To rebuild a mirror:
  1. Apply the most recent full bundle.
  2. Apply deltas in order of createdAt.
  3. Re-run integrity checks (mirror verify <path>).
• Audit logging: Export Center logs mirror.bundle.created, mirror.delta.applied, and mirror.encryption.enabled events. Consume them in the central observability pipeline.

7. Validation checklist (Trivy / mirror bundles)

• Download and verify:
  • stella export download <exportId> --format mirror
  • stella export verify <exportId>
• Delta ordering:
  • Ensure manifest.diff.json.baseExportId exists locally before applying delta.
  • Track applied order in appliedExportIds.log.
• Trivy adapter (if enabled):
  • stella export trivy-validate --bundle mirror-YYYYMMDD.tar.zst --policy ./policies/export-center.rego
• Dry-run import:
  • stella export mirror-validate --bundle mirror-YYYYMMDD.tar.zst --dry-run
• Post-import checks:
  • Recompute SHA256 for manifest.yaml and a sample data file; compare to manifest.
  • Run mirror verify (Offline Kit) and confirm zero mismatches.
  • Confirm OCI annotations io.stellaops.export.profile/run/manifest-digest/provenance-ref match the bundle being applied.

8. Troubleshooting

9. References

• Export Center Overview
• Export Center Architecture
• Export Center API reference
• Export Center CLI Guide
• Concelier mirror runbook
• Aggregation-Only Contract reference

Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.