Scanner Artifact Store Migration (MinIO → RustFS)

Audience: Scanner platform operators and deployment engineers cutting an existing estate over to the RustFS artifact store.

Overview

RustFS is the default artifact store for the Stella Ops Scanner plane. Existing deployments running MinIO (or any S3-compatible backend) must migrate their stored SBOM artefacts to RustFS before switching the Scanner hosts to scanner.artifactStore.driver = "rustfs".

This runbook covers the recommended migration workflow and its validation steps.

Prerequisites

• RustFS service deployed and reachable from the Scanner control plane (http(s)://rustfs:8080).
• Existing MinIO/S3 credentials with read access to the current bucket.
• CLI environment with the StellaOps source tree (for the migration tool) and dotnet 10 SDK.
• Maintenance window sized to copy all artefacts (migration is read-only on the source bucket).

1. Snapshot source bucket (optional but recommended)

If the MinIO deployment offers versioning or snapshots, take one before migrating. For non-versioned deployments, capture an external backup (e.g., mc mirror to offline storage).

2. Dry-run the migrator

dotnet run --project src/Tools/RustFsMigrator -- \
  --s3-bucket scanner-artifacts \
  --s3-endpoint http://stellaops-minio:9000 \
  --s3-access-key stellaops \
  --s3-secret-key dev-minio-secret \
  --rustfs-endpoint http://stellaops-rustfs:8080 \
  --rustfs-bucket scanner-artifacts \
  --prefix scanner/ \
  --dry-run

The dry-run enumerates keys and reports the object count without writing to RustFS. Use this to estimate migration time.

3. Execute migration

Remove the --dry-run flag to copy data. Optional flags:

• --immutable – mark all migrated objects as immutable (X-RustFS-Immutable).
• --retain-days 365 – request retention (in days) via X-RustFS-Retain-Seconds.
• --rustfs-api-key-header / --rustfs-api-key – provide auth headers when RustFS is protected.

The tool streams each object from S3 and performs an idempotent PUT to RustFS preserving the key structure (e.g., scanner/layers/<sha256>/sbom.cdx.json.zst).

4. Verify sample objects

Pick a handful of SBOM digests and confirm:

1. GET /api/v1/buckets/<bucket>/objects/<key> returns the expected payload (size + SHA-256).
2. Scanner WebService configured with scanner.artifactStore.driver = "rustfs" can fetch the same artefacts (Smoke test: GET /api/v1/scanner/sboms/<digest>?format=cdx-json).

5. Switch Scanner hosts

Update configuration (Helm/Compose/environment) to set:

scanner:
  artifactStore:
    driver: rustfs
    endpoint: http://stellaops-rustfs:8080
    bucket: scanner-artifacts
    timeoutSeconds: 30

Redeploy Scanner WebService and Worker. Monitor logs for RustFS upload/download messages and Prometheus scrape (rustfs_requests_total).

6. Cleanup legacy MinIO (optional)

After a complete migration and validation period, decommission the MinIO bucket or repurpose it for other components (Concelier still supports S3). Ensure backups reference RustFS snapshots going forward.

Troubleshooting

• Uploads fail (HTTP 4xx/5xx): Check RustFS logs and confirm API key headers. Re-run the migrator for the affected keys.
• Missing objects post-cutover: Re-run the migrator with the specific --prefix. The tool is idempotent and safely overwrites existing objects.
• Performance tuning: Run multiple instances of the migrator with disjoint prefixes if needed; the RustFS API is stateless and supports parallel PUTs.