Runbook — Registry added but images not scanned / deploy gate blocked on missing SBOM
Symptom. An operator connects a container registry, but a release/deploy of one of its images is blocked with sbom-readiness: absent (HTTP 409 at the deploy gate), even though the image exists in the registry. GET /api/v1/sbom-readiness?digest=… returns absent for the image digest and no scan was ever submitted.
Root cause (pre–Sprint 20260607_001). Adding/connecting a registry only ran a test-connection probe; it never enumerated the registry’s repositories or submitted a scan, and there was no scan-on-push. SBOM-readiness therefore stayed absent for every image digest, and the gate (which requires an SBOM) blocked with nothing to evaluate.
Fix (Sprint 20260607_001 — registry auto-scan). When a registry integration becomes Active, integrations-web enumerates its repositories/tags, resolves each tag to its image digest, and appends a row to integrations.registry_image_discoveries. Scanner.Worker’s RegistryImageDiscoveredBridgeService consumes that outbox and submits a readiness-gated scan per digest, so readiness flips absent → pending → scanning → ready automatically. A periodic catch-up sweep covers registries that were Active before the feature shipped, tags pushed after the on-connect scan, and on-connect failures.
1. Confirm the symptom
# Readiness for the blocked digest (replace digest + tenant):
curl -s "https://<host>/api/v1/sbom-readiness?digest=sha256:<digest>&tenant=<tenant>" | jq
# state: "absent" → no scan exists yet (this runbook)
# state: "failed" → a scan ran and failed; see step 5 (FailureReason explains why)
# state: "scanning"/"pending" → a scan is in flight; wait and re-check
# state: "ready" → an SBOM exists; the gate should pass — investigate the gate, not the scan
2. Confirm auto-scan is enabled
Auto-scan on connect follows the service default Integrations:RegistryAutoScan:Enabled (default true), overridable per integration via ExtendedConfig.autoScanOnConnect.
- Check the integration’s
extendedConfigdoes not setautoScanOnConnect: false. - Check the registry integration is
Activeand authenticated (an anonymous/Failedregistry is never enumerated — registries must carry credentials, S046-001). - For GitLab Container Registry,
ExtendedConfig.gitlabGroupPath(orgitlabGroupId) must be set — GitLab CE denies/v2/_catalogto non-admin PATs, so enumeration uses the GitLab registry API which needs the group. Without it, enumeration is skipped (logged).
3. Confirm the discovery outbox is being written
# Rows appended by enumeration for this tenant (most recent first):
psql "$STELLAOPS_PG" -c "SELECT id, repository_path, tag, image_digest, source, discovered_at \
FROM integrations.registry_image_discoveries \
WHERE tenant_id = '<tenant>' ORDER BY id DESC LIMIT 20;"
- Rows present → enumeration worked; go to step 4 (the Scanner bridge).
- No rows → enumeration did not run or found nothing. Check integrations-web logs for
registry.auto-scan.on-connect/registry.auto-scan.sweepsummaries and any enumeration warnings (denied catalog, missing GitLab group, credential unresolved).
4. Confirm the Scanner bridge submitted a scan
# The deterministic scan id is SHA-256(seed)[..40] over
# ("stellaops.scanner.registry-image.v1", tenant, registryHost, repoPath, digest).
# Look it up by target digest instead:
psql "$STELLAOPS_PG" -c "SELECT scan_id, status, failure_reason, updated_at \
FROM scanner.scan_runtime_state \
WHERE tenant_id = '<tenant>' AND target_digest = 'sha256:<digest>' ORDER BY updated_at DESC;"
Pending/Running→ the scan is in flight; readiness will flip toreadyshortly.- No row → the bridge has not consumed the outbox yet. Confirm
Scanner:Worker:RegistryImageDiscoveredTrigger:Enabledistrueand the worker has scanner storage + the durable Redis queue configured (the bridge only registers when both are present). Check worker logs forregistry.image.scan-submitted.
5. If readiness is failed
A failed digest is re-submitted at most FailedRetryMaxAttempts times and no more often than FailedRetryMinInterval since the last attempt (Scanner:Worker:RegistryImageDiscoveredTrigger:*). After the budget is exhausted the auto-scan stops to avoid hot-looping a permanently-unscannable image; the readiness endpoint keeps FailureReason so you can see why. Common causes: unreachable manifest, unsupported media type, missing pull credential. Fix the underlying cause, then force a re-scan (step 6).
6. Force a scan manually (escape hatch)
The auto-scan path removes the need for this, but a manual submit still works:
curl -s -X POST "https://<host>/api/v1/scans" \
-H "Authorization: Bearer <token>" -H "Content-Type: application/json" \
-d '{"digest":"sha256:<digest>","tenant":"<tenant>"}'
7. Tuning / disabling
- Periodic sweep (off by default):
Integrations:RegistryAutoScan:Sweep:Enabled=true, plusInterval,InitialDelay,MaxIntegrationsPerSweep,MaxImagesPerSweep(the global per-sweep image cap prevents flooding the scan queue). - Per-registry scope:
ExtendedConfig.scanRepoPattern/scanTagPatternglobs (e.g. only:dev/:latest, not every tag) and the serviceMaxReposPerRegistry/MaxTagsPerRepocaps. - Disable on-connect auto-scan: set
Integrations:RegistryAutoScan:Enabled=false(service-wide) orExtendedConfig.autoScanOnConnect=false(per integration).
References
- Scanner dossier — “Registry auto-scan trigger”:
docs/modules/scanner/architecture.md - Integrations dossier — “Registry auto-scan”:
docs/modules/integrations/architecture.md - Sprint:
docs/implplan/SPRINT_20260607_001_Integrations_auto_scan_on_registry_add.md
