Registries & Masked Image References (operator guide)
Audience: the person installing and operating a Stella Ops deployment who needs to connect a container registry, understand the stella+oci:// reference they see everywhere, and (for hosts with no direct registry reachability) run the agent pull-through cache.
Scope: connecting a registry and what auto-scan does; the masked stella+oci:// reference vs. the real pull URL and which to use where; digest pinning; the agent loopback / pull-through path (one diagram); central upstreams and cache policy; air-gap notes.
Authoritative sources (this guide is a synthesis; the code wins):
- ADR-033 — Unified Registry Access (
stella+oci://masked refs) - ADR-034 — Agent secret-access broker, ADR-035 — digest-centric content broker
- Runbook: Registry bridge pull-through cache
- Registry integration + auto-scan: Integrations module architecture
1. Connecting a registry
1.1 Add a registry integration
Registries are added as integrations in the Console: Integrations → Registry Admin (route /ops/integrations/registry-admin, also /setup/integrations/registry-admin), or via POST /api/v1/integrations. A registry-typed integration provides type/provider, endpoint, and — because registry integrations must carry credentials — a secret reference, not a raw credential:
- credentials are addressed as a secret reference (
vault://…,builtin://…,authref://…) resolved through the routing secret provider (see the Secrets & Keys guide); Stella never stores an inline plaintext registry credential in a masked ref, deployment bundle, log, or audit payload (ADR-033 D4).
Supported registry providers (descriptor matrix in the Integrations dossier) span self-hosted and cloud-hosted registries you pull from / scan: generic OCI Distribution (registry:2), Harbor, Zot, Quay, Sonatype Nexus, JFrog Artifactory, GitLab Container Registry, GitHub Container Registry (GHCR), Docker Hub, and the cloud registries ACR / ECR / GCR.
“Cloud registry” here means a registry you scan — not a cloud KMS. The on-prem-first rule (no cloud-managed KMS for secret backends) is a separate concern; scanning images that happen to live in ACR/ECR/GCR is fully supported. The agent pull-through bridge (§4) additionally rejects public-cloud upstream hosts as bridge targets — that is a bridge-routing rule, not a scan-target rule.
1.2 What auto-scan does
Once a registry integration is connected and healthy, RegistryImageEnumerator discovers images so they are SBOM-ready (provider-aware):
- Generic OCI (Docker Hub /
registry:2/ Harbor / Quay / …):/v2/_catalog→/v2/{repo}/tags/list→ manifest HEAD (Docker-Content-Digest) to resolve each tag to a digest, reusing the connector’s Distribution v2 Bearer-challenge dance. - GitLab CE (GitLab Container Registry): the GitLab group/project registry API (GitLab CE denies
/v2/_catalogto non-admin PATs), reading the group fromExtendedConfig.gitlabGroupPath/gitlabGroupId.
Enumeration is bounded (MaxReposPerRegistry, MaxTagsPerRepo), deterministically ordered (repo asc, tag asc), honours the ExtendedConfig.scanRepoPattern / scanTagPattern globs, and is fail-soft (a single repo/tag failure is logged and skipped). A registry that is Failed / anonymous / credential-unresolvable is never enumerated (fail-closed) — connect valid credentials first.
2. The masked stella+oci:// reference vs. the real pull URL
Stella addresses every image by an opaque masked reference. This is the stored, displayed, and audited image identity — it is not Docker-pullable.
stella+oci://<backend-selector>/<object-path>[:tag][@sha256:<digest>]
The sha256 digest is the machine identity; masking is routing and presentation (ADR-033 D1). Server-side deployment code resolves the masked ref to a real, Docker-valid reference before compose / Docker / an agent ever sees it (ADR-033 D5) — a bare stella-ops/... string would be misread by Docker as a Docker Hub namespace, which is exactly why the stella+oci:// scheme exists.
2.1 Backend selectors
| Selector | Meaning | Pull behaviour |
|---|---|---|
registries/<slug-or-guid> | An external integrated registry (the integration slug; GUID fallback for old rows) | Resolves to the integration’s real host: <endpoint>/<object-path>[:tag]@sha256:<digest> |
bundle | Air-gap / deployment-agent-served content | Resolves to a loopback OCI endpoint on the target host (§4) |
meta | Internal OCI metadata / referrer registry (SBOM, VEX, attestation, decision referrers by subject digest) | Metadata-only — does not serve pullable image layers (fail-closed if asked to) |
registry | Internal primary image registry (System of Record) | Reserved — layer pull is fail-closed until that store is provisioned |
2.2 Which to use where
| Where | Use |
|---|---|
| Release identity, UI, audit, evidence, copy/paste between operators | the masked stella+oci://… ref |
Actual docker pull / compose transport | the real ref (host + repo @sha256:…) — produced server-side by resolution; you rarely type it |
Both are shown side-by-side in the tooling; the masked ref is primary identity, the real host/repo/digest ref is the transport endpoint. stella registry resolve converts between them without a backend call:
stella registry resolve "stella+oci://registries/primary-gitlab/core:dev"
stella registry resolve "registry.example.com/example-corp/app/core:dev" --reverse
3. Digest pinning
Deployment is digest-pinned. A masked ref may carry a tag for display or human entry, but every machine path resolves to @sha256:<64 lowercase hex>, and if both a tag and a digest are present the digest wins. The resolver rejects a non-digest deploy ref fail-closed (“Deploy ref must be digest-pinned (sha256:<64hex>)”) — tags are mutable and are frozen to a digest before deploy.
This mirrors the supply-chain rule for the platform’s own images: deploy by digest, not by tag. Air-gap / bundle refs are held to the same digest-pinned requirement.
4. The agent loopback / pull-through path
Hosts with no direct registry reachability (or fully air-gapped hosts) pull through a loopback OCI registry the deployment agent exposes. The masked ref resolves to a 127.0.0.1 real ref so Docker pulls from the target’s own loopback address (Docker permits plain-HTTP pulls from 127.0.0.0/8 with no daemon-wide insecure-registries change; remote SSH deploys use a reverse tunnel).
Real ref rewritten to loopback (digest preserved):
127.0.0.1:<agentPort>/registries/<slug>/<object-path>[:tag]@sha256:<digest>
masked ref digest-pinned pull
Release/Deploy ───────────────► Resolver ─────────────────────────────┐
(stella+oci://registries/ (server-side, │
<slug>/core@sha256:…) MaskedImageRefResolver) ▼
┌──────────────────────┐
Docker on │ Agent loopback :port │
target ◄─┤ (127.0.0.1 only — │
pulls │ non-loopback bind │
by digest │ rejected fail-closed)│
└───────────┬───────────┘
│ L1 miss
┌─────────────────────────────┼───────────────┐
▼ ▼ ▼
L1 cache (disk, L2 shared store Upstream registry
CacheRoot, digest-keyed) (optional, promote (real host, creds via
hit ⇒ serve local on hit) secret provider)
Read path: L1 (agent disk) → L2 (optional shared, promote on hit) → upstream pull-through. Content is digest-keyed end to end, so identical content dedups to one object fleet-wide.
4.1 Two fetch models (know which one you run)
- Standalone pull-through (runbook §1–§8): the agent itself fetches upstream, resolving the upstream
credentialRefvia the ADR-034 secret broker at pull time. - Digest-centric content broker (runbook §9, ADR-035): on the deploy path the edge agent holds no standing upstream credential and no L2 key — it fetches content by digest from the orchestrator over its authorized mTLS channel, scoped to the deployment’s pinned digest by a sealed capability it cannot forge or widen. The orchestrator alone holds the upstream creds and the L2 key. The bytes are digest-verified by the orchestrator before serving and re-verified by the agent on receipt (defence-in-depth).
4.2 Enable the bridge
Configured under Agent:LocalRegistry (loopback-only; a non-loopback BindAddress is rejected at startup):
{ "Agent": { "LocalRegistry": {
"Enabled": true,
"Mode": "PullThrough", // or "BundleOnly" for air-gap
"BindAddress": "127.0.0.1", // loopback only; ::1 also allowed
"Port": 8021,
"CacheRoot": "/var/lib/stellaops/agent-registry-cache" // durable, backed-up path
}}}
Set a durable
CacheRooton a persistent, backed-up path. If unset it defaults underLocalApplicationData(deliberately not%TEMP%); a temp path would be wiped by host maintenance and cold-start (or, inBundleOnly, break an air-gap agent whose only image source is the imported bundle).
5. Upstreams + cache policy
5.1 Which real registry a slug bridges to (upstreams)
A masked pull for registries/<slug>/… resolves to a configured upstream host; an unknown or disabled slug is refused (fail-closed). Two provisioning paths:
- Agent-side static config (
Agent:LocalRegistry:Upstreams) — the only routing source an air-gap agent uses: sealed with the agent, immutable at runtime, no network needed. - Central control-plane CRUD (
/api/v1/registry/upstreams) — operators author the slug → host map centrally; connected agents opt in to pull the effective (non-secret) map (UpstreamRefresh, off by default). CLI:stella registry upstream list/set/delete.
Authoring is fail-closed: the slug must match ^[a-z0-9][a-z0-9-]*$ (no /, dots, or whitespace — it is split on the first /); a credentialRef must be a recognised secret reference (a bare user:password is rejected, and credentialRef is never returned on GET /effective); and the upstream host must be self-hosted / on-prem — public cloud-managed registry hosts (*.amazonaws.com, *.azurecr.io, gcr.io, *.pkg.dev) are rejected with a 400. Authoring scope is registry.admin(note the dot, distinct from registry:cache:admin); read views need registry:read.
5.2 Cache policy (size cap, watermarks, eviction, pinning)
Cache policy is authored centrally, scoped Global < Tenant < Upstream < Repository (field-level fallthrough). Scope authoring is registry:cache:admin; reads registry:read. Defaults: Enabled=true, MaxCacheBytes=0 (unbounded), high/low watermarks 90/75.
stella registry cache set-policy --scope Global \
--max-bytes 21474836480 --high 90 --low 75 --enabled true # 20 GiB cap
- Size-capped LRU with hysteresis: once usage crosses the high watermark, eviction drains to the low watermark;
GETbumps last-access so hot content survives.MaxCacheBytes=0disables eviction (unbounded). - Pinning (never evicted): bundle-imported (air-gap) content is pinned intrinsically; operators can also pin a digest.
Unbounded default + durable root = unbounded disk. With
MaxCacheBytes=0the cache grows with every unique layer. Set a real cap on any finite-disk host.MaxIdleis stored on the policy but the live L1 evictor does not yet act on it — bound the cache withMaxCacheBytes, notMaxIdle.
Troubleshooting (low hit ratio, eviction churn, unknown-slug not-found, 403/400 on authoring, air-gap fetch attempts) is tabulated in the runbook §8, alongside the Grafana bridge-cache dashboard.
6. Air-gap notes
- Run the agent in
Mode=BundleOnly: its only image source is an imported OCI bundle. The agent verifies (sha256) and imports the OCI layout, and that content is pinned intrinsically — an aggressive cache policy can never evict an air-gap agent’s only image source. - The agent’s routing table stays immutable and auditable: there is intentionally no add-upstream mutation surface on the agent loopback; an air-gap agent uses only its sealed static
Agent:LocalRegistry:Upstreamsand ignores the central map andUpstreamRefresh/PolicyRefreshpull channels. - Air-gap refs are still digest-pinned (§3) and served from the bundle-backed content store over the same
/v2/loopback shape. - A
BundleOnlyagent that shows upstream fetches is misconfigured — confirmOciLayoutBundlePathimported and the content is pinned (runbook §8).
Related
- Registry credentials + the secret provider: Secrets & Keys guide
- Pull-through cache operations, upstreams, dashboard: registry-bridge-cache.md
- ADRs: ADR-033, ADR-034, ADR-035, ADR-024 (internal OCI metadata registry)
- Code:
src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.Deployment/Registry/MaskedImageRefResolver.cs,src/ReleaseOrchestrator/__Agents/StellaOps.Agent.Registry/
