PacksRegistry Architecture
Audience: developers, operators, and release engineers working on signed pack import, mirror, lifecycle, and offline seed flows.
PacksRegistry is the signed content registry for Stella Ops packs. It stores and serves task packs, policy/data packs, provenance, attestations, lifecycle state, mirror metadata, compliance summaries, and offline seed exports.
Source Locations
| Area | Path |
|---|---|
| Web API | src/JobEngine/StellaOps.PacksRegistry/StellaOps.PacksRegistry.WebService/ |
| Core services/models | src/JobEngine/StellaOps.PacksRegistry/StellaOps.PacksRegistry.Core/ |
| Infrastructure repositories | src/JobEngine/StellaOps.PacksRegistry/StellaOps.PacksRegistry.Infrastructure/ |
| Worker | src/JobEngine/StellaOps.PacksRegistry/StellaOps.PacksRegistry.Worker/ |
| Tests | src/JobEngine/StellaOps.PacksRegistry/StellaOps.PacksRegistry.Tests/ |
| API reference | docs/modules/packsregistry/api-reference.md |
| OpenAPI | docs/modules/packsregistry/openapi/v1.json |
Boundary
PacksRegistry is data-pack-only in the pluginized compose model. Importing a pack verifies and stores signed content; it must not execute code supplied by the pack. Executable behavior remains host-owned unless a future sprint creates a dedicated signed adapter contract with admission, probe, and sandbox rules. Scheduler ISchedulerJobPlugin bundles are not task packs; they are mounted under devops/plugins/scheduler/<profile>/ and admitted by the Scheduler host, not by PacksRegistry import.
The following remain declarative content:
- pack manifests
- policy and task definitions
- DSSE attestations
- provenance records
- mirror metadata
- lifecycle and parity records
- offline seed exports
Runtime Flow
- API clients upload or request packs through the web service.
PackService,AttestationService,LifecycleService,MirrorService,ParityService, andExportServicevalidate input and coordinate storage.- Repositories persist records through the configured driver.
- Signature verification is performed by the configured
IPackSignatureVerifier; verification decisions affect pack status but do not load executable assemblies. - The worker performs background pack maintenance and mirror work using the same core services and storage contracts.
Authentication & Authorization
PacksRegistry is an Authority resource server. Both halves of the identity are claim-derived — no raw header, query value, or body field is ever a tenancy or authorization source.
Authentication. Two paths converge on one scheme (StellaOpsBearer):
- Gateway-proxied (the normal path). The Router gateway validates the caller’s token, strips the
Authorizationheader, and forwards a signed identity envelope (X-StellaOps-Identity-Envelope+ HMAC signature).UseIdentityEnvelopeAuthentication()runs beforeUseAuthentication()and mints aStellaRouterEnvelopeprincipal from the verified envelope; the JwtBearerOnMessageReceivedbridge insideAddStellaOpsResourceServerAuthenticationthen accepts that principal as aStellaOpsBeareridentity. The native/api/v1/packsendpoints are published through Router. RAR-6 repoints Console source to that native root; the temporary^/api/v1/jobengine/registry/packs(.*)rewrite remains until the served Console is deployed and verified on the native path. - Direct bearer. A JWT is validated against
Authority:ResourceServer(issuer + JWKS).
Authority:ResourceServer:Authority is required: outside Development/Testing the host aborts at startup without it (a missing issuer would otherwise surface as a 500 on every request, including anonymous /healthz, because UseAuthentication() initializes handlers before any endpoint runs). Dev/Testing fall back to the local-harness issuer; authorization is not relaxed.
Authorization. Every /api/v1 endpoint is gated by RequireAuthorization with a named policy that binds the canonical scopes from src/Authority/StellaOps.Authority/StellaOps.Auth.Abstractions/StellaOpsScopes.cs:
Policy (PacksRegistryPolicies) | Scope requirement | Endpoints |
|---|---|---|
packsregistry.read | packs.read or packs.write | all reads (write implies read so a publisher can read back what it wrote) |
packsregistry.write | packs.write | every mutation, including the sensitive ones below |
/api/v1/buildinfo is the one deliberate anonymous endpoint (platform-wide image provenance for the operator verify aggregator), alongside /healthz and the static OpenAPI stubs. A test (PacksRegistryAuthorizationTests.EveryApiEndpoint_CarriesAPacksRegistryAuthorizationPolicy) fails the build if any other endpoint ships ungated.
Tenancy. The data tenant comes exclusively from the validated stellaops:tenant claim (UseStellaOpsTenantMiddleware + TryRequireTenant). A body/query tenant that disagrees with the claim is rejected with 403 tenant_claim_mismatch; a request with no tenant claim is rejected with 400 tenant_missing and never defaulted. Pack metadata, content, provenance and manifest reads use (tenant_id, pack_id) as the lookup key. A caller cannot probe a pack globally: when that key is not visible in the claim tenant, the read returns the same non-enumerating 404 whether the pack id is absent everywhere or belongs to another tenant. The installation allow-list can still reject an otherwise authenticated tenant with 403; write payload/query tenant mismatches remain 403.
Not authorization. The optional shared PacksRegistry:Auth:ApiKey (X-API-Key) carries no identity, tenant, or scope, and is skipped entirely when unset. It is a supplementary pre-shared secret only; it can never grant access on its own. PacksRegistry:Auth:AllowedTenants is likewise a coarse allow-list, not an authorization control.
Bypass networks are deliberately NOT configured for this service.
StellaOpsBypassEvaluatorgrants a full authorization bypass to any request from a listed CIDR that carries noAuthorizationheader — and gateway-proxied requests are precisely that shape. Listing the compose subnet would bypass the scope requirement on every gateway call and silently undo the scope gate. PacksRegistry has no in-cluster S2S HTTP caller that needs one.
API Surface
All routes are under /api/v1 (StellaOps.PacksRegistry.WebService/Program.cs).
| Method | Route | Policy |
|---|---|---|
| POST | /packs | write |
| POST | /packs/stream | write — bounded descriptor plus streamed content/provenance |
| GET | /packs | read |
| GET | /packs/{packId} | read |
| GET | /packs/{packId}/content | read |
| GET | /packs/{packId}/provenance | read |
| GET | /packs/{packId}/manifest | read |
| POST | /packs/{packId}/signature | write — signature rotation |
| POST | /packs/re-envelope | write — bulk re-envelope; accepts an operator private key in the body |
| POST | /packs/{packId}/attestations | write |
| POST | /packs/{packId}/attestations/stream | write — bounded descriptor plus streamed attestation |
| GET | /packs/{packId}/attestations | read |
| GET | /packs/{packId}/attestations/{type} | read |
| GET | /packs/{packId}/parity | read |
| POST | /packs/{packId}/parity | write |
| GET | /packs/{packId}/lifecycle | read |
| POST | /packs/{packId}/lifecycle | write |
| POST | /export/offline-seed | write — full-tenant content export |
| POST | /mirrors | write — mirror configuration |
| GET | /mirrors | read |
| POST | /mirrors/{id}/sync | write |
| GET | /compliance/summary | read |
Non-/api/v1 surface: /healthz (anonymous) and the generated /openapi/v1.json contract. The historical /openapi/packs.json and /openapi/pack-manifest.json URLs remain as temporary redirect aliases to that one generated authority; their old static files are not served because they predate claim-only tenancy, origin/generation and bounded streaming/paging. There is no approval endpoint — lifecycle state set/get is the nearest surface — and no dedicated version-listing endpoint; GET /packs lists packs. See also api-reference.md and openapi/.
The existing JSON/base64 POST contracts remain supported. Their encoded request bodies are capped before minimal-API model binding (16 MiB decoded content and, for a pack, 16 MiB decoded provenance); an over-limit request returns typed 413 before any repository reservation. Large artifacts use the vendor binary routes. Every other JSON POST is also bounded before model binding: ordinary lifecycle, parity, signature, mirror, sync and export controls use a small cap derived from their central UTF-8 field limits, while re-envelope has a separate finite envelope that includes two 32 KiB PEM transports. Both declared Content-Length and chunked bodies enforce the same boundary. The binary body starts with a four-byte unsigned big-endian JSON descriptor length, then that exact bounded UTF-8 descriptor, then the exact content bytes and (for a pack when declared) exact provenance bytes. The pack descriptor carries name, version, canonical digest, content length, optional signature/provenance URI, optional provenance digest and length, and metadata; the attestation descriptor carries type, canonical digest, content length and notes. HTTP Content-Length is a required header in the generated contract and must equal the prefix, descriptor and declared segment lengths. Short, trailing, digest-mismatched, duplicate-field, unknown-field or malformed descriptors fail before a ready row, durable blob, storage event, outbox item or audit fact can publish. The descriptor limit is derived from the central UTF-8 field budgets and worst-case JSON escaping; it does not silently narrow the existing Unicode or metadata contract. The vendor media types are application/vnd.stellaops.pack.v1+octet-stream and application/vnd.stellaops.attestation.v1+octet-stream; other media types receive 415.
Compose Contract
Implemented in source, not deployed (owner ruling 2026-08-19; JOB-11 tree slice 2026-08-20). PacksRegistry supports seed-delivered and upload-capable packs at the same time. JOB-11 now supplies the tenant-keyed catalog, immutable reader, durable atomic writer, forward migrations plus in-process composition and isolated acceptance harnesses in the source tree. Platform’s active/retiring verification-trust producer has landed and PacksRegistry consumes its exact trust-set/epoch wire contract (with a deterministic legacy-scalar compatibility trust set when no Platform source is configured). JOB-11’s focused Windows, privileged-Linux, isolated-publish and real API/restart/export matrix is complete at source commit
a83cb2f6f4(integrated on main as8f5eb3d78c); re-verify with the exact targeted commands inSPRINT_20260722_012JOB-11’s 2026-08-25 Execution Log. JOB-12 stages the current compose shape and rollback package but performs no activation or isolated rehearsal. The standing host remains on its previously deployed image until an attended window, and JOB-13 remains blocked.
The two storage roles are physically separate and may never resolve to the same path or to ancestor/descendant paths:
| Role | Container path | Mutability | Authority |
|---|---|---|---|
| operator/offline seed | /app/etc/plugins/packsregistry/packs | immutable, mounted :ro | signed seed manifest + mounted bytes |
| API uploads | /var/lib/stellaops/packsregistry/uploads | durable writable named volume | PacksRegistry API after scope, tenant, digest and signature validation |
Current packsregistry-web consumes the PacksRegistry:Storage:ObjectStore:* contract. The born-new jobengine-web must adopt that same contract atomically with JOB-9’s PacksRegistry domain wiring; its current health/doctor skeleton does not advertise ignored storage settings. Driver=dual-fs requires distinct SeedFs:RootPath and UploadFs:RootPath; the existing Driver=seed-fs is a compatibility seed-only posture and must return a typed 503 packs_upload_store_unavailable for blob-writing mutations rather than attempting to write the seed path. Metadata-only lifecycle reads/transitions remain available. It is not the destination posture. Canonical-path and reparse/symlink checks fail startup if either root escapes its declared root or the roots overlap. Production dual-store is supported only for local Linux filesystems for which startup can establish architecture-stable statx directory identity plus /proc/self/mountinfo backing-path and mount-mode proof. Remote NFS, FUSE, CIFS/SMB and other network-backed exports fail closed until a stable remote identity and immutability proof is implemented; path-bearing mount metadata alone is not sufficient. Windows remains a Development/Testing ordering seam and cannot start a Production writable dual store. Never mount one path read-only in one compose layer and writable in another. Production also proves from /proc/self/mountinfo that the configured signed SeedFs authority itself is mounted read-only; opening files with read access is not a substitute for an immutable mount. Development/Testing keeps an explicit local-harness escape.
The seed root is populated by an Offline Kit or operator-delivered content set. A configured root must contain a signed seed-manifest.json with a generation id, tenant, canonical pack identities, payload/provenance/attestation digests, signature material and declared entry count. Reconciliation sorts entries ordinally, verifies path containment, declared count, every digest and every required signature before changing the active catalog. The configured manifest byte ceiling may not exceed the process-safe 64 MiB hard maximum; a bounded UTF-8 token prepass enforces the per-pack 64-entry, key/value and 64 KiB metadata contracts before typed graph deserialization. A missing manifest, an undeclared file, a count mismatch, an invalid signature, or an empty generation without an explicit allowEmpty declaration quarantines that generation and makes readiness fail; zero files must never be reported as a successful import by accident.
Do not mount PacksRegistry content under /app/plugins. That path is reserved for signed executable plugin bundles owned by services with executable plugin contracts.
Identity, collisions and source selection
- Logical identity is
(tenant_id, pack_id), where one canonicalizer shared by API ingest, seed reconciliation, persistence and export derivespack_idfrom normalized name + version. Tenant comes only from the validatedstellaops:tenantclaim. JOB-11 repository reads take tenant as part of the key and the forward migrations remove the globalpack_idunique index; an older deployed image is not evidence that this tree contract is live. - Blob identity is the verified SHA-256 digest and kind (
content,provenance, or the genericattestationkind), under the owning tenant prefix. Attestation type remains catalog identity and never becomes a path component. User text is never converted into a filesystem path by the current OS-dependentNormalizeSegmentroutine. - The same canonical identity + the same content/provenance digests is an idempotent duplicate. A ready
uploadorseed+uploadretry is read-only: caller-owned metadata, signature, provenance, timestamps and audit facts cannot be replaced. A readyseedplus a verified identical upload is the one monotonic exception. The reservation records a durable operation claim while the seed row remains visible; finalization compare-and-swaps onlyseedtoseed+upload, clears that exact claim, preserves the seed generation and every existing metadata field, and appends exactly one origin-expansion audit/outbox fact. Concurrent callers and retries observe the immutable result. Seed reconciliation applies the symmetric verifieduploadtoseed+uploadunion while preserving upload metadata. The origin remains visible in list, manifest, audit and offline export output.seed+uploadreads the immutable seed copy first and uses the upload copy only when the seed artifact is absent, after verifying the same digest. - The same canonical identity + a different digest is supply-chain drift, not an overwrite rule. An API request receives
409 pack_identity_digest_conflict; a seed generation is quarantined and readiness fails. Neither source silently shadows the other, regardless of arrival order. - Different tenants may use the same
pack_idwithout collision. Within one tenant, an attestation key is(pack_id, type, digest); a different digest for the same(pack_id, type)is a new evidence revision, not an unconditional update.
Seed bytes, provenance and seed-supplied signature envelopes are immutable. Upload, signature rotation, re-envelope, lifecycle, attestation, or future delete operations must never call a write or delete method on the seed store. Re-envelope records a new operator envelope/history entry; upload attestations go to the upload store. Removing an upload reference can garbage-collect only an unreferenced upload blob after the database proves it is unreachable; it cannot remove a seed origin or mounted byte. If an active seed generation disappears from the mount, startup/readiness quarantines the catalog rather than silently tombstoning its rows.
Seed-generation evidence is append-only. Every reconciliation attempt records its manifest digest and disposition, exactly one singleton row names the active generation, and an already-applied generation is fully revalidated against the manifest, catalog and mounted bytes on every restart. Reusing a generation id with different bytes records a new quarantined attempt without overwriting the original applied marker. An atomic G1 to G2 switch handles every G1-bound omission in the same transaction: a seed-only row is retained as quarantined and cannot be listed or read; a seed+upload row may become upload only after the upload copy and digest are reverified, with the old seed binding cleared. No omitted row remains ready against bytes that are no longer mounted.
Durable signature floor and stale-replica fencing
Signature verification produces three pieces of mutation evidence: canonical region profile, accepting key id and the Platform trust epoch observed with that verification. Before a tenant’s re-envelope sweep starts, PacksRegistry persists a floor keyed by exactly (tenant_id, region_profile). The active key id is the monotonic security identity. The trust epoch is not ordered and is not an equality fence: Platform can publish a new epoch for the same active key after a revocation or trust-set observation. Every distinct (tenant, profile, active key, epoch) observation is therefore retained append-only in pack_signature_floor_epochs, while the current floor row advances to the latest observed epoch. Floor rows and mutation comparisons do not cross-couple profiles. The re-envelope fixed-point gate below is deliberately tenant-wide because every profile sweep mutates the same tenant catalog.
An A-to-B key transition is authorized only when the coherent publication that accepted B lists the currently floored A as retiring/superseded. Once B is the floor, a stale replica still holding A cannot roll the floor back: its stale publication does not identify current B as retiring, so the attempt receives 409 pack_signature_floor_regression. The floor is installed under the global storage-admission lock before the first sweep page, rechecked at the final fixed point, retained after completion and advanced atomically on the next rotation. It is never cleared as a cleanup or checkpoint action.
The same admission lock applies the floor to every API upload, signature rotation, re-envelope write and seed-origin expansion/publication. The verified evidence is preserved through the reservation. Once a floor exists, a new or changed row must present the exact floored profile and active key id; a stale key receives 409 pack_signature_floor_mismatch before a catalog claim, capacity reservation or blob write. Pack-id ordering is irrelevant, so a later replica cannot use a new lower-id row to bypass the fence. An existing same-generation seed marker may still perform its full restart parity revalidation against the live active+retiring trust set during grace, but a changed/new generation row for the fenced tenant requires the exact active floor key.
Staging and seed-expansion claims persist profile, accepting_key_id and trust_epoch alongside the operation id. Revival must match the stored profile and accepting key; epoch differences are allowed only as append-only provenance for that same security identity. A different key/profile receives 409 pack_upload_claim_verification_mismatch. Floor advancement and re-envelope completion also refuse with 409 re_envelope_claims_active while a tenant has an in-flight staging or seed-expansion claim. This prevents one replica from changing the trust boundary around another replica’s partially published mutation; exact reconciliation must complete or clean the claim first.
Persistence
PacksRegistry supports the same durability posture as the rest of Stella Ops: startup must converge the module schema, and pack content must survive service restart. File-system repository tests exist for deterministic local behavior, and Postgres/object-store drivers are selected through service configuration.
JOB-11 replaces the former ambiguous IPacksRegistryBlobStore seam in the PostgreSQL composition with IImmutablePacksRegistrySeedReader and IPacksRegistryUploadStore. Pack and attestation repositories now reserve tenant-keyed staging rows, write content-addressed upload bytes through the atomic writer, then make the row ready with storage-event/outbox facts in one transaction. Reads are bound to the catalog origin and never fall back to inline PostgreSQL bytes after the legacy reconciler clears them. The local filesystem repository remains a Development/Testing harness; it is not the production dual-store implementation.
Pre-JOB external-store upgrade gate
The pre-JOB binary used SeedFs:RootPath as a mutable external payload store and wrote the old packs/<NormalizeSegment tenant>/<NormalizeSegment pack>/... layout while PostgreSQL BYTEA columns were placeholders. JOB-11 does not reinterpret that path as the new signed seed. An upgrading estate must map the historical path explicitly to LegacySeedFs:RootPath; the new immutable signed SeedFs:RootPath and writable UploadFs:RootPath remain distinct, non-overlapping authorities.
This is intentionally a non-rolling maintenance upgrade. Operators stop and drain every old writer and present LegacySeedFs read-only. Before 002 changes any domain DDL, the application performs a bounded read-only inventory of every legacy row and the complete old tree, rejects path aliases, links/special files, unmatched evidence, missing or mismatched pack/provenance/attestation bytes, noncanonical/unbounded caller text, invalid signatures and capacity violations, and verifies the current trust policy. Once that zero-mutation gate is green it installs a durable, versioned transition journal and database mutation fence, stages every claimed artifact idempotently into UploadFs, and appends the final installation marker last. A crash in this phase remains an explicit non-serving transition-in-progress state and resumes from the immutable journal; it is not silently compensated. The migration validates the exact marker as its first legacy-lineage guard, uses a private transaction setting, applies the schema and removes the fence in the same transaction. Platform-first migration discovery therefore fails closed when the marker is absent. Rejected preflight leaves rows, xmin, indexes/constraints, migration ledger and the legacy tree unchanged. Owner-approved audited repair or PostgreSQL/filesystem snapshot recovery is required for invalid historical data; truncation, normalization, destructive orphan cleanup and grandfathered unreachable identities are forbidden.
Any exact UUID-valued metadata-only row shape synthesized by the applied 001 baseline (legacy digest, zero-byte placeholder, and no addressable signature/provenance) is preserved as upgrade evidence but classified quarantined; no particular UUID is fabricated or treated as the sentinel. It is excluded from ready list/get/content/export and no canonical alias or blob is fabricated for it. Other legacy-inline rows remain subject to literal SHA-256 verification and are not covered by that narrow sentinel rule.
The stable API/export origin strings are legacy-inline, seed, upload, and seed+upload. List, pack/attestation responses, manifests, catalog export and storage audit facts use those exact values; seed-backed rows also expose their generation id.
Upload commit order is binding:
- Validate the claim tenant,
packs.write, request shape, canonical identity, logical catalog budgets, content digests and production signature policy before reserving anything. - Under the shared storage-admission advisory lock, claim
(tenant_id, pack_id, expected digests)asstagingunder a unique key and content-guarded statement. An ordinary already-readyidentical claim returns the immutable existing result; a different digest conflicts. A ready seed-origin claim instead records an operation id and server timestamp without hiding the row. Pack and attestation reservations use the same lock, so cleanup cannot race a new reference. - After the durable claim exists, take a short PostgreSQL capacity transaction, subtract every active crash-surviving physical reservation from the writable mount’s observed free bytes, and persist this operation’s absent-target byte reservation. Then stream upload blobs through the fixed-shard
.control/upload-temps/v1/<shard>namespace on the same volume, flush file data and its source parent, atomically rename with create-new semantics, and durably flush destination and source parents. Existing final bytes are verified before free-space admission, so an idempotent retry cannot fail507merely because the already-written blob filled the volume. Upload-store construction is side-effect-free so a rejected pre-DDL catalog cannot mutate UploadFs; fixed control directories and the deterministic writable probe are initialized only after the green transition fence or on the first ordinary storage operation. - In one final PostgreSQL transaction, change the row to
ready, bind each artifact to its recorded origin, preserve the ordinary upload operation as its idempotency key (or clear the exact ready/seed expansion claim), and append the storage event, outbox entry and durable mutation audit. List/get/export read onlyreadyrows. A later best-effort audit emitter is not the durability boundary. A crash before step 3 leaves a resumable claim; a crash after step 3 leaves an immutable blob discoverable through that same claim, including a still-visible ready/seed origin expansion. The final transaction also removes the operation’s physical reservation.
Stale-claim reconciliation uses the database-owned server timestamp and configured grace period, not a caller-supplied orphan assertion. Under the same admission lock it row-locks the exact operation, proves no staging, ready upload, seed+upload, or other pending seed-expansion reference uses each digest, and calls the store’s typed age-checked delete before removing that exact claim. An expired ready/seed expansion clears only its claim and stays visible as seed. If an atomic seed generation replacement quarantines that row while its expansion is still pending, the same age/reference proof removes only the abandoned upload artifacts and clears the claim while retaining the quarantined seed-omission row as evidence. An abandoned staging row is quarantined until its artifacts are reconciled and then removed so identity and its durable capacity reservation are released in the same transaction. Exact-claim cleanup uses the operation’s striped filesystem lease; anonymous temp cleanup scans only bounded, fair batches from the fixed temp namespace and takes the separate global writer fence for the delete batch, never recursively walking final artifacts under exclusion. The claim-bound typed delete covers both the final content address and address-hashed temp files in that central namespace. Missing artifacts are an idempotent success, too-recent artifacts retain the claim, and a completed seed+upload final artifact is never collected.
Seed reconciliation validates the whole mounted generation first, then takes one database advisory lock and one transaction to insert content-guarded origin rows and the generation-complete marker. The marker becomes active only after every declared entry is present and every prior-generation omission has been quarantined or safely downgraded as described above. Any identity/digest conflict rolls back the catalog transaction and appends a quarantined attempt; partial generations are never served. A same-marker restart performs the complete parity check again rather than short-circuiting.
Schema evolution is forward-only. The live PacksRegistry lineage receives an additive migration, and the born-new StellaOps.JobEngine.Persistence lineage receives the equivalent new migration after its existing 005 migration; the already-proven consolidated baseline is not rewritten. Existing inline payload rows are classified legacy-inline and moved to the upload store only by a digest-verified, resumable reconciliation before inline bytes are cleared; the exact baseline metadata sentinel is retained quarantined as described above. No migration may add sample, demo, QA or seed-pack content (§2.11); seed data enters through the signed mounted source, and lab data enters through the product API.
Capacity, health and failure semantics
The writable store declares per-pack bytes, per-tenant bytes/objects, total bytes/objects and minimum-free-space limits. Logical catalog budgets are checked under the storage-admission lock and the durable staging/seed-expansion claim is committed first. A short, bounded PostgreSQL capacity transaction then records only absent-target physical bytes against the full canonical upload mount; no global filesystem capacity handle spans streaming I/O. Finalization removes the reservation with the catalog/audit transaction, and exact stale-claim cleanup reclaims it after typed artifact cleanup. A full store returns 507 packs_upload_capacity_exhausted; filesystem permission/I/O faults are storage failures (503/500 as appropriate), never 400, 401, 403 or an authentication failure class.
JOB-11 startup fails closed for invalid/overlapping roots, an unavailable writable volume in dual-fs, a bad seed manifest/generation, migration failure, or reconciliation conflict. Its mutation and read paths expose typed 409, 503, and 507 storage errors. The richer readiness surface described here — active-generation parity, non-mutating upload capability, reservation backlog and capacity high-water checks, including explicit degraded-read reporting — remains JOB-12 criterion 2 and must not be inferred from the existing process-only /healthz.
The legacy metadata driver Storage:Driver=filesystem is a local Development/Testing harness and startup rejects it in Production. Production uses PostgreSQL plus signed seed reconciliation in supported seed-fs (read-only) or dual-fs mode; writable dual mode additionally requires the atomic upload store and Linux durability/root-independence proofs. It cannot bypass staging, transactional facts, tenant capacity, or typed storage failures through the file adapter.
Retention, ownership and offline semantics
P13 classes remain: pack catalog/lifecycle/mirror configuration and seed-generation bindings are operational; seed attempts, storage events, signature history, attestations and audit are immutable, append-only evidence; parity is derived/rebuildable. JOB-11 applies bounded UTF-8/count contracts and keyset/streamed reads, but it defines no evidence-deletion duration because no owner-approved retention horizon exists. Partition/window execution and its recovery evidence are explicit downstream JOB-12/operations debt; no implementation may infer a TTL or prune evidence meanwhile. The outbox is operational pending/idempotent state whose delivery and published-row disposition is owned by the JOB-12 dispatcher. Artifact mutations are content-guarded and cannot reassign tenant or immutable payload identity; narrowly-scoped lifecycle, mirror, parity and idempotency upserts remain explicit. Lifecycle, parity and mirror repositories compare semantic content without generated timestamps: an exact retry returns the retained durable row, performs no physical update, and emits no duplicate transition audit; a changed value advances both state and evidence. No current applicable pack is removed by age. PostgreSQL stores metadata/state only after legacy-inline reconciliation; full payload copies live once in their selected content store.
Under P16 the volume/prefix owner is packsregistry until consolidation and jobengine after the JOB-9 family cutover; the physical upload volume is carried, not recreated. The seed mount is an installation input, not a tenant-owned database and not a writable object-store prefix. Both stores still partition keys by the claim-derived tenant and never grant cross-tenant enumeration.
POST /api/v1/export/offline-seed remains tenant-scoped. JOB-11 exports ready packs from both origins in canonical identity order with catalog origin/generation metadata, digests, verified content, provenance and exact attestation revisions; when a complete byte set is requested it also emits catalog-manifest.json and, when an export signing key is configured, a DSSE envelope over that manifest’s SHA-256 as the sibling root entry catalog-manifest.json.dsse.json (F-JOB13-1; key supply, envelope shape and verification in docs/security/pack-signing-and-rbac.md §7.1). The consumer-side verifier ships in product as the closed, BCL-only SDK StellaOps.PacksRegistry.OfflineSeed.Contracts (entry names, payload type, schema pin, the shared base64 rule, trust roots from a public-key PEM or a verification-trust-set/v1 document) behind stella verify offline-seed; the producer takes its wire identifiers from the same SDK (SPRINT_20260914_003 JEF-5). There is still no server-side import of this archive: seed-fs ingest reads seed-manifest.json, a different artifact by design. With no key the manifest ships unsigned and the missing envelope says so. It deliberately does not emit seed-manifest.json: signing a generation and proving symmetric, byte/digest-stable re-import are JOB-12, not claims of the JOB-11 slice. Offline operation never weakens signature validation: production seed ingest uses the configured trust root; only the explicit Development/Testing harness may accept the integrity-only unsigned path. ADR-028 active/retiring overlap and post-revocation consumption now uses the landed Platform trust-set producer (fe8efd293945…, with the later duplicate-SPKI producer guard) and records the manifest-accepting key plus exact trust epoch in append-only generation-attempt evidence. Execution of the fresh-process acceptance matrix, rather than an absent producer seam, is the remaining blocker.
Production preparation and ZIP streaming use one tenant-scoped, read-only REPEATABLE READ PostgreSQL transaction for every catalog family and inline payload. External payload addresses are content-addressed and digest-validating. The archive is therefore wholly one snapshot even while uploads, rotations or audit writes continue; it never combines independently timed count/list/read passes. Preparation verifies the artifact-entry bound before response bytes and records a durable attempt fact. Completion/cancellation facts use a short host-lifetime token and cannot relabel a fully written archive when only the terminal audit append fails. The configured upload-object plus seed-manifest-entry ceilings, plus six fixed catalog entries, must fit the explicit 192 MiB ZIP central-directory budget (262,144 entries); invalid configurations fail at startup. The default 100,000 + 100,000 limits remain fully exportable without an imaginary caller-side split operation.
The File and InMemory adapters are Development/Testing compatibility stores, not the Production export authority. Their fixed-point snapshot deliberately caps each retained catalog family at 4,096 records. Audit itself remains append-only and may exceed that local export ceiling; the 4,097th audit record makes local export fail with typed 413 without truncating or deleting any evidence. Lifting that harness-only limit requires a real continuation or durable high-water contract. Likewise, IAuditRepository.ListPageAsync is a finite page view rather than a concurrent durable tail. PostgreSQL may allocate a lower audit sequence in an uncommitted transaction while a higher sequence becomes visible; the Production export is correct because all of its pages share one repeatable-read snapshot. No live tail consumer is authorized to infer exactly-once delivery from the page seam; such a tail belongs to the downstream audit owner.
Current-host rollout adds the read-only seed mount and a separate retained upload named volume in one compose change. The born-new jobengine-web later takes the same two mounts and keys while its metadata connection changes to STELLAOPS_POSTGRES_JOBENGINE_CONNECTION; JOB-9 must not copy blob bytes or create a fresh upload volume. Rollback retains the upload volume and database rows. Before the first dual-capable image exists, rollback may restore the old seed-read-only image (new uploads become temporarily unavailable, never deleted); after activation, rollback targets the last known green dual-capable image. A rollback never remounts the seed path writable.
Test Expectations
Focused tests should target src/JobEngine/StellaOps.PacksRegistry/StellaOps.PacksRegistry.Tests/StellaOps.PacksRegistry.Tests.csproj.
Required behavioral coverage:
- upload and retrieve pack metadata/content
- isolated real-host
POST /api/v1/packsproof against a writable upload volume, followed by restart/read/export proof; no live estate is used - seed store rejects every mutation; upload store cannot resolve inside the seed root
- seed-manifest non-vacuity, digest/signature verification, deterministic reconciliation and quarantine on identity drift; real RSA non-empty seed acceptance plus independent manifest- and pack-signature tamper refusal
- identical cross-source ingest is idempotent; conflicting digest is 409/quarantined; concurrent identical requests preserve caller metadata and create one ready row/blob/audit event; only the verified monotonic
seedtoseed+uploadorigin union may mutate an existing ready row - crash recovery at every blob/metadata boundary, including ready/seed origin expansion; stale cleanup uses the shared admission lock, database age/ownership and reference proof
- G1 to G2 omissions, same-generation restart parity, generation-id reuse evidence and the exact applied-baseline sentinel quarantine
- origin/generation visible and stable in list/manifest/offline export
- capacity/readiness/error classification, including existing-blob retry before capacity, 507, writable-root failure and filesystem faults never mapped to auth
- signature verifier accept/reject paths
- lifecycle and parity transitions
- attestation upload/retrieval
- offline seed export
- mirror sync status
- confirmation that pack import does not execute pack-provided code
- scope authorization: an unauthenticated caller is refused (401) and a
packs.read-only caller is refused (403) on signature rotation, re-envelope, mirror config, offline export, and upload (PacksRegistryAuthorizationTests)
