Stella Ops JobEngine

The JobEngine module is the consolidated orchestration domain for two subdomains:

• Scheduler — scheduled scan orchestration. It re-evaluates already-cataloged images when intelligence changes, runs nightly and ad-hoc rescans targeting only impacted images via the ImpactIndex, and drives graph jobs, policy runs/simulations, and vulnerability-resolver jobs.
• PacksRegistry — task-pack registry management. It stores, versions, and serves task-pack bundles.

Scope note (2026-05-30 reconciliation): The generic “Source & Job Orchestrator” control plane — sources, quotas, throttles, incidents, circuit breakers, DAG dashboards, lease/claim Task Runner bridge, and live deployment runs — is not this module. That surface is owned by the separate src/ReleaseOrchestrator/ module (docs/modules/release-orchestrator/). The legacy StellaOps.JobEngine.* libraries (Core/Infrastructure/Worker/Tests) and the standalone TaskRunner were removed in April 2026; Scheduler and PacksRegistry are the active subdomains. See docs/modules/jobengine/architecture.md for the deeper module architecture. Note that the architecture dossier still opens with legacy “Source & Job Orchestrator” framing (§1–3 describe the removed orchestrator schema and orch:* lease/replay surface); the code-verified JobEngine compatibility surface and the Release Orchestrator route translations live in §4 onward.

Responsibilities

• Schedule and re-run scan/analysis jobs (CRON plus Concelier/Excititor/manual triggers), targeting only impacted images via the ImpactIndex; emit report-ready events for Notify.
• Drive graph build/overlay jobs, policy runs and policy simulations, and vulnerability resolver jobs.
• Provide a read-only JobEngine compatibility surface (/api/v1/jobengine/jobs, /dag) so Console job views resolve to durable Scheduler Run state.
• Store, version, approve, and serve task-pack bundles (PacksRegistry).
• Stream run progress over SSE and emit structured audit to the Timeline unified audit sink.

Key components

• Scheduler WebService (StellaOps.Scheduler.WebService) — HTTP API plus embedded worker background services by default (Scheduler:Worker:Embedded=true).
• Scheduler Worker.Host (StellaOps.Scheduler.Worker.Host) — optional standalone worker for scale-out (Embedded=false).
• PacksRegistry WebService + Worker (StellaOps.PacksRegistry.*).
• Queue transport (StellaOps.Scheduler.Queue) — pluggable Valkey/Redis Streams or NATS JetStream; PacksRegistry blobs use the seed-fs object store.
• Job plugins (ISchedulerJobPlugin) — built-ins registered unconditionally in Program.cs: Scan (scan, ScanJobPlugin) and Doctor (doctor, SchedulerDoctorJobPlugin), both host-owned in StellaOps.Scheduler.WebService. Additional plugins are discovered from StellaOps.Scheduler.Plugin.* DLLs via PluginHost.LoadPlugins; production optional mounted plugins include StellaOps.Scheduler.Plugin.AdvisoryAI (advisory-ai-analysis, profile scheduler-ai) and StellaOps.Scheduler.Plugin.FeedProcessor (feed-processor, profile scheduler-feed). The shipped StellaOps.Scheduler.Plugin.AuditCleanse (audit-cleanse) is an assembly-loaded placeholder plugin, not a built-in.
• Worker SDKs — StellaOps.JobEngine.WorkerSdk.Go and StellaOps.JobEngine.WorkerSdk.Python.

Storage

• Scheduler: PostgreSQL schema scheduler (SchedulerDbContext); auto-migrates embedded SQL on startup. Persisted resolver-job state lives in scheduler.jobs; audit flows to timeline.unified_audit_events.
• PacksRegistry: PostgreSQL schema packs (metadata/state) + seed-fs object store (blob payloads).
• There is no orchestrator schema in this module.

Integrations & dependencies

• Authority for authN/Z. Scheduler authorization policies map to three scopes: scheduler:read (read paths), scheduler:operate (mutating/job-trigger paths), and scheduler:admin. The JobEngine compatibility surface (/api/v1/jobengine/*) accepts either scheduler:read or orch:read. The service also adds graph:read and graph:write to its required token scopes and uses the api://scheduler audience by default (Program.cs, StellaOpsScopes).
• Scheduler’s by-CVE ImpactIndex projection calls Concelier GET /advisories/observations?alias=... through the embedded worker. Concelier resolves tenancy from the validated stellaops:tenant claim, so compose wires the dedicated stellaops-scheduler-concelier-observations client-credentials service account with only vuln:view; the resolver passes the run tenant as the token tenant parameter and still emits X-StellaOps-TenantId only for compatibility.
• PacksRegistry WebService is an Authority resource server. Every /api/v1 endpoint requires either the packsregistry.read policy (packs.read or packs.write) or the packsregistry.write policy (packs.write). The optional configured X-API-Key is a supplementary gate after scope authorization, not an authentication substitute; tenancy is resolved from the validated stellaops:tenant claim.
• ImpactIndex / BOM-Index for impacted-image selection; Scanner, Policy, and Cartographer for job execution.
• Notify event queue for worker/exception notifications; Timeline for unified audit.
• Offline Kit for state export/import during air-gap refreshes.

Service endpoints (gateway)

• Scheduler: https://scheduler.stella-ops.local (slot 19, port 8080) — env STELLAOPS_SCHEDULER_URL.
• PacksRegistry: https://packsregistry.stella-ops.local (slot 34, port 8080) — env STELLAOPS_PACKSREGISTRY_URL.

Primary Scheduler API surface (code-verified)

• GET/POST /api/v1/scheduler/schedules + /{id} (GET/PATCH/DELETE) and /{id}/pause, /{id}/resume — schedule CRUD and lifecycle.
• GET/POST /api/v1/scheduler/runs, /{runId} (GET), /{runId}/deltas, /{runId}/stream (SSE), /{runId}/cancel, /{runId}/retry, /preview, /queue/lag.
• GET/POST /api/v1/scheduler/policy/runs + /{runId} (GET).
• GET/POST /api/v1/scheduler/policies/simulations + /{id}, /{id}/stream (SSE), /preview, /{id}/cancel, /{id}/retry, /metrics.
• POST /api/v1/scheduler/vuln/resolver/jobs, GET /jobs/{jobId}, GET /metrics.
• GET /api/v1/scheduler/failure-signatures/best-match.
• Graph jobs: POST /graphs/build, POST /graphs/overlays, GET /graphs/jobs, POST /graphs/hooks/completed, GET /graphs/overlays/lag.
• Inbound webhooks (Concelier/Excititor export notifications): POST /events/conselier-export, POST /events/excitor-export (HMAC-SHA256). The route literals use the legacy conselier/excitor spelling as defined in source; do not “correct” them.
• JobEngine compatibility (Console job views): GET /api/v1/jobengine/jobs, /{jobId}, /{jobId}/detail, and GET /api/v1/jobengine/dag/job/{jobId}/parents + /children.
• Health: GET /healthz, GET /readyz (anonymous). PacksRegistry surface is rooted at /api/v1/packs (plus /api/v1/export/offline-seed, /api/v1/mirrors, /api/v1/compliance/summary).
• Note: many /api/v1/jobengine/* gateway paths (runs, quotas, deadletter, pack-runs, stream, audit, sources, slos) and all /api/v1/release-orchestrator/* paths are translated to the Release Orchestrator service, not to this module. See architecture.md §4 (Release Orchestrator compatibility routes).

Operational notes

• Authority authentication is mandatory outside Testing/local-harness mode; the service fails fast on insecure configuration. Header-based auth is restricted to Testing/Development local harness.
• Clean default startup auto-migrates the scheduler schema without inserting demo triggers/jobs/ runs/workers; demo seed migrations are opt-in via SCHEDULER_BOOTSTRAP_ENABLED=true.
• Inbound Concelier/Excititor export webhooks authenticate via HMAC-SHA256 and are rate-limited via a Redis-backed limiter; startup fails fast if enabled without queue configuration.
• Scheduler:Worker:Concelier controls the CVE -> affected-PURL projection source. BaseAddress enables the Concelier observations client; AuthScope defaults to vuln:view; and Authority:* enables tenant-scoped client-credentials bearer minting for deployments where Concelier rejects header-only requests.

Related documents

• architecture.md— module architecture and API surface (§4 onward is the code-verified JobEngine/Release-Orchestrator surface; §1–3 retain legacy orchestrator framing).
• event-envelope.md— job/run event-envelope draft.
• job-export-contract.md— job/run export payload for Findings Ledger.
• Release Orchestrator — owner of the generic source/job orchestration control plane (see the scope note above).