Unknowns Registry

Status: Implemented (Grey Queue API partially wired — see Implementation Status) Source: src/Unknowns/ Container / Router service: stellaops-unknowns-web (router service name unknowns, port 8080) Owner: Signals Guild, Policy Guild

Purpose

The Unknowns Registry tracks ambiguities discovered during vulnerability scanning that cannot be definitively resolved — packages without known provenance, binaries without build-IDs/debug symbols, dependencies not covered by any advisory feed, unbounded version ranges, unsupported formats, and gaps in transitive dependency chains. It models “what we don’t know” as first-class, queryable risk signals using bitemporal semantics, then prioritizes them with a composite triage score and surfaces provenance hints to help operators resolve each unknown.

It is a standalone HTTP microservice (not a Policy Engine sub-library): the Unknowns WebService exposes the registry over the Stella Router, persists to its own PostgreSQL schema, and auto-migrates on startup.

Quick Links

• Architecture
• API reference
• Grey Queue state machine

Components

Service:

• StellaOps.Unknowns.WebService — Minimal-API HTTP service (Program.cs) exposing the Unknowns and Grey Queue endpoints, OpenAPI/Swagger, health/readiness probes, Stella Router integration, scope-based authorization, tenant middleware, audit emission, and localization.

Libraries:

• StellaOps.Unknowns.Core — Domain models (Unknown, GreyQueueEntry), enums, repository interfaces (IUnknownRepository, IGreyQueueRepository), provenance-hint builders, the native-binary classifier, the unknown ranker, and the grey-queue state machine.
• StellaOps.Unknowns.Persistence — PostgreSQL/EF Core persistence: UnknownsDataSource, UnknownsDbContext, PostgresUnknownRepository, embedded SQL migrations, and AddStartupMigrations wiring.
• StellaOps.Unknowns.Services — Background/operational services: GreyQueueWatchdogService, UnknownsSlaMonitorService, UnknownsLifecycleService, UnknownsMetricsService, SlaCalculator, UnknownsSlaHealthCheck. Note: as of this writing these are referenced only by their own test project and are not referenced/hosted by the WebService (see Implementation Status).

Key Features

• First-class Unknown state: Explicit modeling of uncertainty (UnknownKind classification) rather than hiding gaps as false negatives.
• Bitemporal tracking: Valid-time (valid_from/valid_to) and system-time (sys_from/sys_to) columns, with point-in-time as_of queries and history retrieval.
• Triage scoring & bands: A composite score (popularity, exploit potential, uncertainty, graph centrality, staleness) assigns each unknown a TriageBand of Hot / Warm / Cold, driving a HOT processing queue.
• Provenance hints: Structured hypotheses with confidence levels and suggested actions, with a combined-confidence rollup and a high-confidence query.
• Native binary classification: Dedicated kinds for missing/unknown build-IDs, unresolved native libraries, heuristic (dlopen) dependencies, and unsupported binary formats.
• Grey Queue (evidence pipeline): A signed/replayable evidence-bundle queue (SBOM slice, advisory snippet, VEX evidence, diff traces, reachability evidence) with deterministic fingerprints, trigger conditions (feed/tool/CVE/PURL), a multi-state workflow, and SLA banding.
• Multi-tenant isolation: Row-Level Security on the unknowns.unknown table keyed by the app.tenant_id session variable.

API Surface

All endpoints are exposed via the Stella Router under the unknowns service. The tenant is supplied via the X-Tenant-Id header. Two named authorization policies gate access: Unknowns.Read (scope unknowns:read) and Unknowns.Write (scope unknowns:write).

Unknowns (/api/unknowns, requires unknowns:read)

• GET /api/unknowns — List with pagination; supports skip, take, asOf (bitemporal), kind, severity query filters.
• GET /api/unknowns/{id} — Get a single unknown (with hints).
• GET /api/unknowns/{id}/hints — Provenance hints only, plus best hypothesis / combined confidence / primary suggested action.
• GET /api/unknowns/{id}/history — Bitemporal history for the unknown.
• GET /api/unknowns/triage/{band} — Unknowns in a given triage band (Hot/Warm/Cold).
• GET /api/unknowns/hot-queue — HOT unknowns for immediate processing.
• GET /api/unknowns/high-confidence — Unknowns whose hints exceed minConfidence (default 0.7).
• GET /api/unknowns/summary — Counts by kind, severity, and triage band plus total open.

Grey Queue (/api/grey-queue)

Read endpoints require unknowns:read; mutating endpoints require unknowns:write and emit audit events (AuditModules.Unknowns).

• GET /api/grey-queue — List (filter by status / reason).
• GET /api/grey-queue/{id} — Get entry by ID.
• GET /api/grey-queue/by-unknown/{unknownId} — Get entry by unknown ID.
• GET /api/grey-queue/ready — Entries ready for processing.
• GET /api/grey-queue/triggers/feed/{feedId} | /triggers/tool/{toolId} | /triggers/cve/{cveId} — Entries matching a trigger.
• GET /api/grey-queue/summary — Summary statistics (counts per status, averages, oldest-pending hours).
• GET /api/grey-queue/{id}/transitions — Valid next states for an entry.
• POST /api/grey-queue — Enqueue a new entry (audit action enqueue).
• POST /api/grey-queue/{id}/process — Mark processing (start_processing).
• POST /api/grey-queue/{id}/result — Record a processing result (record_result).
• POST /api/grey-queue/{id}/resolve — Resolve (resolve).
• POST /api/grey-queue/{id}/dismiss — Dismiss (dismiss).
• POST /api/grey-queue/{id}/assign — Assign for review (assign; requires an assignee).
• POST /api/grey-queue/{id}/escalate — Escalate to the security team (escalate).
• POST /api/grey-queue/{id}/reject — Reject (reject).
• POST /api/grey-queue/{id}/reopen — Reopen a closed entry (reopen).
• POST /api/grey-queue/expire — Expire old entries (expire).

Health

• GET /healthz — Liveness (anonymous; ignores DB).
• GET /readyz — Readiness (anonymous; exercises the DB probe via the ready-tagged database check).
• GET /health — Combined health (retained for backwards compatibility).

Domain Model & Enums

Unknown(StellaOps.Unknowns.Core/Models/Unknown.cs) — bitemporal record with subject (SubjectHash, SubjectType, SubjectRef), Kind, optional Severity, source references (SourceScanId/SourceGraphId/SourceSbomDigest), bitemporal columns, resolution tracking, the full scoring breakdown, rescan scheduling, evidence hashes (EvidenceSetHash/GraphSliceHash), audit fields, and provenance hints.

Enums (Unknown.cs):

• UnknownSubjectType: Package, Ecosystem, Version, SbomEdge, File, Runtime, Binary.
• UnknownKind: MissingSbom, AmbiguousPackage, MissingFeed, UnresolvedEdge, NoVersionInfo, UnknownEcosystem, PartialMatch, VersionRangeUnbounded, UnsupportedFormat, TransitiveGap, and the native-binary kinds MissingBuildId, UnknownBuildId, UnresolvedNativeLibrary, HeuristicDependency, UnsupportedBinaryFormat.
• UnknownSeverity: Critical, High, Medium, Low, Info.
• ResolutionType: FeedUpdated, SbomProvided, ManualMapping, Superseded, FalsePositive, WontFix.
• TriageBand: Hot (score ≥ 0.70), Warm (0.40–0.69), Cold (< 0.40).

GreyQueueEntry(StellaOps.Unknowns.Core/Models/GreyQueueEntry.cs) — references an Unknown, carries the DSSE-style evidence bundle (with per-artifact SHA-256 hashes), trigger conditions, processing metadata (attempts / MaxAttempts = 10), and a status workflow.

Enums (GreyQueueEntry.cs):

• GreyQueueStatus: Pending, Processing, Retrying, UnderReview, Escalated, Resolved, Rejected, Failed, Expired, Dismissed. Transitions are enforced by GreyQueueStateMachine.
• GreyQueueReason: InsufficientVex, ConflictingVex, MissingReachability, AmbiguousIdentity, FeedNotAvailable, ToolUnsupported, BinaryAnalysisInconclusive, BackportUncertain, MultipleInterpretations, AwaitingUpstreamAdvisory.
• GreyQueueResolution: FeedUpdate, ToolUpdate, VexReceived, ManualResolution, Superseded, NotApplicable, Expired.

Configuration

Persistence is configured from the Postgres configuration section (bound to PostgresOptions); the schema name defaults to unknowns (UnknownsDataSource.DefaultSchemaName). Authentication/authorization, CORS, localization, audit emission, and the Stella Router (Router section, service name unknowns) are configured in Program.cs.

The StellaOps.Unknowns.Services library defines options records (GreyQueueWatchdogOptions, UnknownsSlaOptions) for the SLA/watchdog/lifecycle background services. These are not bound or hosted by the WebService today (see Implementation Status), so their config keys are not part of the deployed service surface.

Authorization Scopes

Defined in the canonical catalog StellaOps.Auth.Abstractions/StellaOpsScopes.cs:

• unknowns:read (StellaOpsScopes.UnknownsRead) — query/view unknowns and grey-queue entries.
• unknowns:write (StellaOpsScopes.UnknownsWrite) — mutate unknowns and grey-queue entries.
• unknowns:admin (StellaOpsScopes.UnknownsAdmin) — defined in the catalog; not currently referenced by any Unknowns endpoint policy.

Storage

• Database: PostgreSQL, schema unknowns (plus the unknowns_app helper schema for the RLS tenant-context function require_current_tenant()).
• Auto-migration: Embedded SQL migrations under StellaOps.Unknowns.Persistence/Migrations/ are applied on startup via AddStartupMigrations (module Unknowns.Persistence). The archived Migrations/_archived/** set is explicitly excluded from embedding.
• Migrations: one live file, 001_v1_unknowns_baseline.sql — the collapsed pre-1.0 chain (bitemporal table, types, indexes, views, scoring/temporal functions, RLS; provenance-hint columns + GIN indexes + validation function; idempotent scoring-column repair). The pre-collapse filenames (001_initial_schema.sql, 002_provenance_hints.sql, 003_unknowns_scoring_drift_repair.sql) survive only under Migrations/_archived/pre_1.0/mig061/ and are not embedded.
• Table: unknowns.unknown — the single bitemporal table, with Row-Level Security policy unknown_tenant_isolation.
• Views: unknowns.current (valid + known now), unknowns.resolved, unknowns.triage_summary (counts/score stats by band & kind).
• Functions: unknowns.as_of(...), unknowns.count_by_kind(...), unknowns.count_by_severity(...), unknowns.assign_band(...), unknowns.calc_popularity_score(...), unknowns.calc_staleness_score(...), unknowns.calc_composite_score(...), unknowns.update_updated_at() (trigger), unknowns.validate_provenance_hints(...).
• PostgreSQL enum types: unknowns.subject_type, unknowns.unknown_kind, unknowns.unknown_severity, unknowns.resolution_type, unknowns.triage_band.

There is no grey_queue table in the current migrations, and no concrete IGreyQueueRepository implementation is registered. The Grey Queue domain model, endpoints, and background services exist, but their persistence is not yet wired (see Implementation Status).

Integration Points

Stella Router / Authority: Registered as the unknowns router microservice; authentication via the Stella resource-server integration with scope-based policies and tenant middleware.

Audit: Grey Queue mutations emit audit events under AuditModules.Unknowns (enqueue, start_processing, record_result, resolve, dismiss, expire, assign, escalate, reject, reopen).

Notifications: Escalations publish an EscalationNotification through an optional INotificationPublisher; no production implementation is registered today (test stubs only).

Dependencies

• PostgreSQL (registry storage, bitemporal audit, RLS tenant isolation).
• StellaOps Authority / Router (authentication, scope policies, service mesh).
• StellaOps Audit Emission (grey-queue action auditing).
• StellaOps Localization (endpoint descriptions).

Related Documentation

• Architecture, API reference, and Grey Queue state machine.
• Module agent guide: src/Unknowns/AGENTS.md.
• PostgreSQL operational patterns runbook.
• ADR-004: forward-only migrations.

Implementation Status

Implemented and wired into the WebService:

• Bitemporal unknowns.unknown registry with EF Core/PostgreSQL persistence and startup auto-migration.
• Unknowns read API (list/get/hints/history/triage-band/hot-queue/high-confidence/summary).
• Triage scoring, bands, and provenance hints (model, SQL functions, and read endpoints).
• Scope-based authorization, tenant header/RLS isolation, health/readiness probes, Stella Router integration.

Partially implemented (present in source but NOT wired into the deployed WebService):

• Grey Queue persistence: The /api/grey-queue endpoints and IGreyQueueRepository interface exist, but no concrete repository implementation is registered in the WebService and no grey_queue table exists in the migrations. Calls into these endpoints cannot currently resolve their repository dependency.
• Background workers / SLA monitoring / metrics: StellaOps.Unknowns.Services (watchdog, SLA monitor, lifecycle, metrics emitting unknowns_queue_depth, unknowns_sla_compliance_rate, unknowns_processing_time_seconds, unknowns_resolution_time_hours, unknowns_state_transitions_total) is referenced only by its test project, not by the WebService. No background workers run in the deployed service.

Key differentiator: Unlike scanners that hide uncertainty, Stella Ops makes “what we don’t know” an explicit, queryable, bitemporal signal with deterministic scoring and provenance hints — important for air-gapped deployments and zero-day scenarios where external validation is unavailable.

Documentation note: Earlier revisions of this page described a StellaOps.Policy.Unknowns library with an “unknowns budget” gate, “decay heuristics”, “Unknown-Reachable / Unknown-Unreachable” states, a policy-schema location, and UnknownBudgetOptions/UnknownsDecayOptions/UnknownsRescanOptions config classes. None of those exist in src/. If an unknowns-budget Policy gate is intended, it should be tracked as a separate forward design item, not described here as implemented.