StellaOps VEX Consensus Lens

VEX Lens produces a deterministic, provenance-rich consensus view of VEX statements so downstream policy and exploration surfaces can trust what “affected” or “not affected” means per artifact. It separates raw ingestion (Excititor) from adjudication, weighs issuers cryptographically, and keeps conflicts transparent for auditors and tooling.

Scope & responsibilities

• Normalise OpenVEX, CSAF VEX, and CycloneDX VEX documents from Excititor into lens-ready tuples without violating the Aggregation-Only Contract.
• Apply issuer weights, signature verification results, freshness decay, and policy overrides to compute consensus and confidence for each (artifact, advisory) pair.
• Emit consensus snapshots, conflict summaries, and DSSE attestations consumed by Policy Engine, Vulnerability Explorer, Advisory AI, and Task Runner workflows.
• Provide simulation/export APIs and Offline Kit bundles so tenants can rehearse policy changes and mirror consensus data in air-gapped environments.

Architecture snapshot (Sprint 30 groundwork)

• StellaOps.VexLens service & workers — orchestrate normalisation, trust weighting, mode-driven consensus (weighted-vote default; highest-weight / lattice / authoritative-first also available), and persistence into the vexlens schema (vexlens.consensus_projections, vexlens.consensus_inputs, vexlens.consensus_conflicts, plus noise-gating tables) on PostgreSQL.
• Issuer Directory integration — maintains publisher metadata, keys, and trust tiers that feed weighting engines and revocation workflows.
• Consensus APIs — mounted under /api/v1/vexlens (POST /consensus, POST /consensus:withProof, POST /consensus:batch, GET /projections, GET /projections/{projectionId}, GET /projections/latest, GET /projections/history, GET /conflicts, GET /stats, POST /deltas/compute, GET /gating/statistics, POST /gating/snapshots/{snapshotId}/gate, and the issuers/* CRUD group). The read-only group is guarded by the vexlens.read scope; the issuers/* group requires vexlens.write. Both are canonical Authority scopes (StellaOpsScopes.VexLensRead / VexLensWrite) and the shipped devops/etc/authority/plugins/standard.yaml grants them to the first-party Console and CLI clients. Forward migration 010_authority_stella_ops_ui_vexlens_read_scope.sql additively repairs an existing or drifted stella-ops-ui row so clean upgrades do not require hand-granted SQL. (The earlier /v1/vex/* paths were never implemented. The export/* route group exists in ExportEndpointExtensions.MapExportEndpoints and requires vexlens.read, but Program.cs only calls MapVexLensEndpoints(), so the export routes are not currently mounted.)
• Explainability traces — capture derived-from chains, conflicting issuers, and trust deltas to power UI drilldowns and CLI audits.
• Recompute orchestration — Orchestrator jobs trigger recompute on Excititor deltas, issuer updates, or policy knob changes with deterministic ordering and SRM manifests.

Provenance-aware trust weighting (new)

• Connector metadata contract. Excititor connectors now emit vex.provenance.* fields (provider id/name/kind, trust.weight, trust.tier, human-readable trust.note, cosign.*, and ordered pgp.fingerprints). VEX Lens must ingest these keys verbatim so the trust engine can reason about issuer pedigree without hitting external registries for every statement.
• Weight calculation. Lens uses the supplied trust.weight as the baseline score, then multiplies by freshness decay and justification scope multipliers. Missing weights default to the Issuer Directory profile, but connector-provided values take precedence so Ubuntu/SUSE mirror feeds can tune their relative influence.
• Integrity hints. Presence of vex.provenance.cosign.* or pgp.fingerprints is preserved for verification policy, but it is not itself proof. Lens applies the cryptographically verified confidence tier only when configured local DSSE trust roots verify the signature and issuer trust validation succeeds; missing roots and JWS defaults fail closed.
• Policy exposure. Consensus APIs surface the original provenance payload inside each sources[] entry so Policy Engine, Advisory AI, and Console can explain why a lower-tier issuer lost a conflict (e.g., different trust.tier or missing fingerprints). See the updated payload reference in guides/consensus-json.md.

Current workstreams (Q4 2025)

• VEXLENS-30-001..004 — build normalisation pipeline, product mapping library, and trust weighting engine (in progress; dependencies captured in src/VexLens/StellaOps.VexLens/TASKS.md).
• VEXLENS-30-005..007 — expose consensus APIs and export flows, aligning docs with future /docs/vex/consensus-*.md deliverables.
• DOCS-VEX-30-001..004 — broader documentation set (overview, algorithm, API) tracked alongside the module implementation plan; update this README again once those artefacts merge.

Latest updates

• 2026-04-27: Default DSSE/JWS signature handling now fails closed when no cryptographic verifier is configured. Parsing a structurally valid envelope returns an unavailable verification result instead of a Valid trust signal.
• 2026-04-28: DSSE verification can use configured local PEM trust roots via VexLens:Trust:SignatureTrustRootPaths and the shared Attestation verifier. JWS inputs without local Fulcio or JWKS proof material remain unavailable.
• 2026-04-28: Compact JWS verification now supports local Fulcio roots when the protected header includes x5c, the chain validates to a configured local root, the leaf certificate is a non-CA signing certificate with an accepted Fulcio issuer, and the leaf certificate verifies the signature. Missing roots, missing x5c, unsupported critical headers/algorithms, non-signing leaf certificates, missing/untrusted issuers, and tampered signatures remain fail-closed.
• 2026-04-28: Compact JWS verification with CheckRevocation=true now requires local revocation bundle material in the configured trust-root paths. Missing/malformed bundles and revoked Fulcio leaf certificates or JWKS public-key fingerprints fail closed.
• 2026-04-28: Detached compact JWS verification now binds SignatureVerificationRequest.Content payload bytes to SignatureVerificationRequest.DetachedSignature before applying local Fulcio/JWKS/revocation checks. Detached-shaped JWS without request payload binding, payload-part mismatches, and altered payload bytes fail closed.
• 2026-04-24: Runtime cache and orchestrator-ledger defaults tightened: AddVexLens no longer registers process-local rationale/source-trust caches, and OrchestratorLedgerEventEmitter now requires a concrete ledger client instead of silently dropping events.
• 2026-04-24: AddVexLens now defaults consensus projection storage to PostgreSQL and requires an explicit connection string for runtime use. Issuer directory, consensus event emitter, snapshot store, and gating statistics store defaults fail closed until the host registers durable/runtime implementations; AddVexLensForTesting is the explicit in-memory harness.
• 2025-11-30: Docs refresh per docs/implplan/SPRINT_0332_0001_0001_docs_modules_vex_lens.md; added observability runbook stub and TASKS mirror.
• Observability assets: runbooks/observability.md and runbooks/dashboards/vex-lens-observability.json (offline import).

Integrations & dependencies

• Excititor supplies signature-verified VEX observations and issuer hints.
• Policy Engine consumes consensus verdicts for suppression/waiver logic and exposes trust controls to operators.
• Vulnerability Explorer & Advisory AI render consensus badges, conflicts, and rationale in investigative workflows.
• Orchestrator & Scheduler run backfills, recomputes, and incident routing driven by Lens change streams.
• Notify / Task Runner receive conflict and override events for operator actions once notification bridges ship.

Data & observability

• PostgreSQL tables (vexlens schema, migrations 001_consensus_projections.sql / 002_noise_gating_state.sql, auto-applied via AddStartupMigrations): vexlens.consensus_projections, vexlens.consensus_inputs, vexlens.consensus_conflicts, plus the noise-gating tables vexlens.noise_gate_raw_snapshots, vexlens.noise_gate_gated_snapshots, and vexlens.noise_gate_statistics, managed with tenant isolation and deterministic indexes. (vex_consensus / vex_consensus_history survive only as unused VexLensStorageOptions collection-name defaults; vex_conflict_queue does not exist in code.)
• Metrics (meter StellaOps.VexLens): vexlens.consensus.computed_total, vexlens.consensus.conflicts_total, vexlens.consensus.confidence, vexlens.consensus.duration_seconds, vexlens.consensus.status_changes_total, plus signature/trust/normalization counters.
• Traces/logs: spans vexlens.compute_consensus, vexlens.store_projection, vexlens.query_projections, etc. with issuer details; structured logs capture trust adjustments and reconciliation outcomes.
• Offline bundles include consensus.jsonl, conflicts.jsonl, manifest + DSSE signatures, enabling mirror deployments and replay validation.

Implementation Status

Phase 1 – Core lens service (In Progress)

• Normalization pipeline: CSAF/OpenVEX/CycloneDX format support
• Product mapping library with conservative scope scoring
• Trust weighting functions: issuer tier, freshness decay, scope quality
• Consensus algorithm (mode-driven; weighted-vote default) with deterministic ordering and input_hash for replay
• Persistence: vexlens.consensus_projections / consensus_inputs / consensus_conflicts tables (PostgreSQL vexlens schema)
• Connector-supplied trust weights/tiers from Excititor vex.provenance.* contract

Phase 2 – API & integrations (Partially shipped)

• REST endpoints shipped under /api/v1/vexlens/* (consensus compute/proof/batch, projections, conflicts, gating, issuers). The export/* group is implemented but not yet mapped in Program.cs. A read-by-query-string /vex/consensus and a simulate endpoint are roadmap, not implemented; consensus is computed on demand via POST.
• Policy Engine threshold integration and simulation support
• Vuln Explorer UI chips for consensus signals
• VEX Lens change events for downstream consumers

Phase 3 – Issuer Directory & signatures (Planned)

• Issuer registry with CRUD, audit logs, CSAF publisher import
• Key management and signature verification
• RBAC enforcement and tenant overrides
• Revocation runbooks and trust recalculation

Phase 4 – Console & CLI experiences (Planned)

• Console module: evidence table, quorum bar, conflicts, simulation drawer
• CLI commands (roadmap): stella vex consensus list/show/simulate/export — not yet implemented. The shipped CLI surface under stella vex is lens analyze / lens explain / apply (plus generate, validate, query, advisory, and a separate vex providers group).
• Saved views, filters, JSON/CSV output support

Phase 5 – Recompute & performance (Planned)

• Recompute scheduling: policy activation, Excititor deltas
• Caching strategy and load tests (10M records/tenant, P95 < 500ms)
• Observability dashboards and Offline Kit exports
• Backpressure handling and incident surfacing

Key Acceptance Criteria

• Consensus results reproducible across VEX formats with deterministic digests
• Signature verification influences trust weights without pipeline failure
• Policy simulations show quorum shifts without persisting state
• Issuer Directory enforces RBAC, audit logs, key rotation
• Recompute pipeline handles deltas with backpressure management
• Performance: P95 < 500ms for 100-row pages at 10M records/tenant

Technical Decisions & Risks

• Product mapping ambiguity: conservative scoring, manual overrides, warnings, policy review
• Issuer compromise: cryptographic signature verification, trust weighting, tenant overrides, revocation runbooks; structure-only DSSE/JWS parsing is an unavailable state and must not increase trust.
• Evidence storms: batching, worker sharding, orchestrator rate limiting, priority queues
• Performance: caching, indexing, load tests, quota enforcement
• Offline gaps: deterministic exports, manifest hashes, Offline Kit tests

Provenance-Aware Trust Weighting (Current Focus)

• Connector metadata contract: vex.provenance.* fields with provider id/name/kind
• Weight calculation: trust.weight baseline × freshness × justification scope
• Integrity hints: cosign.* and pgp.fingerprints feed signature policy only after configured cryptographic verification succeeds
• Policy exposure: original provenance in sources[] for explain workflows

Key docs & references

• architecture.md— implementation-ready blueprint covering inputs, algorithm, APIs, storage, observability, and exports.
• scoring.md— future risk scoring model and formula reference.
• guides/aggregation.md— Aggregation-Only Contract boundaries for VEX ingestion and downstream consumers.
• Operations: operations/deployment.md, operations/offline-kit.md— deployment guides and offline bundle preparation.
• Sprint tracking in docs/implplan/SPRINT_0332_0001_0001_docs_modules_vex_lens.md; module engineering tasks in src/VexLens/StellaOps.VexLens/TASKS.md; doc-side planning in implementation_plan.md.

Epic alignment

• Epic 7 — VEX Consensus Lens: deterministic VEX adjudication, issuer directory, consensus exports.
• Related epics: Issuer Directory (keys & trust), Vulnerability Explorer overlays, Policy trust tuning.