VEX Observation Model (vex_observations)

Authored 2025-11-14 for Sprint 120 (EXCITITOR-LNM-21-001). This document is the canonical schema description for Excititor’s immutable observation records. It unblocks downstream documentation tasks (DOCS-LNM-22-002) and aligns the WebService/Worker data structures with Mongo persistence.

Excititor ingests heterogeneous VEX statements, normalizes them under the Aggregation-Only Contract (AOC), and persists each normalized statement as a VEX observation. These observations are the source of truth for:

• Advisory AI citation APIs (/v1/vex/observations/{vulnerabilityId}/{productKey})
• Graph/Vuln Explorer overlays (batch observation APIs)
• Evidence Locker + portable bundle manifests
• Policy Engine materialization and audit trails

All observation documents are immutable. New information creates a new observation record linked by observationId; supersedence happens through Graph/Lens layers, not by mutating this collection.

Storage & routing

Aspect	Value
Collection	vex_observations (Mongo)
Upstream generator	VexObservationProjectionService (WebService) and Worker normalization pipeline
Primary key	{tenant, observationId}
Required indexes	{tenant, vulnerabilityId}, {tenant, productKey}, {tenant, document.digest}, {tenant, providerId, status}
Source of truth for	/v1/vex/observations, Graph batch APIs, Excititor → Evidence Locker replication

Canonical document shape

{
  "tenant": "default",
  "observationId": "vex:obs:sha256:...",
  "vulnerabilityId": "CVE-2024-12345",
  "productKey": "pkg:maven/org.example/app@1.2.3",
  "providerId": "ubuntu-csaf",
  "status": "affected",                // matches VexClaimStatus enum
  "justification": {
    "type": "component_not_present",
    "reason": "Package not shipped in this profile",
    "detail": "Binary not in base image"
  },
  "detail": "Free-form vendor detail",
  "confidence": {
    "score": 0.9,
    "level": "high",
    "method": "vendor"
  },
  "signals": {
    "severity": {
      "scheme": "cvss3.1",
      "score": 7.8,
      "label": "High",
      "vector": "CVSS:3.1/..."
    },
    "kev": true,
    "epss": 0.77
  },
  "scope": {
    "key": "pkg:deb/ubuntu/apache2@2.4.58-1",
    "purls": [
      "pkg:deb/ubuntu/apache2@2.4.58-1",
      "pkg:docker/example/app@sha256:..."
    ],
    "cpes": ["cpe:2.3:a:apache:http_server:2.4.58:*:*:*:*:*:*:*"]
  },
  "anchors": [
    "#/statements/0/justification",
    "#/statements/0/detail"
  ],
  "document": {
    "format": "csaf",
    "digest": "sha256:abc123...",
    "revision": "2024-10-22T09:00:00Z",
    "sourceUri": "https://ubuntu.com/security/notices/USN-0000-1",
    "signature": {
      "type": "cosign",
      "issuer": "https://token.actions.githubusercontent.com",
      "keyId": "ubuntu-vex-prod",
      "verifiedAt": "2024-10-22T09:01:00Z",
      "transparencyLogReference": "rekor://UUID",
      "trust": {
        "tenantId": "default",
        "issuerId": "ubuntu",
        "effectiveWeight": 0.9,
        "tenantOverrideApplied": false,
        "retrievedAtUtc": "2024-10-22T09:00:30Z"
      }
    }
  },
  "aoc": {
    "guardVersion": "2024.10.0",
    "violations": [],                    // non-empty -> stored + surfaced
    "ingestedAt": "2024-10-22T09:00:05Z",
    "retrievedAt": "2024-10-22T08:59:59Z"
  },
  "metadata": {
    "provider-hint": "Mainline feed",
    "source-channel": "mirror"
  }
}

Field notes

• tenant – logical tenant resolved by WebService based on headers or default configuration.
• observationId – deterministic hash (sha256) over {tenant, vulnerabilityId, productKey, providerId, statementDigest}. Never reused.
• status + justification – follow the OpenVEX semantics enforced by StellaOps.Excititor.Core.VexClaim.
• scope – includes canonical key plus normalized PURLs/CPES; deterministic ordering.
• anchors – optional JSON-pointer hints pointing to the source document sections; stored as trimmed strings.
• document.signature – mirrors VexSignatureMetadata; empty if upstream feed lacks signatures.
• aoc.violations – stored if the guard detected non-fatal issues; fatal issues never create an observation.
• metadata – reserved for deterministic provider hints; keys follow vex.* prefix guidance.

Determinism & AOC guarantees

1. Write-once – once inserted, observation documents never change. New evidence creates a new observationId.
2. Sorted collections – arrays (anchors, purls, cpes) are sorted lexicographically before persistence.
3. Guard metadata – aoc.guardVersion records the guard library version (docs/aoc/guard-library.md), enabling audits.
4. Signatures – only verification metadata proven by the Worker is stored; WebService never recomputes trust.
5. Time normalization – all timestamps stored as UTC ISO-8601 strings (Mongo DateTime).

API mapping

API	Source fields	Notes
/v1/vex/observations/{vuln}/{product}	tenant, vulnerabilityId, productKey, scope, statements[]	Response uses VexObservationProjectionService to render statements, document, and signature fields.
/vex/aoc/verify	document.digest, providerId, aoc	Replays guard validation for recent digests; guard violations here align with aoc.violations.
Evidence batch API (Graph)	statements[], scope, signals, anchors	Format optimized for overlays; resuces document to digest/URI.

Related work

• EXCITITOR-GRAPH-24-* relies on this schema to build overlays.
• DOCS-LNM-22-002 (Link-Not-Merge documentation) references this file.
• EXCITITOR-ATTEST-73-* uses document.digest + signature to embed provenance in attestation payloads.