Aggregation-Only Contract Reference

Audience: Ingestion engineers, DevOps and QA operators, and security reviewers working on Concelier, Excititor, and other collectors. Scope: The governing AOC rule set — contract invariants, raw schemas, the error model, API/CLI surfaces, idempotency rules, migration, observability, and compliance checklists.

The Aggregation-Only Contract (AOC) is the governing rule set that keeps Stella Ops ingestion services deterministic, policy-neutral, and auditable. It applies to Concelier, Excititor, and any future collectors that write raw advisory or VEX documents.

1. Purpose and Scope

• Defines the canonical behaviour for advisory_raw and vex_raw collections and the linkset hints they may emit.
• Applies to every ingestion runtime (StellaOps.Concelier.*, StellaOps.Excititor.*), the Authority scopes that guard them, and the DevOps/QA surfaces that verify compliance.
• Complements the high-level architecture in Concelier Architecture and the Authority enforcement documented in Authority Architecture.
• Paired guidance: see the guard-rail checkpoints in AOC Guardrails, the implementation reference in AOC Guard Library, and the CLI usage covered in Vulnerability Explorer CLI.

2. Philosophy and Goals

• Preserve upstream truth: ingestion only captures immutable raw facts plus provenance, never derived severity or policy decisions.
• Defer interpretation: Policy Engine and downstream overlays remain the sole writers of materialised findings, severity, consensus, or risk scores.
• Make every write explainable: provenance, signatures, and content hashes are required so operators can prove where each fact originated.
• Keep outputs reproducible: identical inputs must yield identical documents, hashes, and linksets across replays and air-gapped installs.

3. Contract Invariants

4. Raw Schemas

4.1 advisory_raw

4.2 vex_raw

4.3 Linkset Fields

• purls: fully qualified Package URLs extracted from raw ranges or product nodes.
• cpes: Common Platform Enumerations when upstream docs provide them.
• aliases: Any alternate advisory identifiers present in the payload.
• references: Array of { type, url } pairs pointing back to vendor advisories, patches, or exploits.
• reconciled_from: Provenance of linkset entries (JSON Pointer or field origin) to make automated checks auditable.

Canonicalisation rules:

• Package URLs are rendered in canonical form without qualifiers/subpaths (pkg:type/namespace/name@version).
• CPE values are normalised to the 2.3 binding (cpe:2.3:part:vendor:product:version:*:*:*:*:*:*:*).
• Connector mapping stages are responsible for the canonical form; ingestion trims whitespace but otherwise preserves the original order and duplicate entries so downstream policy can reason about upstream intent.

4.4 advisory_observations

advisory_observations is an immutable projection of the validated raw document used by Link‑Not‑Merge overlays. Fields mirror the JSON contract surfaced by StellaOps.Concelier.Models.Observations.AdvisoryObservation.

5. Error Model

Consumers should map these codes to CLI exit codes and structured log events so automation can fail fast and produce actionable guidance. The shared guard library (StellaOps.Aoc.AocError) emits consistent payloads (code, message, violations[]) for HTTP APIs, CLI tooling, and verifiers.

6. API and Tooling Interfaces

• Concelier ingestion (StellaOps.Concelier.WebService)
  • POST /ingest/advisory: accepts upstream payload metadata; server-side guard constructs and persists raw document.
  • GET /advisories/raw/{id} and filterable list endpoints expose raw documents for debugging and offline analysis.
  • POST /aoc/verify: runs guard checks over recent documents and returns summary totals plus first violations.
• Excititor ingestion (StellaOps.Excititor.WebService) mirrors the same surface for VEX documents.
• CLI workflows (stella aoc verify, stella sources ingest --dry-run) surface pre-flight verification; see the CLI aoc command reference.
• Authority scopes: the advisory:ingest, advisory:read, vex:ingest, and vex:read scopes enforce least privilege; see Authority Architecture for the scope grammar.

7. Idempotency and Supersedes Rules

1. Compute content_hash before any transformation; use it with (source.vendor, upstream.upstream_id) to detect duplicates.
2. If a document with the same hash already exists, skip the write and log a no-op.
3. When a new hash arrives for an existing upstream document, insert a new record and set supersedes to the previous _id.
4. Keep supersedes chains acyclic; collectors must resolve conflicts by rewinding before they insert.
5. Expose idempotency counters via metrics (ingestion_write_total{result=ok|noop}) to catch regressions early.

8. Migration Playbook

1. Freeze ingestion writes except for raw pass-through paths while deploying schema validators.
2. Snapshot existing collections to _backup_* for rollback safety.
3. Strip forbidden fields from historical documents into a temporary advisory_view_legacy used only during transition.
4. Enable PostgreSQL JSON schema validators for advisory_raw and vex_raw.
5. Run collectors in --dry-run to confirm only allowed keys appear; fix violations before lifting the freeze.
6. Point Policy Engine to consume exclusively from raw collections and compute derived outputs downstream.
7. Delete legacy normalisation paths from ingestion code and enable runtime guards plus CI linting.
8. Roll forward CLI, Console, and dashboards so operators can monitor AOC status end-to-end.

9. Observability and Diagnostics

• Metrics: ingestion_write_total{result=ok|reject}, aoc_violation_total{code}, ingestion_signature_verified_total{result}, ingestion_latency_seconds, advisory_revision_count.
• Traces: spans ingest.fetch, ingest.transform, ingest.write, and aoc.guard with correlation IDs shared across workers.
• Logs: structured entries must include tenant, source.vendor, upstream.upstream_id, content_hash, and violation_code when applicable.
• Dashboards: DevOps should add panels for violation counts, signature failures, supersedes growth, and CLI verifier outcomes for each tenant.

10. Security and Tenancy Checklist

• Enforce Authority scopes (advisory:ingest, vex:ingest, advisory:read, vex:read) and require tenant claims on every request.
• Maintain pinned trust stores for signature verification; capture verification result in metrics and logs.
• Ensure collectors never log secrets or raw authentication headers; redact tokens before persistence.
• Validate that Policy Engine remains the only identity with permission to write effective_finding_* documents.
• Verify offline bundles include the raw collections, guard configuration, and verifier binaries so air-gapped installs can audit parity.
• Document operator steps for recovering from violations, including rollback to superseded revisions and re-running policy evaluation.

11. Compliance Checklist

• [ ] Deterministic guard enabled in Concelier and Excititor repositories.
• [ ] PostgreSQL validators deployed for advisory_raw and vex_raw.
• [ ] Authority scopes and tenant enforcement verified via integration tests.
• [ ] CLI and CI pipelines run stella aoc verify against seeded snapshots.
• [ ] Observability feeds (metrics, logs, traces) wired into dashboards with alerts.
• [ ] Offline kit instructions updated to bundle validators and verifier tooling.
• [ ] Security review recorded covering ingestion, tenancy, and rollback procedures.

Last updated: 2025-10-27 (Sprint 19).