Link-Not-Merge (LNM) Observation & Linkset Schema

Frozen v1 (add-only) — approved 2025-11-17 for CONCELIER-LNM-21-001/002/101.

Goals

Immutable storage of raw advisory observations per source/tenant.
Deterministic linksets built from observations without merging or mutating originals.
Stable across online/offline deployments; replayable from raw inputs.

Status

Frozen v1 as of 2025-11-17; further schema changes must go through ADR + sprint gating (CONCELIER-LNM-22x+).
Canonical JSON Schemas + signed manifest live in docs/modules/concelier/schemas/ (advisory observation, linkset, offline bundle). Verify with openssl dgst -sha256 -verify schema-signing-pub.pem -signature schema.manifest.sig schema.manifest.json.

Unified Issue Projection Supplement

The Stella Ops Mirror issue skeleton does not alter frozen LNM v1 observation or linkset JSON schemas. It is a PostgreSQL projection over existing advisory, distro, VEX, and future finding observations:

vuln.issue_observations stores source-specific issue observations.
vuln.issue_linksets groups observations by stable issue key.
vuln.issue_evidence_refs keeps pointers back to raw/source tables.

The projection preserves Link-Not-Merge semantics: source rows remain distinct evidence, and the issue linkset is a navigation/indexing aid rather than an ingestion-time truth merge. Details are tracked in docs/modules/concelier/unified-issue-projection.md.

Observation document (PostgreSQL JSON Schema excerpt)

{
  "bsonType": "object",
  "required": ["_id","tenantId","source","advisoryId","affected","provenance","ingestedAt"],
  "properties": {
    "_id": {"bsonType": "objectId"},
    "tenantId": {"bsonType": "string"},
    "source": {"bsonType": "string", "description": "Adapter id, e.g., ghsa, nvd, cert-de"},
    "advisoryId": {"bsonType": "string"},
    "title": {"bsonType": "string"},
    "summary": {"bsonType": "string"},
    "severities": {
      "bsonType": "array",
      "items": {"bsonType": "object", "required": ["system","score"],
        "properties": {"system":{"bsonType":"string"},"score":{"bsonType":"double"},"vector":{"bsonType":"string"}}}
    },
    "affected": {
      "bsonType": "array",
      "items": {"bsonType":"object","required":["purl"],
        "properties": {
          "purl": {"bsonType":"string"},
          "package": {"bsonType":"string"},
          "versions": {"bsonType":"array","items":{"bsonType":"string"}},
          "ranges": {"bsonType":"array","items":{"bsonType":"object",
            "required":["type","events"],
            "properties": {"type":{"bsonType":"string"},"events":{"bsonType":"array","items":{"bsonType":"object"}}}}},
          "ecosystem": {"bsonType":"string"},
          "cpe": {"bsonType":"array","items":{"bsonType":"string"}},
          "cpes": {"bsonType":"array","items":{"bsonType":"string"}}
        }
      }
    },
    "references": {"bsonType": "array", "items": {"bsonType":"string"}},
    "scopes": {"bsonType":"array","items":{"bsonType":"string"}},
    "relationships": {
      "bsonType": "array",
      "items": {"bsonType":"object","required":["type","source","target"],
        "properties": {
          "type":{"bsonType":"string"},
          "source":{"bsonType":"string"},
          "target":{"bsonType":"string"},
          "provenance":{"bsonType":"string"}
        }}
    },
    "weaknesses": {"bsonType":"array","items":{"bsonType":"string"}},
    "published": {"bsonType": "date"},
    "modified": {"bsonType": "date"},
    "provenance": {
      "bsonType": "object",
      "required": ["sourceArtifactSha","fetchedAt"],
      "properties": {
        "sourceArtifactSha": {"bsonType":"string"},
        "fetchedAt": {"bsonType":"date"},
        "ingestJobId": {"bsonType":"string"},
        "signature": {"bsonType":"object"}
      }
    },
    "ingestedAt": {"bsonType": "date"}
  }
}

Observation invariants

Immutable: no in-place updates; new revision → new document with supersedesId optional pointer.
Deterministic keying: _id derived from hash(tenantId|source|advisoryId|provenance.sourceArtifactSha) to keep inserts idempotent in replay.
Normalization guardrails: version ranges must be stored as raw-from-source; no inferred merges.

Append-Only Contract (AOC) — LNM-21-004

The Aggregation-Only Contract (AOC) ensures observations are immutable after creation. This is enforced by IAdvisoryObservationWriteGuard.

Write disposition rules

Existing Hash	New Hash	Disposition	Action
null/empty	any	`Proceed`	Insert new observation
X	X (identical)	`SkipIdentical`	Idempotent re-insert, no write
X	Y (different)	`RejectMutation`	Reject with `AppendOnlyViolationException`

Supersession model

When an advisory source publishes a revised version of an advisory:

A new observation is created with its own unique observationId and contentHash.
The new observation MAY carry a supersedesId pointing to the previous observation.
The original observation remains immutable — it is never updated or deleted.
Linksets are rebuilt to include all non-superseded observations; superseded observations remain queryable for audit but excluded from active linkset aggregation.

Implementation checklist (LNM-21-004)

[x] IAdvisoryObservationWriteGuard interface with ValidateWrite(observation, existingContentHash) method.
[x] AdvisoryObservationWriteGuard implementation enforcing append-only semantics.
[x] AppendOnlyViolationException for mutation rejections.
[x] DI registration via AddConcelierAocGuards() extension.
[x] Unit tests covering Proceed/SkipIdentical/RejectMutation scenarios.
[x] Legacy merge logic deprecated with [Obsolete] and gated by NoMergeEnabled feature flag (defaults to true).
[x] Roslyn analyzer StellaOps.Concelier.Analyzers.NoMergeApiAnalyzer emits warnings for merge API usage.

Linkset document

{
  "bsonType":"object",
  "required":["_id","tenantId","advisoryId","source","observations","createdAt"],
  "properties":{
    "_id":{"bsonType":"objectId"},
    "tenantId":{"bsonType":"string"},
    "advisoryId":{"bsonType":"string"},
    "source":{"bsonType":"string"},
    "observations":{"bsonType":"array","items":{"bsonType":"objectId"}},
    "normalized": {
      "bsonType":"object",
      "properties":{
        "purls":{"bsonType":"array","items":{"bsonType":"string"}},
        "versions":{"bsonType":"array","items":{"bsonType":"string"}},
        "ranges": {"bsonType":"array","items":{"bsonType":"object"}},
        "severities": {"bsonType":"array","items":{"bsonType":"object"}}
      }
    },
    "confidence": {"bsonType":"double", "description":"Optional correlation confidence (0–1)"},
    "conflicts": {"bsonType":"array","items":{"bsonType":"object",
      "required":["field","reason"],
      "properties":{
        "field":{"bsonType":"string"},
        "reason":{"bsonType":"string"},
        "values":{"bsonType":"array","items":{"bsonType":"string"}},
        "sourceIds":{"bsonType":"array","items":{"bsonType":"string"}}
      }}},
    "createdAt":{"bsonType":"date"},
    "builtByJobId":{"bsonType":"string"},
    "provenance": {"bsonType":"object","properties":{
      "observationHashes":{"bsonType":"array","items":{"bsonType":"string"}},
      "toolVersion" : {"bsonType":"string"},
      "policyHash" : {"bsonType":"string"}
    }}
  }
}

Linkset invariants

Built from a set of observation IDs; never overwrites observations.
Carries the hash list of source observations for audit/replay.
Deterministic sort: observations sorted by source, advisoryId, fetchedAt before hashing.
Conflicts are additive only and now carry optional sourceIds[] to trace which upstream sources produced divergent values.

Indexes (PostgreSQL)

Observations: { tenantId:1, source:1, advisoryId:1, provenance.fetchedAt:-1 } (compound for ingest); { provenance.sourceArtifactSha:1 } unique to avoid dup writes.
Linksets: { tenantId:1, advisoryId:1, source:1 } unique; { observations:1 } sparse for reverse lookups.

Tables

advisory_observations — raw per-source docs (immutable).
advisory_linksets — derived normalized aggregates with observation pointers and hashes.

Determinism & replay

Replay rebuild: order observations by fetchedAt, recompute linkset hash list, ensure byte-identical linkset JSON.
All timestamps UTC ISO-8601; no server-local time.
String normalization: lowercase source, trim/normalize PURLs, stable sort arrays.

Sample documents

See docs/modules/concelier/samples/observation-ghsa.json and docs/modules/concelier/samples/linkset-ghsa.json (added with this draft) for concrete payloads.

Approval path

Architecture + Concelier Core review this document.
If accepted, freeze JSON Schema and roll into src/Concelier/__Libraries/StellaOps.Concelier.Storage.Postgres migrations.
Update consumers (policy/CLI/export) to read from linksets only; deprecate Merge endpoints.

Tracking: CONCELIER-LNM-21-001/002/101; Sprint 110 blockers (Concelier/Excititor waves).