Policy Engine Data Pipeline

Audience: policy authors, platform engineers, and reviewers who need to understand — end to end — how evidence becomes a vulnerability verdict in Stella Ops.

Overview

This document gives an architectural view of how Stella Ops feeds data to the Policy Engine for vulnerability risk decisions. The pipeline spans multiple subsystems: SBOM generation (per-ecosystem analyzers), runtime observation via agents, static/dynamic call-graph analysis, binary fingerprinting, and evidence-weighted scoring.

Design principle: Every policy decision links back to concrete, verifiable evidence. The system aggregates evidence from multiple sources without silently merging conflicts (aggregation-not-merge); conflicts surface as a contested state for human review rather than being collapsed into a single value.

Altitude note. This is a Tier-2 conceptual map. It deliberately describes the shape of each contract, schema, and enum, and points to the authoritative source. Exact enum members, field lists, config keys, and numeric weights live in src/ and the module dossiers — verify those before relying on a specific value. Where this doc previously enumerated internals, it now carries a 1–2 line summary plus a pointer.

Master Data Flow Diagram

┌─────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                            POLICY ENGINE DATA PIPELINE - COMPLETE VIEW                                   │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────┘

                                    ┌──────────────────────┐
                                    │   CONTAINER IMAGE    │
                                    │   (OCI/Docker/tar)   │
                                    └──────────┬───────────┘
                                               │
                    ┌──────────────────────────┼──────────────────────────┐
                    │                          │                          │
                    ▼                          ▼                          ▼
    ┌───────────────────────┐   ┌───────────────────────┐   ┌───────────────────────┐
    │   LAYER EXTRACTION    │   │   BINARY EXTRACTION   │   │   RUNTIME DEPLOYMENT  │
    │   (FS + Metadata)     │   │   (ELF/PE/Mach-O)     │   │   (Docker/containerd) │
    └───────────┬───────────┘   └───────────┬───────────┘   └───────────┬───────────┘
                │                           │                           │
    ┌───────────┴───────────┐   ┌───────────┴───────────┐   ┌───────────┴───────────┐
    ▼                       ▼   ▼                       ▼   ▼                       ▼
┌─────────┐           ┌─────────┐               ┌─────────┐               ┌─────────┐
│LANGUAGE │           │   OS    │               │ BINARY  │               │ RUNTIME │
│ANALYZERS│           │ANALYZERS│               │ INDEX   │               │ AGENTS  │
└────┬────┘           └────┬────┘               └────┬────┘               └────┬────┘
     │                     │                         │                         │
     ▼    ▼    ▼                                                               ▼
┌─────────────────────┐   ┌─────────────────────┐   ┌─────────────────────────────────┐
│      SBOM           │   │   SYMBOL MANIFEST   │   │     RUNTIME SIGNALS             │
│  (CycloneDX/SPDX)   │   │   (Function Hashes) │   │  (Function Calls + Syscalls)    │
└──────────┬──────────┘   └──────────┬──────────┘   └───────────────┬─────────────────┘
           │                         │                               │
           ▼    ▼                                                    ▼
    ┌─────────────────────┐                               ┌─────────────────────┐
    │    CALL GRAPH       │                               │   HOT SYMBOL INDEX  │
    │    CONSTRUCTION     │◄──────────────────────────────│   (Invocation Freq) │
    │  (Static + Dynamic) │                               └─────────────────────┘
    └──────────┬──────────┘
               │
               ▼
    ┌─────────────────────────────────────────────────────────────────────────────┐
    │                         REACHABILITY ANALYSIS                                │
    │  Entry-point detection · path BFS · K4 lattice resolution · lattice state    │
    └──────────────────────────────────┬──────────────────────────────────────────┘
                                       │
    ┌──────────────────────────────────┼──────────────────────────────────────────┐
    ▼                                  ▼                                          ▼
┌─────────────────────┐     ┌─────────────────────┐     ┌─────────────────────────────┐
│  ADVISORY MATCHING  │     │   VEX STATEMENTS    │     │   EXCEPTION REGISTRY        │
│  Concelier          │     │   VexLens consensus │     │   Time-bounded waivers      │
│  (NVD/GHSA/OSV)     │     │   (OpenVEX)         │     │   (Exception store)         │
└──────────┬──────────┘     └──────────┬──────────┘     └─────────────┬───────────────┘
           └───────────────────────────┼───────────────────────────────┘
                                       ▼
                    ┌─────────────────────────────────────────┐
                    │      EVIDENCE-WEIGHTED SCORING (EWS)    │
                    │  Reachability · Runtime · VEX ·         │
                    │  Provenance · Policy  (weights → src)   │
                    └───────────────────┬─────────────────────┘
                                        ▼
                    ┌─────────────────────────────────────────┐
                    │           POLICY ENGINE                 │
                    │  K4 lattice evaluation + ordered gates  │
                    └───────────────────┬─────────────────────┘
                                        ▼
                    ┌─────────────────────────────────────────┐
                    │   VERDICT (PASS/WARN/FAIL/SKIP)         │
                    │   + explain trace + DSSE attestation    │
                    └─────────────────────────────────────────┘

The factor weights, lattice states, and gate set shown abbreviated above are detailed (as shapes + pointers) in the sections that follow. Do not treat the box labels as exact identifiers.

1. SBOM Generation Layer

1.1 Language analyzers

The Scanner runs one analyzer per language ecosystem. Each extracts package manifests, lockfiles, and dependency trees and emits packages keyed by purl (e.g. pkg:nuget/..., pkg:maven/..., pkg:npm/..., pkg:pypi/..., pkg:golang/..., pkg:cargo/..., pkg:composer/..., pkg:gem/...).

Shape: a set of *.Analyzers.Lang.<Ecosystem> plugins, each reading that ecosystem’s manifest/lockfile formats and producing purl-keyed components with transitive dependency edges. A Native analyzer covers ELF/PE/Mach-O binaries (see §4).

Authoritative source (do not re-enumerate here — the set changes):

1.2 OS / distribution analyzers

OS-level analyzers extract installed system packages with distribution-specific version semantics (APK 1.2.3-r4; DPKG epoch:upstream-debian; RPM NEVRA).

Shape: one analyzer per package database / packaging system, reading the on-disk package DB and emitting purl-keyed OS packages.

Authoritative source: src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS.* (Apk, Dpkg, Rpm, Pacman, Portage, Homebrew, Pkgutil, MacOsBundle, Windows.Chocolatey, Windows.Msi, Windows.WinSxS, …). See Scanner Architecture.

1.3 Surface analyzers

Surface analyzers extract metadata beyond packages: filesystem attributes, environment, secrets, SBOM validation, and capability detection (exec/network/crypto/etc.). Outputs are properties (property:fs/…, property:env/…) and findings (finding:secret/…).

Authoritative source: src/Scanner/__Libraries/StellaOps.Scanner.Surface* (Surface.FS, Surface.Env, Surface.Secrets, Surface.Validation) and StellaOps.Scanner.VulnSurfaces.

1.4 SBOM composition

Analyzer outputs are deduplicated, relationship-inferred, and composed into standardized SBOM formats, then wrapped for attestation.

Shape:

2. Runtime Observation Layer

2.1 Runtime agent architecture

The Signals module collects runtime observability (function-call and syscall events) and correlates them to container identity. On Linux this uses kernel-level probes (kprobe/kretprobe, tracepoint, uprobe) feeding a per-CPU ring buffer; a userspace collector polls events, resolves symbols, unwinds stacks, and tags each event with a container ID.

Shape (event records): a function-call event and a syscall event, each carrying a timestamp, container ID, pid/tid, the symbol or syscall identity, a stack trace, and latency. The exact struct fields belong to the agent contracts.

Authoritative source: src/Signals/StellaOps.Signals.RuntimeAgent/ (agent + ingest contracts, e.g. IRuntimeFactsIngest, RuntimeAgentBase, DotNetEventPipeAgent). See Signals Architecture.

2.2 Container observation (Zastava)

Zastava observes container lifecycle and runtime posture across Docker / containerd / CRI-O via the runtime API socket: it extracts image digest, tracks container IDs, records lifecycle transitions (create/start/stop), and evaluates runtime posture.

Shape: lifecycle events keyed by image digest + container ID, plus a RuntimePosture enum that ranks observation depth from “no observation / static-only” up to “full instrumentation” — each level setting the evidence ceiling that runtime can contribute to scoring. Enumerate the posture members and per-event fields from source, not here.

Authoritative source: the Zastava observer and posture enum under src/Signals/; see Signals Architecture and Reachability Drift Alert Flow.

2.3 Hot Symbol Index

Runtime observations are aggregated into a Hot Symbol Index used for reachability correlation: per (image digest, function) it tracks invocation counts, first/last observed timestamps, distinct callers, and a representative stack, keyed so the policy layer can ask “was a function in this vulnerable package ever executed, and how often?”

Shape: a PostgreSQL-backed index keyed on image digest + function (with a purl lookup path and a hot-path ranking by invocation count). The exact column set and indexes are owned by the persistence layer.

Authoritative source: model src/Signals/StellaOps.Signals/Models/HotSymbolIndex.cs; repository src/Signals/StellaOps.Signals/Persistence/IHotSymbolRepository.cs; schema in src/Signals/__Libraries/StellaOps.Signals.Persistence/Migrations/*.sql; HTTP surface src/Signals/StellaOps.Signals/Api/HotSymbolsController.cs.

3. Call Graph Analysis Layer

3.1 ReachGraph architecture

The ReachGraph module ingests SBOM + binary symbols + an advisory overlay (CVE → package → affected function, where known) into a property graph, then runs analytics (clustering / centrality) for impact analysis.

Shape (graph model):

Worked fragment (illustrative):

[EntryPoint:main.js] ──CALLS──► [Function:lodash.template]
       │                                  │
   ──ENTRY_FOR──► [Package:myapp@1.0] ◄──DEPENDS_ON── [Package:lodash@4.17]
                                                          ▲
                                       [CVE-2021-23337] ──┘ AFFECTS

Authoritative source: src/ReachGraph/StellaOps.ReachGraph.WebService and the graph contracts under src/Graph/StellaOps.Graph.Api/Contracts/. See Call Graph Analysis.

3.2 Reachability slice service

BFS path-finding answers reachability queries such as “Is CVE-X reachable from any entry point?” by traversing CALLS/IMPORTS edges from entry points to affected functions and returning the shortest path (or proving none exists).

Shape: slice queries over the graph — package slice, CVE slice, file slice, entry-point slice — each returning the set of paths (or absence of paths) relevant to that anchor. See Call Graph Analysis.

3.3 Reachability lattice (8-state)

Reachability is modeled as an 8-state lattice ordered by evidence strength, progressing from Unknown toward confirmed states as static and runtime evidence accumulate, with a top contested state for conflicting evidence. Each state maps to a VEX justification and a confidence band, which is how reachability feeds the EWS reachability factor (§5).

Shape: the lattice rises from Unknown through static states, then runtime states, to multi-source confirmed states, capped by a contested state; “more information” joins upward and conflicts surface as contested (aggregation-not-merge).

Authoritative source — use these enum members, not prose copies:

The two enums above serve different layers (core lattice vs. evidence contract). When you need exact state names or transition rules, read the enums — earlier revisions of this doc invented state names that do not exist in either source.

4. Binary Analysis Layer

4.1 Binary identity extraction

The BinaryIndex module detects executable format by magic bytes and extracts a stable build identity: ELF .note.gnu.build-id, PE CodeView GUID + age, Mach-O LC_UUID. It also captures architecture, linked libraries, and exported/imported symbols.

Shape: a BinaryIdentity record (build-id, format, arch, linked libraries, exported/imported symbols, sections). Read the field set from source rather than mirroring it.

Authoritative source: src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.Fingerprints/ (Models, Generators, Storage).

4.2 Function fingerprinting

Functions are fingerprinted with normalization so that semantically identical code matches across builds: a normalized basic-block hash (registers → r0,r1…, immediates → <addr>, opcodes preserved), a control-flow-graph structural hash, and a sorted string-references hash. These combine into a composite fingerprint alongside instruction count and cyclomatic complexity.

Shape: FunctionFingerprint { name, basicBlockHash, cfgHash, stringRefsHash, instructionCount, cyclomaticComplexity, compositeHash }, where compositeHash = SHA256(bbHash || cfgHash || strHash). Exact field names and hashing recipe live in source.

Authoritative source: src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.Fingerprints/Models/ (e.g. VulnFingerprint.cs) and Generators/.

4.3 Patch detection

The patch-diff engine compares a scanned binary’s per-function fingerprints against known-patched references and scores similarity (CFG similarity + block similarity) to decide patched / likely-patched / uncertain / likely-vulnerable. This drives backport detection — a package may declare a vulnerable version yet contain a backported fix, yielding a BACKPORTED_PATCH evidence type with a confidence that can justify NOT_AFFECTED despite the declared version.

Shape: similarity = weighted blend of CFG and basic-block similarity, bucketed by thresholds into a patched/vulnerable verdict with a confidence. The exact weights and threshold cutoffs are tuning parameters — read them from the matching engine source rather than this doc.

Authoritative source: src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.Fingerprints/Matching/ and Pipeline/.

5. Evidence-Weighted Scoring (EWS)

5.1 Scoring model

Each finding’s confidence is a weighted sum of normalized per-factor scores:

FinalScore = Σ (Weight_i × NormalizedScore_i)        Σ Weight_i = 1.0 ,  NormalizedScore_i ∈ [0,1]
   i ∈ { Reachability, Runtime, VEX, Provenance, Policy }

Factors (shape): five factors — Reachability (call-path strength), Runtime (observed execution), VEX (vendor statement consensus), Provenance (build / supply-chain attestation), Policy (exception / override status). Reachability carries the largest weight and Policy the smallest; the precise weights are configuration, not a doc constant.

Authoritative source — read the weights here, do not trust a copied table:

Per-factor normalization curves (e.g. how a lattice state or runtime observation maps to a [0,1] score) are owned by the normalizers in src/Signals/.../EvidenceWeightedScore/Normalizers/ and the policy confidence services. Treat any inline value tables as illustrative only.

5.2 Worked example (illustrative)

For a finding like CVE-2021-23337 in lodash@4.17.15, each factor is normalized to [0,1], multiplied by its weight, and summed; the total × 100 yields a confidence (e.g. a mid-60s “medium-high” score for a statically-reachable, unobserved, no-VEX, partially-attested finding with no exception). The arithmetic is mechanical — the inputs (lattice state, runtime hits, VEX disposition, provenance level, exception status) and the weights come from the sources in §5.1.

6. Policy Engine Decision Layer

6.1 K4 Belnap four-valued logic

Evidence is combined under Belnap’s four-valued logic so that “no evidence” and “conflicting evidence” are distinct from true/false:

            ⊤  Both (conflict: evidence says true AND false)
           ╱ ╲
          T   F   (True / False)
           ╲ ╱
            ⊥  Unknown (neither: no evidence either way)

The full JOIN/MEET truth tables are implemented in source; do not maintain a copy here.

Authoritative source: src/Policy/__Libraries/StellaOps.Policy/TrustLattice/K4Lattice.cs (and the determinization variant src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/K4Lattice.cs). See Policy Architecture.

6.2 Policy gates

Findings pass through an ordered set of gates; each gate can pass, downgrade to WARN, skip, or fail a finding based on its configuration. Representative gates: a minimum-confidence gate (skip below an evaluate threshold, WARN below a fail threshold), a reachability-requirement gate (require a minimum lattice state for high/critical severities), and an unknowns-budget gate (fail when unparseable/unknown packages exceed a count or percentage budget).

Shape: each gate is an IPolicyGate-style component with its own typed options; gates are registered and ordered by a selector/registry, and their thresholds are configuration. The live gate set is larger than the three examples above and evolves — enumerate it from the directory, not from this doc.

Authoritative source: src/Policy/__Libraries/StellaOps.Policy/Gates/ (e.g. MinimumConfidenceGate.cs, ReachabilityRequirementGate.cs, UnknownsBudgetGate.cs, CvssThresholdGate.cs, EvidenceFreshnessGate.cs, SbomPresenceGate.cs, SignatureRequiredGate.cs, VexProofGate.cs, FixChainGate.cs, …) plus the gate registry/selector in the same directory and src/Policy/StellaOps.Policy.Engine/Gates/.

A SeverityThresholdGate appearing in earlier revisions of this doc has no source counterpart; severity thresholds are applied via the CVSS/threshold gates above. Verify gate names against the directory.

6.3 Complete decision flow

   INPUTS                         PROCESSING                     OUTPUT
   ──────                         ──────────                     ──────
 SBOM ┐
 Advisories ┼──► FINDING MATCHER  (package × advisory → findings)
 VEX ┘                 │
                       ▼
 Reachability ──► EVIDENCE AGGREGATOR
 Runtime ──────►   • collect all evidence sources
                   • resolve conflicts via K4 logic (§6.1)
                   • compute EWS confidence (§5)
                       │
                       ▼
 Exceptions ┐
 Unknowns ──┼──► GATE EVALUATOR  (ordered gates, §6.2)
 Policy ────┘          │
                       ▼
                  VERDICT BUILDER ──► VERDICT (PASS/WARN/FAIL/SKIP)
                    • any FAIL → FAIL; any WARN → WARN; else PASS    + explain trace
                    • emit explain trace + DSSE attestation          + DSSE attestation

See Policy Evaluation Flow for the full sequence and Risk Score Dashboard Flow for downstream aggregation.

7. Data Contracts (shape + pointers)

The pipeline’s inputs and outputs are typed contracts. Their shapes are summarized below; the canonical field-by-field definitions live in source and in the OpenAPI/JSON-schema artifacts for each service — do not treat the sketches below as the schema of record.

Inputs:

Outputs:

Authoritative source: