BinaryIndex — Architecture
Audience: Scanner and Concelier engineers, plus reviewers evaluating how Stella Ops detects vulnerable code at the binary level.
BinaryIndex is the Stella Ops module that maps binary identity (not package version strings) to known-vulnerable code. It backs binary-first detection during scans, supplying evidence that survives backports, custom builds, stripped metadata, and static linking.
Ownership: Scanner Guild + Concelier Guild Status: ACTIVE Version: 1.7.1 Related: High-Level Architecture, Scanner Architecture, Concelier Architecture, Hybrid Diff Stack
1. Overview
The BinaryIndex module provides a vulnerable binaries database that enables detection of vulnerable code at the binary level, independent of package metadata. This addresses a critical gap in vulnerability scanning: package version strings can lie (backports, custom builds, stripped metadata), but binary identity doesn’t lie.
1.1 Problem Statement
Traditional vulnerability scanners rely on package version matching, which fails in several scenarios:
- Backported patches - Distros backport security fixes without changing upstream version
- Custom/vendored builds - Binaries compiled from source without package metadata
- Stripped binaries - Debug info and version strings removed
- Static linking - Vulnerable library code embedded in final binary
- Container base images - Distroless or scratch images with no package DB
1.2 Solution: Binary-First Vulnerability Detection
BinaryIndex provides three tiers of binary identification:
| Tier | Method | Precision | Coverage |
|---|---|---|---|
| A | Package/version range matching | Medium | High |
| B | Build-ID/hash catalog (exact binary identity) | High | Medium |
| C | Function fingerprints (CFG/basic-block hashes) | Very High | Targeted |
Tier B catalog lookups are performed over three identity dimensions:
BuildId + BuildIdTypeBinaryKeyFileSha256
1.2.1 Tier C Fingerprint Runtime Contract
Tier C function fingerprint matching is implemented as deterministic offline-safe byte-window analysis:
IFunctionFingerprintExtractor(StellaOps.BinaryIndex.Builders) derives basic-block hashes, CFG hash, string-reference hashes, constants, and call-targets from bounded binary byte windows (not seed-only synthetic placeholders).SignatureMatcher/ISignatureMatcher(StellaOps.BinaryIndex.Analysis) applies configurable weighted matching across basic-block, CFG, string-reference, and constant signals.- Golden CVE fixtures include required high-impact package coverage for
openssl,glibc,zlib, andcurl, and verification checks enforce this coverage during feature rechecks.
1.2.2 Analyzer Source Metadata Contract
BinaryIndex owns the local analyzer source metadata contract used by evidence-producing tools such as stella attest fixchain. The contract type lives in StellaOps.BinaryIndex.Contracts.Analyzers and uses schema version stellaops.binaryindex.analyzer-source-metadata.v1.
Metadata fields:
| Field | Required | Description |
|---|---|---|
schemaVersion | yes | Must be stellaops.binaryindex.analyzer-source-metadata.v1. |
analyzerName | yes | Stable analyzer name, for example StellaOps.BinaryIndex. |
analyzerVersion | yes | Analyzer version that produced the diff evidence. |
sourceDigest | yes | Real sha256:<64 hex> digest of the analyzer source bundle or source tree used for the evidence run. |
sourceRef | no | Local bundle URI or repository/source reference. |
buildRecipeDigest | no | sha256:<64 hex> digest of the analyzer build or packaging recipe, when available. |
Example:
{
"schemaVersion": "stellaops.binaryindex.analyzer-source-metadata.v1",
"analyzerName": "StellaOps.BinaryIndex",
"analyzerVersion": "2.1.0",
"sourceDigest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"sourceRef": "file:///opt/stellaops/binaryindex/source.tar.zst",
"buildRecipeDigest": "sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"
}
Trust rule:
- Consumers must hash the exact metadata file bytes and compare that digest to a trusted
sha256:<64 hex>metadata digest pin before accepting any field from the metadata. - If the metadata bytes do not match the trusted pin, the metadata is untrusted and consumers must fail closed.
- If
sourceDigestis missing, malformed, or conflicts with an explicit operator-supplied analyzer source digest, consumers must fail closed. - Consumers must not synthesize
sha256:unknownor any placeholder analyzer source identity.
1.3 Module Scope
In Scope:
- Binary identity extraction (Build-ID, PE CodeView GUID, Mach-O UUID)
- Binary-to-advisory mapping database
- Fingerprint storage and matching engine
- Fix index for patch-aware backport handling
- Integration with Scanner.Worker for binary lookup
Out of Scope:
- Binary disassembly/analysis (provided by Scanner.Analyzers.Native)
- Runtime binary tracing (provided by Zastava)
- SBOM generation (provided by Scanner)
1.4 Pluginized Analyzer and Data-Pack Boundary
BinaryIndex participates in pluginized compose through mounted roots. As of commit ee5bd3f9d0 (2026-06-07, WS1 Batch-1) the runtime slice is now a real signed mounted-bundle analyzer loader, not just a classification/probe boundary: MountedDisassemblyRuntimePluginLoader admits the Iced / B2R2 disassembler bundles through a fail-closed admission chain and registers the survivors in the disassembly plugin registry (previously disassembly was never registered in DI, so it was dead in prod). See Mounted disassembly runtime plugin loader below for the admission chain, bundle layout, and the docker-compose.plugins.binaryindex.yml overlay.
Canonical compose roots for binaryindex-web:
| Purpose | Host path | Container path |
|---|---|---|
| Executable analyzer bundle | devops/plugins/binaryindex/recommended/, devops/plugins/binaryindex/harness/ | /app/plugins/binaryindex/recommended, /app/plugins/binaryindex/harness |
| Operator config/data-pack registry | devops/etc/plugins/binaryindex/ | /app/etc/plugins/binaryindex |
| Analyzer trust roots | devops/etc/certificates/trust-roots/plugins/binaryindex/ | /app/trust-roots/plugins/binaryindex |
| Probe scratch | named volume | /var/lib/stellaops/plugin-scratch/binaryindex |
binaryindex-web also mounts Router/Messaging bundle roots in base compose for shared Router registration transport: devops/plugins/router/base -> /app/plugins/router/base, devops/plugins/messaging/base -> /app/plugins/messaging/base, and devops/etc/certificates/trust-roots/plugins/router -> /app/trust-roots/plugins/router. These read-only mounts are platform transport infrastructure, not BinaryIndex analyzer bundles.
Runtime status is exposed on GET /internal/plugins/status and POST /internal/plugins/probe. The probe report currently classifies:
| Classification | Catalog IDs | Current behavior |
|---|---|---|
| Host core | binaryindex.identity.build-id, binaryindex.parser.elf, binaryindex.parser.pe, binaryindex.parser.macho, binaryindex.fingerprint.stable-hash | Reported from the host catalog and expected to work without optional plugin mounts. |
| Optional executable analyzer plugin | binaryindex.disassembly.iced, binaryindex.disassembly.b2r2, binaryindex.disassembly.ghidra, binaryindex.corpus.builder | Iced and B2R2 are signed bundles for recommended/harness profiles and are admitted by MountedDisassemblyRuntimePluginLoader when mounted (commit ee5bd3f9d0); the shared recommended, harness, and BinaryIndex-only overlays now bind the loader profile and trust-root path explicitly. The probe reporter merges live loaded/rejected loader statuses into these rows. When no bundle is mounted they report notMounted (non-required). Ghidra and corpus-builder remain notMounted (no executable bundle is produced for them yet). The 2026-06-11 live overlay HTTP probe verified Iced and B2R2 mounted/admitted/loaded/responded. |
| Data pack | binaryindex.datapack.golden-corpus, binaryindex.datapack.known-build-catalog, binaryindex.datapack.signature-pack, binaryindex.datapack.threshold-profile | Non-executable config/content; these rows are disabled, non-required, and must not execute pack-provided code. |
The compose overlays mount recommended and harness analyzer roots for binaryindex-web; base behavior is host-owned and has no mounted executable bundle. The loader that validates manifests/checksums/signatures before code load landed in commit ee5bd3f9d0 (see §1.4.2); the 2026-06-11 live overlay-up HTTP probe proved the mounted signed Iced/B2R2 rows report through /internal/plugins/probe.
The base runtime path was proven on 2026-06-07: mounted-mode stellaops/binaryindex-web:dev pruned plugin payloads from /app, the image audit passed, the recreated container was healthy, and POST /internal/plugins/probe returned isComplete=true with five required host-core rows responded and no blockers.
devops/build/package-runtime-plugins.ps1 produces signed recommended/harness bundles for binaryindex.disassembly.iced and binaryindex.disassembly.b2r2 when invoked with -Module binaryindex -Profile recommended|harness -SignBinaryIndexBundles and a Cosign key pair or -UseOfflineDevSigner. It does not produce Ghidra or corpus-builder executable bundles.
StellaOps.Symbols.Server does not expose /internal/plugins/* and has no executable analyzer plugin contract, so compose overlays do not mount BinaryIndex analyzer bundle roots into the symbols service.
As of the 2026-06-07 mounted-runtime slice, StellaOps.BinaryIndex.WebService does not compile the optional StellaOps.BinaryIndex.Disassembly.B2R2 analyzer implementation or the Router/Messaging transport implementation payload directly. The B2R2/multi-architecture lifter and the Iced disassembler are now loaded from mounted signed bundles by MountedDisassemblyRuntimePluginLoader (commit ee5bd3f9d0); they report notMounted only when no bundle is mounted, and the probe catalog reflects the live loader status otherwise. StellaOps.BinaryIndex.Worker still compiles StellaOps.BinaryIndex.Builders for its reproducible-build job; that worker path is a later corpus-builder loader split, not closed by this WebService source-coupling slice.
1.4.2 Mounted disassembly runtime plugin loader (signed bundle admission)
MountedDisassemblyRuntimePluginLoader (src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.Disassembly/MountedDisassemblyRuntimePluginLoader.cs, commit ee5bd3f9d0) is the BinaryIndex signed mounted-bundle loader. It is wired into StellaOps.BinaryIndex.WebService/Program.cs via AddMountedDisassemblyRuntimePlugins(builder.Configuration), which captures the resulting catalog so the /internal/plugins/* endpoints reflect the live registry. Before this commit, IDisassemblyPlugin was never registered in DI — disassembly was dead in production. The loader is fail-closed: every reject path registers zero plugins (asserted by the 24 loader tests, 23 admission + 1 live forcing-function against the committed signed bundles), and survivors are de-duplicated by Capabilities.PluginId.
Admission chain (BinaryIndexAnalyzerBundleAdmission, per bundle directory):
- Manifest binding —
manifest.jsonmust declaremodule=binaryindex, the analyzer contract versionbinaryindex.analyzer.v1(the producer-stamped manifestschemaVersionisstellaops.binaryindex.analyzer-bundle.v1), the matching profile, and a well-formed assembly descriptor whoseanalyzerIdmatches the static probe catalog id. - Payload + per-DLL SHA-256 + path-traversal guard — the payload digest is bound and each shipped DLL is hashed; rooted/escaping assembly paths are rejected.
- Detached RSA-PKCS1-SHA256 verification —
<assembly>.sigis verified viaOfflineDevRsaSha256PluginVerifier(AllowUnsigned=false) against the mounted BinaryIndex trust root. Only then is the entry type activated viaActivatorUtilities.
Configuration (BinaryIndex:RuntimePlugins):
| Key | Default | Purpose |
|---|---|---|
BinaryIndex:RuntimePlugins:RootPath | /app/plugins/binaryindex | Root containing profile directories. |
BinaryIndex:RuntimePlugins:Profile | recommended | Profile directory to load (<RootPath>/<Profile>). |
BinaryIndex:RuntimePlugins:TrustRootPath | /app/etc/certificates/trust-roots/plugins/binaryindex/cosign.pub | Trust-root public key the detached-signature verifier checks against. Compose overlays set this explicitly to the mounted /app/trust-roots/plugins/binaryindex/cosign.pub. |
Bundle / trust-root layout (recommended / harness profiles):
| Purpose | Host path | Container path |
|---|---|---|
| Signed analyzer bundle | devops/plugins/binaryindex/recommended/<bundle>/ (manifest.json + <assembly>.dll + <assembly>.dll.sig) | /app/plugins/binaryindex/recommended/<bundle> |
| Operator config/data-pack registry | devops/etc/plugins/binaryindex/ | /app/etc/plugins/binaryindex |
| Analyzer trust root | devops/etc/certificates/trust-roots/plugins/binaryindex/cosign.pub | /app/trust-roots/plugins/binaryindex |
| Probe scratch | named volume binaryindex-plugin-scratch | /var/lib/stellaops/plugin-scratch/binaryindex |
Note: all BinaryIndex analyzer overlays set
BinaryIndex__RuntimePlugins__TrustRootPathexplicitly. If an operator writes a custom overlay, that value must match the trust-root volume mount or every bundle is rejected fail-closed.
Bundle producer: devops/build/package-runtime-plugins.ps1 -Module binaryindex -Profile recommended|harness -SignBinaryIndexBundles with a Cosign key pair, or -UseOfflineDevSigner for the offline-dev RSA/SHA-256 root. It produces signed Iced + B2R2 bundles only (no Ghidra or corpus-builder executable bundles). The generated cosign.pub is git-ignored under the binaryindex trust-root directory.
Opt-in compose overlay: devops/compose/docker-compose.plugins.binaryindex.yml layers read-only mounts of the recommended bundle root + trust root onto binaryindex-web and binds BinaryIndex:RuntimePlugins to that mounted material. The shared docker-compose.plugins.recommended.yml and docker-compose.plugins.harness.yml profiles do the same for their respective profile roots. Apply the module overlay with COMPOSE_EXTRA_FILES=docker-compose.plugins.binaryindex.yml ./scripts/compose-cli.ps1 up. Read-only mounts only; the overlay never masks /app/plugins.
Probe / status surface: BinaryIndexAnalyzerPluginProbeReporter merges the live loaded/rejected loader statuses into the probe catalog, so GET /internal/plugins/status and POST /internal/plugins/probe reflect the registry (this fixed the silent-green probe — previously the rows could not show a loaded optional analyzer).
Live runtime probe passed on 2026-06-11. With
COMPOSE_EXTRA_FILES=docker-compose.plugins.binaryindex.yml, the recreatedbinaryindex-webcontainer became healthy, kept the basestellaops/binaryindex-web:devimage, mounted analyzer/config/trust roots read-only, andPOST /internal/plugins/probereported Iced and B2R2 mounted/admitted/loaded/responded with no blockers.
2. Architecture
2.1 System Context
┌──────────────────────────────────────────────────────────────────────────┐
│ External Systems │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Distro Repos │ │ Debug Symbol │ │ Upstream Source │ │
│ │ (Debian, RPM, │ │ Servers │ │ (GitHub, etc.) │ │
│ │ Alpine) │ │ (debuginfod) │ │ │ │
│ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ │
└───────────│─────────────────────│─────────────────────│──────────────────┘
│ │ │
v v v
┌──────────────────────────────────────────────────────────────────────────┐
│ BinaryIndex Module │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Corpus Ingestion Layer │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ DebianCorpus │ │ RpmCorpus │ │ AlpineCorpus │ │ │
│ │ │ Connector │ │ Connector │ │ Connector │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ v │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Processing Layer │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ BinaryFeature│ │ FixIndex │ │ Fingerprint │ │ │
│ │ │ Extractor │ │ Builder │ │ Generator │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ v │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Storage Layer │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ PostgreSQL │ │ RustFS │ │ Valkey │ │ │
│ │ │ (binaries │ │ (fingerprint │ │ (lookup │ │ │
│ │ │ schema) │ │ blobs) │ │ cache) │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ v │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Query Layer │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ IBinaryVulnerabilityService │ │ │
│ │ │ - LookupByBuildIdAsync(buildId) │ │ │
│ │ │ - LookupByFingerprintAsync(fingerprint) │ │ │
│ │ │ - LookupBatchAsync(identities) │ │ │
│ │ │ - GetFixStatusAsync(distro, release, sourcePkg, cve) │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
│
v
┌──────────────────────────────────────────────────────────────────────────┐
│ Consuming Modules │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Scanner.Worker │ │ Policy Engine │ │ Findings Ledger │ │
│ │ (binary lookup │ │ (evidence in │ │ (match records) │ │
│ │ during scan) │ │ proof chain) │ │ │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
2.2 Component Breakdown
2.2.1 Corpus Connectors
Plugin-based connectors that ingest binaries from distribution repositories.
public interface IBinaryCorpusConnector
{
string ConnectorId { get; }
ImmutableArray<string> SupportedDistros { get; }
Task<CorpusSnapshot> FetchSnapshotAsync(CorpusQuery query, CancellationToken ct = default);
IAsyncEnumerable<PackageInfo> ListPackagesAsync(CorpusSnapshot snapshot, CancellationToken ct = default);
IAsyncEnumerable<ExtractedBinary> ExtractBinariesAsync(PackageInfo pkg, CancellationToken ct = default);
}
Implementations (one per distro library StellaOps.BinaryIndex.Corpus.{Debian,Rpm,Alpine}):
DebianCorpusConnector- Debian/Ubuntu packages + debuginfoRpmCorpusConnector- RHEL/Fedora/CentOS + SRPMAlpineCorpusConnector- Alpine APK + APKBUILD
Each connector pulls package bytes/indexes through a *MirrorPackageSource:
DebianMirrorPackageSource- Debian package index/payload mirror sourceAlpineMirrorPackageSource- Alpine package index/payload mirror source with cache fallbackRpmMirrorPackageSource- RPM primary metadata/package mirror source with cache fallback
2.2.2 Binary Feature Extractor
Extracts identity and features from binaries. Reuses existing Scanner.Analyzers.Native capabilities.
public interface IBinaryFeatureExtractor
{
bool CanExtract(Stream stream);
Task<BinaryIdentity> ExtractIdentityAsync(Stream stream, CancellationToken ct = default);
Task<BinaryMetadata> ExtractMetadataAsync(Stream stream, CancellationToken ct = default);
}
// Full identity (StellaOps.BinaryIndex.Core.Models). A single BuildId + BuildIdType
// pair carries the ELF GNU Build-ID, PE CodeView, or Mach-O UUID (BuildIdType ∈
// {gnu-build-id, pe-cv, macho-uuid}) — there are no separate PeCodeViewGuid/MachoUuid
// fields.
public sealed record BinaryIdentity
{
public Guid Id { get; init; }
public required string BinaryKey { get; init; } // build_id || file_sha256
public string? BuildId { get; init; }
public string? BuildIdType { get; init; }
public required string FileSha256 { get; init; }
public string? TextSha256 { get; init; }
public string? Blake3Hash { get; init; }
public required BinaryFormat Format { get; init; } // enum: Elf | Pe | Macho
public required string Architecture { get; init; }
public string? OsAbi { get; init; }
public BinaryType? Type { get; init; }
public bool IsStripped { get; init; }
public Guid? FirstSeenSnapshotId { get; init; }
public Guid? LastSeenSnapshotId { get; init; }
}
// Lightweight metadata (no expensive hashing): Format, Architecture, BuildId,
// BuildIdType, OsAbi, Type, IsStripped, plus PE-specific (PeTimestamp, IsPe32Plus)
// and Mach-O-specific (Is64Bit, IsUniversalBinary) fields.
public sealed record BinaryMetadata { /* ... */ }
2.2.3 Fix Index Builder
Builds the patch-aware CVE fix index from distro sources. The builder is ingestion only (per-distro build methods streaming FixEvidence); fix-status queries are served by IBinaryVulnerabilityService.GetFixStatusAsync (§2.2.6), not by this interface.
public interface IFixIndexBuilder
{
IAsyncEnumerable<FixEvidence> BuildDebianIndexAsync(DebianFixIndexRequest request, CancellationToken ct = default);
IAsyncEnumerable<FixEvidence> BuildAlpineIndexAsync(AlpineFixIndexRequest request, CancellationToken ct = default);
IAsyncEnumerable<FixEvidence> BuildRpmIndexAsync(RpmFixIndexRequest request, CancellationToken ct = default);
}
// Per-distro requests carry the raw source artifacts the parsers consume:
// DebianFixIndexRequest : Distro, Release, SourcePkg, Changelog?, Patches?[], Version?, SnapshotId?
// AlpineFixIndexRequest : Release, SourcePkg, ApkBuild, SnapshotId? (Distro is always "alpine")
// RpmFixIndexRequest : Distro, Release, SourcePkg, SpecContent, SnapshotId?
// Shared fix enums (StellaOps.BinaryIndex.Core.Models):
public enum FixState { Fixed, Vulnerable, NotAffected, Wontfix, Unknown }
public enum FixMethod { SecurityFeed, Changelog, PatchHeader, UpstreamPatchMatch }
The
binaries.cve_fix_index.methodcolumn persistsFixMethodas the stringssecurity_feed | changelog | patch_header | upstream_match(theupstream_matchcolumn value corresponds to theUpstreamPatchMatchenum case).
2.2.4 Fingerprint Generator
Generates function-level fingerprints for vulnerable code detection.
Each generator produces a single FingerprintAlgorithm. Callers pick a generator by its Algorithm and feed one function/code window at a time via FingerprintInput:
public interface IVulnFingerprintGenerator
{
FingerprintAlgorithm Algorithm { get; }
Task<FingerprintOutput> GenerateAsync(FingerprintInput input, CancellationToken ct = default);
bool CanProcess(FingerprintInput input);
}
// FingerprintInput : BinaryData (byte[]), Architecture, CveId, Component (all required),
// plus optional FunctionName, BaseAddress, SourceFile, SourceLine, Purl.
// FingerprintOutput : Hash (byte[]) + algorithm-specific metadata.
The persisted fingerprint algorithm column (binaries.vulnerable_fingerprints.algorithm) accepts basic_block | cfg | control_flow_graph | string_refs | combined.
2.2.5 Semantic Analysis Library
Library:
StellaOps.BinaryIndex.SemanticSprint: 20260105_001_001_BINDEX - Semantic Diffing Phase 1
The Semantic Analysis Library extends fingerprint generation with IR-level semantic matching, enabling detection of semantically equivalent code despite compiler optimizations, instruction reordering, and register allocation differences.
Key Insight: Traditional instruction-level fingerprinting loses accuracy on optimized binaries by ~15-20%. Semantic analysis lifts to B2R2’s Intermediate Representation (LowUIR), extracts key-semantics graphs, and uses graph hashing for similarity computation.
2.2.5.1 Architecture
Binary Input
│
v
B2R2 Disassembly → Raw Instructions
│
v
IR Lifting Service → LowUIR Statements
│
v
Semantic Graph Extractor → Key-Semantics Graph (KSG)
│
v
Graph Fingerprinting → Semantic Fingerprint
│
v
Semantic Matcher → Similarity Score + Deltas
2.2.5.2 Core Components
IR Lifting Service (IIrLiftingService)
Lifts disassembled instructions to B2R2 LowUIR:
public interface IIrLiftingService
{
Task<LiftedFunction> LiftToIrAsync(
IReadOnlyList<DisassembledInstruction> instructions,
string functionName,
LiftOptions? options = null,
CancellationToken ct = default);
}
public sealed record LiftedFunction(
string Name,
ImmutableArray<IrStatement> Statements,
ImmutableArray<IrBasicBlock> BasicBlocks);
Semantic Graph Extractor (ISemanticGraphExtractor)
Extracts key-semantics graphs capturing data dependencies, control flow, and memory operations:
public interface ISemanticGraphExtractor
{
Task<KeySemanticsGraph> ExtractGraphAsync(
LiftedFunction function,
GraphExtractionOptions? options = null,
CancellationToken ct = default);
}
public sealed record KeySemanticsGraph(
string FunctionName,
ImmutableArray<SemanticNode> Nodes,
ImmutableArray<SemanticEdge> Edges,
GraphProperties Properties);
public enum SemanticNodeType { Compute, Load, Store, Branch, Call, Return, Phi }
public enum SemanticEdgeType { DataDependency, ControlDependency, MemoryDependency }
Semantic Fingerprint Generator (ISemanticFingerprintGenerator)
Generates semantic fingerprints using Weisfeiler-Lehman graph hashing:
public interface ISemanticFingerprintGenerator
{
Task<SemanticFingerprint> GenerateAsync(
KeySemanticsGraph graph,
SemanticFingerprintOptions? options = null,
CancellationToken ct = default);
}
public sealed record SemanticFingerprint(
string FunctionName,
string GraphHashHex, // WL graph hash (SHA-256)
string OperationHashHex, // Normalized operation sequence hash
string DataFlowHashHex, // Data dependency pattern hash
int NodeCount,
int EdgeCount,
int CyclomaticComplexity,
ImmutableArray<string> ApiCalls,
SemanticFingerprintAlgorithm Algorithm);
Semantic Matcher (ISemanticMatcher)
Computes semantic similarity with weighted components:
public interface ISemanticMatcher
{
Task<SemanticMatchResult> MatchAsync(
SemanticFingerprint a,
SemanticFingerprint b,
MatchOptions? options = null,
CancellationToken ct = default);
Task<SemanticMatchResult> MatchWithDeltasAsync(
SemanticFingerprint a,
SemanticFingerprint b,
MatchOptions? options = null,
CancellationToken ct = default);
}
public sealed record SemanticMatchResult(
decimal Similarity, // 0.00-1.00
decimal GraphSimilarity,
decimal OperationSimilarity,
decimal DataFlowSimilarity,
decimal ApiCallSimilarity,
MatchConfidence Confidence);
2.2.5.3 Algorithm Details
Weisfeiler-Lehman Graph Hashing:
- 3 iterations of label propagation
- SHA-256 for final hash computation
- Deterministic node ordering via canonical sort
Similarity Weights (Default):
| Component | Weight |
|---|---|
| Graph Hash | 0.35 |
| Operation Hash | 0.25 |
| Data Flow Hash | 0.25 |
| API Calls | 0.15 |
2.2.5.4 Integration Points
The semantic library integrates with existing BinaryIndex components:
DeltaSignatureGenerator Extension:
// Optional semantic services via constructor injection
services.AddDeltaSignaturesWithSemantic();
// Extended SymbolSignature with semantic properties
public sealed record SymbolSignature
{
// ... existing properties ...
public string? SemanticHashHex { get; init; }
public ImmutableArray<string> SemanticApiCalls { get; init; }
}
PatchDiffEngine Extension:
// SemanticWeight in HashWeights
public decimal SemanticWeight { get; init; } = 0.2m;
// FunctionFingerprint extended with semantic fingerprint
public SemanticFingerprint? SemanticFingerprint { get; init; }
Patch Diff Result Storage Contract:
IDiffResultStoreproduction implementations must persistPatchDiffResultrecords durably with deterministic, content-addressed IDs.- The volatile content-addressed store used by unit tests lives under
src/BinaryIndex/__Tests/**. - Re-storing an identical result yields the same ID (idempotent storage), while content changes produce a different ID.
- Rename detection and result-store persistence behaviors are covered by
StellaOps.BinaryIndex.Diff.Testsunit suites.
2.2.5.5 Test Coverage
| Category | Tests | Coverage |
|---|---|---|
| Unit Tests (IR lifting, graph extraction, hashing) | 53 | Core algorithms |
| Integration Tests (full pipeline) | 9 | End-to-end flow |
| Golden Corpus (compiler variations) | 11 | Register allocation, optimization, compiler variants |
| Benchmarks (accuracy, performance) | 7 | Baseline metrics |
2.2.5.6 Current Baselines
Note: Baselines reflect foundational implementation; accuracy improves as semantic features mature.
| Metric | Baseline | Target |
|---|---|---|
| Similarity (register allocation variants) | ≥0.55 | ≥0.85 |
| Overall accuracy | ≥40% | ≥70% |
| False positive rate | <10% | <5% |
| P95 fingerprint latency | <100ms | <50ms |
2.2.5.7 B2R2 LowUIR Adapter
The B2R2LowUirLiftingService implements IIrLiftingService using B2R2’s native lifting capabilities. This provides cross-platform IR representation for semantic analysis.
Key Components:
public sealed class B2R2LowUirLiftingService : IIrLiftingService
{
// Lifts to B2R2 LowUIR and maps to Stella IR model
public Task<LiftedFunction> LiftToIrAsync(
IReadOnlyList<DisassembledInstruction> instructions,
string functionName,
LiftOptions? options = null,
CancellationToken ct = default);
}
Supported ISAs:
- Intel (x86-32, x86-64)
- ARM (ARMv7, ARMv8/ARM64)
- MIPS (32/64)
- RISC-V (64)
- PowerPC, SPARC, SH4, AVR, EVM
IR Statement Mapping:
| B2R2 LowUIR | Stella IR Kind |
|---|---|
| Put | IrStatementKind.Store |
| Store | IrStatementKind.Store |
| Get | IrStatementKind.Load |
| Load | IrStatementKind.Load |
| BinOp | IrStatementKind.BinaryOp |
| UnOp | IrStatementKind.UnaryOp |
| Jmp | IrStatementKind.Jump |
| CJmp | IrStatementKind.ConditionalJump |
| InterJmp | IrStatementKind.IndirectJump |
| Call | IrStatementKind.Call |
| SideEffect | IrStatementKind.SideEffect |
Determinism Guarantees:
- Statements ordered by block address (ascending)
- Blocks sorted by entry address (ascending)
- Consistent IR IDs across identical inputs
- InvariantCulture used for all string formatting
2.2.5.8 B2R2 Lifter Pool
The B2R2LifterPool provides bounded pooling and warm preload for B2R2 lifting units to reduce per-call allocation overhead.
Configuration (B2R2LifterPoolOptions):
| Option | Default | Description |
|---|---|---|
MaxPoolSizePerIsa | 4 | Maximum pooled lifters per ISA |
EnableWarmPreload | true | Preload lifters at startup |
WarmPreloadIsas | [“intel-64”, “intel-32”, “armv8-64”, “armv7-32”] | ISAs to warm |
AcquireTimeout | 5s | Timeout for acquiring a lifter |
Pool Statistics:
TotalPooledLifters: Lifters currently in poolTotalActiveLifters: Lifters currently in useIsWarm: Whether pool has been warmedIsaStats: Per-ISA pool and active counts
Usage:
using var lifter = _lifterPool.Acquire(isa);
var stmts = lifter.LiftingUnit.LiftInstruction(address);
// Lifter automatically returned to pool on dispose
2.2.5.9 Function IR Cache
The FunctionIrCacheService provides Valkey-backed caching for computed semantic fingerprints to avoid redundant IR lifting and graph hashing.
Cache Key Structure:
(isa, b2r2_version, normalization_recipe, canonical_ir_hash)
Configuration (FunctionIrCacheOptions):
| Option | Default | Description |
|---|---|---|
KeyPrefix | “stellaops:binidx:funccache:” | Valkey key prefix |
CacheTtl | 4h | TTL for cached entries |
MaxTtl | 24h | Maximum TTL |
Enabled | true | Whether caching is enabled |
B2R2Version | “0.9.1” | B2R2 version for cache key |
NormalizationRecipeVersion | “v1” | Recipe version for cache key |
Cache Entry (CachedFunctionFingerprint):
FunctionAddress,FunctionNameSemanticFingerprint: The computed fingerprintIrStatementCount,BasicBlockCountComputedAtUtc: ISO-8601 timestampB2R2Version,NormalizationRecipe
Invalidation Rules:
- Cache entries expire after
CacheTtl(default 4h) - Changing B2R2 version or normalization recipe results in cache misses
- Manual invalidation via
RemoveAsync()
Statistics:
- Hits, Misses, Evictions
- Hit Rate
- Enabled status
2.2.5.10 Ops Endpoints
BinaryIndex exposes operational endpoints for health, benchmarking, cache monitoring, and configuration visibility.
| Endpoint | Method | Description |
|---|---|---|
/api/v1/ops/binaryindex/health | GET | Health status with lifter warmness, cache availability |
/api/v1/ops/binaryindex/bench/run | POST | Run benchmark, return latency stats |
/api/v1/ops/binaryindex/cache | GET | Function IR cache hit/miss statistics |
/api/v1/ops/binaryindex/config | GET | Effective lifter-pool + function-cache configuration (see §7.3.1) |
For local/offline startup, StellaOps.BinaryIndex.WebService keeps the same runtime truthfulness contract as live deployments: PostgreSQL is required for binary vulnerability, golden-set, and delta-signature state, and Redis is required for the authoritative resolution cache. Deterministic in-memory helpers are limited to the WebService test project and are not registered by the production host.
Health Response:
{
"status": "healthy",
"timestamp": "2026-01-14T12:00:00Z",
"lifterStatus": "warm",
"lifterWarm": true,
"lifterPoolStats": { "intel-64": 4, "armv8-64": 2 },
"cacheStatus": "enabled",
"cacheEnabled": true
}
Determinism Constraints:
- All timestamps in ISO-8601 UTC format
- ASCII-only output
- Deterministic JSON key ordering
- Secrets/credentials redacted from config endpoint
2.2.6 Binary Vulnerability Service
Main query interface for consumers, defined in StellaOps.BinaryIndex.Core/Services/IBinaryVulnerabilityService.cs. The interface covers identity lookup, fix-index queries, fingerprint matching, DeltaSig (delta signature) matching, and corpus-based function identification:
public interface IBinaryVulnerabilityService
{
// Identity (Build-ID / hashes)
Task<ImmutableArray<BinaryVulnMatch>> LookupByIdentityAsync(
BinaryIdentity identity, LookupOptions? options = null, CancellationToken ct = default);
Task<ImmutableDictionary<string, ImmutableArray<BinaryVulnMatch>>> LookupBatchAsync(
IEnumerable<BinaryIdentity> identities, LookupOptions? options = null, CancellationToken ct = default);
// Patch-aware fix index
Task<FixStatusResult?> GetFixStatusAsync(
string distro, string release, string sourcePkg, string cveId, CancellationToken ct = default);
Task<ImmutableDictionary<string, FixStatusResult>> GetFixStatusBatchAsync(
string distro, string release, string sourcePkg, IEnumerable<string> cveIds, CancellationToken ct = default);
// Function fingerprint matching (byte[] fingerprints)
Task<ImmutableArray<BinaryVulnMatch>> LookupByFingerprintAsync(
byte[] fingerprint, FingerprintLookupOptions? options = null, CancellationToken ct = default);
Task<ImmutableDictionary<string, ImmutableArray<BinaryVulnMatch>>> LookupByFingerprintBatchAsync(
IEnumerable<(string Key, byte[] Fingerprint)> fingerprints, FingerprintLookupOptions? options = null, CancellationToken ct = default);
// DeltaSig (delta-signature) matching
Task<ImmutableArray<BinaryVulnMatch>> LookupByDeltaSignatureAsync(
Stream binaryStream, DeltaSigLookupOptions? options = null, CancellationToken ct = default);
Task<ImmutableArray<BinaryVulnMatch>> LookupBySymbolHashAsync(
string symbolHash, string symbolName, DeltaSigLookupOptions? options = null, CancellationToken ct = default);
// Corpus-based function identification (semantic / instruction / API-call fingerprints)
Task<ImmutableArray<CorpusFunctionMatch>> IdentifyFunctionFromCorpusAsync(
FunctionFingerprintSet fingerprints, CorpusLookupOptions? options = null, CancellationToken ct = default);
Task<ImmutableDictionary<string, ImmutableArray<CorpusFunctionMatch>>> IdentifyFunctionsFromCorpusBatchAsync(
IEnumerable<(string Key, FunctionFingerprintSet Fingerprints)> functions, CorpusLookupOptions? options = null, CancellationToken ct = default);
}
public sealed record BinaryVulnMatch
{
public required string CveId { get; init; }
public required string VulnerablePurl { get; init; }
public required MatchMethod Method { get; init; }
public required decimal Confidence { get; init; }
public MatchEvidence? Evidence { get; init; }
}
// NB: enum includes DeltaSignature; matches are persisted with method values
// range_match | buildid_catalog | fingerprint_match | fix_index (see schema).
public enum MatchMethod { BuildIdCatalog, FingerprintMatch, RangeMatch, DeltaSignature }
GetFixStatusAsync/GetFixStatusBatchAsync return FixStatusResult (state, fixed version, method, confidence, and an optional EvidenceId pointing at the binaries.fix_evidence audit row) — not the FixRecord used internally by the IFixIndexBuilder ingestion path (§2.2.3). Fingerprint lookups take raw byte[] fingerprints plus a FingerprintLookupOptions (min-similarity, max-candidates, architecture/distro/release hints, algorithm), and corpus identification accepts a FunctionFingerprintSet (semantic + instruction + API-call fingerprints) returning CorpusFunctionMatch records with CorpusCveAssociation entries.
3. Data Model
3.1 PostgreSQL Schema (binaries)
The binaries schema stores binary identity, fingerprint, and match data.
BinaryIndex.WebService uses the binaries schema as its authoritative live runtime store for binary vulnerability assertions, fix-index reads, and delta-signature state whenever ConnectionStrings:Default is configured. The service no longer falls back to process-local in-memory stores for these backends in live mode; missing Postgres configuration is treated as a startup configuration error.
CREATE SCHEMA IF NOT EXISTS binaries;
CREATE SCHEMA IF NOT EXISTS binaries_app;
-- RLS helper
CREATE OR REPLACE FUNCTION binaries_app.require_current_tenant()
RETURNS TEXT LANGUAGE plpgsql STABLE SECURITY DEFINER AS $$
DECLARE v_tenant TEXT;
BEGIN
v_tenant := current_setting('app.tenant_id', true);
IF v_tenant IS NULL OR v_tenant = '' THEN
RAISE EXCEPTION 'app.tenant_id session variable not set';
END IF;
RETURN v_tenant;
END;
$$;
3.1.1 Schemas, Migrations, and Tables
Schema is applied by embedded forward-only SQL migrations (Migrations/*.sql), run on the live WebService path via the shared AddStartupMigrations<BinaryIndexRuntimeStorageOptions>(schemaName: "binaries", moduleName: "BinaryIndex", …) (Services/BinaryIndexRuntimePersistenceExtensions.cs:45). (BinaryIndexMigrationRunner in the persistence library is a separate hand-rolled runner that the WebService does not call — verified 2026-07-12: no other .cs file references it.) The schema spans three PostgreSQL schemas (binaries, binary_index, groundtruth), not the single binaries schema implied by §3.1:
There is exactly one live migration file — 001_v1_binaryindex_baseline.sql (30 CREATE TABLE statements) — into which the pre-1.0 chain was collapsed. The table below maps each archived pre-1.0 source file to the schema and tables it contributed to that baseline:
| Folded-in pre-1.0 source (archived) | Schema(s) | Tables created in 001_v1_binaryindex_baseline.sql |
|---|---|---|
001_initial_schema.sql | binaries (+ binaries_app helper; baseline lines 30-31) | binary_identity, corpus_snapshots, binary_package_map, vulnerable_buildids, binary_vuln_assertion, fix_evidence, cve_fix_index, fix_index_priority, vulnerable_fingerprints, fingerprint_corpus_metadata, fingerprint_matches |
002_fingerprint_claims.sql | binary_index (baseline line 472) | function_fingerprints, fingerprint_claims, reproducible_builds, build_outputs |
003_delta_signatures.sql | binaries | delta_signature, signature_pack, signature_pack_entry, delta_sig_match |
004_groundtruth_schema.sql | groundtruth (baseline line 922) | symbol_sources, source_state, raw_documents, symbol_observations, security_pairs, buildinfo_metadata, cve_fix_mapping |
005_validation_kpis.sql | groundtruth | validation_kpis, validation_pair_kpis, kpi_baselines, regression_checks |
The five filenames in the left column are history, not live migrations: they exist only under
Migrations/_archived/pre_1.0/mig061/and are not applied at runtime (the.csprojembeds the non-recursiveMigrations\*.sqlglob). The Symbols server has its own single live baseline,StellaOps.Symbols.Server/Migrations/001_v1_symbols_server_baseline.sql.
Key binaries tables:
| Table | Purpose |
|---|---|
binaries.binary_identity | Known binary identities (Build-ID, hashes, format, arch) |
binaries.binary_package_map | Binary → package mapping per snapshot |
binaries.vulnerable_buildids | Build-IDs known to be vulnerable |
binaries.binary_vuln_assertion | Per-binary vuln assertions (affected/not_affected/fixed/unknown) with evidence refs |
binaries.vulnerable_fingerprints | Function fingerprints for CVEs |
binaries.fingerprint_corpus_metadata | Per-package fingerprinting metadata (function/fingerprint counts) |
binaries.cve_fix_index | Patch-aware fix status per distro/release/source-pkg/CVE |
binaries.fix_evidence | Audit trail (changelog / patch_header / security_feed / upstream_match) |
binaries.fix_index_priority | Source-conflict resolution priority |
binaries.fingerprint_matches | Match results (findings evidence) |
binaries.corpus_snapshots | Corpus ingestion tracking (signed via DSSE) |
All tenant-scoped binaries tables ENABLE/FORCE ROW LEVEL SECURITY with a tenant_id = binaries_app.require_current_tenant() policy; require_current_tenant() reads the app.tenant_id session GUC and raises if unset.
docs/db/schemas/binaries_schema_specification.mdis referenced for full DDL but only covers thebinariesschema; thebinary_indexandgroundtruthschemas are defined solely by their migration files above.
3.2 RustFS Layout
rustfs://stellaops/binaryindex/
fingerprints/<algorithm>/<prefix>/<fingerprint_id>.bin
corpus/<distro>/<release>/<snapshot_id>/manifest.json
corpus/<distro>/<release>/<snapshot_id>/packages/<pkg>.metadata.json
evidence/<match_id>.dsse.json
3.2.1 Fingerprint Blob Storage Truthfulness
FingerprintBlobStorage writes evidence blobs through StellaOps.ObjectStorage.IContentAddressedBlobStore. The default constructor keeps the explicit local CAS root configured by STELLAOPS_BINARYINDEX_FINGERPRINT_BLOB_ROOT. Production hosts register AddBinaryIndexFingerprintBlobStorage(...), which binds BinaryIndex:FingerprintBlobStorage, validates the selected backend on startup, and injects the shared local filesystem or S3-compatible CAS backend. The service must fail closed when neither a local root nor an object-store backend is configured; it must not return storage references without writing bytes.
Current relative layout for both local filesystem and object-store backends:
<root>/
binaryindex/fingerprints/<algorithm>/<prefix>/<fingerprint_id>/sha256-<digest>.bin
binaryindex/fingerprints/refbuilds/<cve_id>/<build_type>/sha256-<digest>.tar.zst
Storage references are relative paths under the configured root or bucket prefix. Writes are digest-verified and idempotent for the same bytes, empty evidence blobs are rejected, and reads recompute SHA-256 from storage. Missing generated paths return null; digest mismatch or path escape attempts throw and must be treated as evidence verification failures.
Production configuration:
{
"BinaryIndex": {
"FingerprintBlobStorage": {
"Provider": "S3Compatible",
"S3": {
"BucketName": "binaryindex-fingerprints",
"ServiceUrl": "http://rustfs:9000",
"Region": "us-east-1",
"ForcePathStyle": true,
"Prefix": "tenant-a",
"AccessKeyId": "<from-secret>",
"SecretAccessKey": "<from-secret>"
}
}
}
}
Provider defaults to FileSystem; RootPath or STELLAOPS_BINARYINDEX_FINGERPRINT_BLOB_ROOT supplies the local CAS root. Missing local root is intentionally not converted into a fake reference: the first write/read fails closed with a configuration error. If any S3 values are present while Provider is omitted, the host treats that as S3-compatible intent and validates S3 rather than silently falling back to local storage. S3-compatible/RustFS mode requires a bucket, absolute HTTP(S) service URL, region, relative prefix, and either both access-key fields or neither for ambient credentials.
Shared CAS pattern. BinaryIndex Fingerprints uses the same shape established by the shared
StellaOps.ObjectStorageS3-compatible CAS layer landed in commitf42604a50(digest-verified writes, idempotent on identical bytes, path-escape rejected, bounded 4xx/5xx propagation). Signals runtime-facts storage adopts the same shape viaRustFsRuntimeFactsArtifactStore(see Signals architecture: Runtime-facts production storage) — both modules reject Filesystem-only configuration in Production and emit structured telemetry on transient object-store failures. No BinaryIndex code change in this sprint; the cross-link exists so operators wiring RustFS understand the same digest-verified contract applies in both modules.
4. Integration Points
4.1 Scanner.Worker Integration
During container scanning, Scanner.Worker queries BinaryIndex for each extracted binary:
sequenceDiagram
participant SW as Scanner.Worker
participant BI as BinaryIndex
participant PG as PostgreSQL
participant FL as Findings Ledger
SW->>SW: Extract binary from layer
SW->>SW: Compute BinaryIdentity
SW->>BI: LookupByIdentityAsync(identity)
BI->>PG: Query binaries.vulnerable_buildids
PG-->>BI: Matches
BI->>PG: Query binaries.cve_fix_index (if distro known)
PG-->>BI: Fix status
BI-->>SW: BinaryVulnMatch[]
SW->>FL: RecordFinding(match, evidence)
4.2 Concelier Integration
BinaryIndex subscribes to Concelier’s advisory updates:
sequenceDiagram
participant CO as Concelier
participant BI as BinaryIndex
participant PG as PostgreSQL
CO->>CO: Ingest new advisory
CO->>BI: advisory.created event
BI->>BI: Check if affected packages in corpus
BI->>PG: Update binaries.binary_vuln_assertion
BI->>BI: Queue fingerprint generation (if high-impact)
4.3 Policy Integration
Binary matches are recorded as proof segments:
{
"segment_type": "binary_fingerprint_evidence",
"payload": {
"binary_identity": {
"format": "elf",
"build_id": "abc123...",
"file_sha256": "def456..."
},
"matches": [
{
"cve_id": "CVE-2024-1234",
"method": "buildid_catalog",
"confidence": 0.98,
"vulnerable_purl": "pkg:deb/debian/libssl3@1.1.1n-0+deb11u3"
}
]
}
}
4b. HTTP API Surface
StellaOps.BinaryIndex.WebService exposes four MVC controllers (plus the ops controller documented in §7.3). All controllers emit application/json and RFC 7807 ProblemDetails for errors (type: https://stellaops.dev/errors/<code>). The gateway proxies these routes to the service.
Authentication. Outside the Testing environment, Program.cs registers AddStellaOpsResourceServerAuthentication and a fallback authorization policy that requires an authenticated bearer token on every route not marked [AllowAnonymous]. No controller declares a per-action scope, and no binaryindex:* scope exists in StellaOpsScopes; authorization is “authenticated caller” only (tenant isolation is enforced downstream by PostgreSQL RLS, §3.1). BypassNetworks env config can exempt in-cluster callers.
4b.1 Resolution API (api/v1/resolve)
ResolutionController — the primary vulnerability-resolution surface. Backed by CachedResolutionService wrapping ResolutionService; results may be cached in Valkey and optionally carry a DSSE attestation (EnableDsseByDefault is true).
| Method | Path | Body / Query | Description |
|---|---|---|---|
| POST | /api/v1/resolve/vuln | VulnResolutionRequest; ?bypassCache=bool | Resolve a single binary identity (package, build_id, hashes, distro_release) → VulnResolutionResponse |
| POST | /api/v1/resolve/vuln/batch | BatchVulnResolutionRequest (items[] + options) | Batch resolve (max 500 items) → BatchVulnResolutionResponse |
4b.2 Golden-Set Curation API (api/v1/golden-sets)
GoldenSetController — CRUD + review-workflow for ground-truth vulnerability signatures (authored by experts). Backed by IGoldenSetStore + IGoldenSetValidator.
| Method | Path | Description |
|---|---|---|
| GET | /api/v1/golden-sets | List with filters (component, status, tags, limit 1–500, offset, orderBy) → GoldenSetListResponse |
| GET | /api/v1/golden-sets/{id} | Get one definition + status → GoldenSetResponse (404 if missing) |
| POST | /api/v1/golden-sets | Create (validated; stored in Draft) → GoldenSetCreateResponse (409 on duplicate) |
| PATCH | /api/v1/golden-sets/{id}/status | Workflow transition → GoldenSetStatusResponse (400 on invalid transition) |
| GET | /api/v1/golden-sets/{id}/audit | Audit log of status changes → GoldenSetAuditResponse |
| DELETE | /api/v1/golden-sets/{id} | Soft-delete (move to Archived) → 204 |
Status workflow enforced by the controller: Draft → InReview → Approved → Deprecated/Archived (plus InReview → Draft reject and Draft → Archived). Statuses: Draft, InReview, Approved, Deprecated, Archived.
4b.3 Patch-Coverage API (api/v1/stats/patch-coverage)
PatchCoverageController — aggregated patch-coverage data for the Patch Map Explorer UI. Backed by IDeltaSignatureRepository.
| Method | Path | Description |
|---|---|---|
| GET | /api/v1/stats/patch-coverage | Per-CVE vulnerable/patched/unknown counts + coverage % for heatmap (?cve=, ?package=, limit 1–500, offset) → PatchCoverageResult |
| GET | /api/v1/stats/patch-coverage/{cveId}/details | Function-level breakdown for a CVE → PatchCoverageDetails (404 if no data) |
| GET | /api/v1/stats/patch-coverage/{cveId}/matches | Paginated matching images (?symbol=, ?state=vulnerable|patched|unknown, limit 1–200, offset) → PatchMatchPage |
5. MVP Roadmap
MVP 1: Known-Build Binary Catalog (Sprint 6000.0001)
Goal: Query “is this Build-ID vulnerable?” with distro-level precision.
Deliverables:
binariesPostgreSQL schema- Build-ID to package mapping tables
- Basic CVE lookup by binary identity
- Debian/Ubuntu corpus connector
MVP 2: Patch-Aware Backport Handling (Sprint 6000.0002)
Goal: Handle “version says vulnerable but distro backported the fix.”
Deliverables:
- Fix index builder (changelog + patch header parsing)
- Distro-specific version comparison
- RPM corpus connector
- Scanner.Worker integration
MVP 3: Binary Fingerprint Factory (Sprint 6000.0003)
Goal: Detect vulnerable code independent of package metadata.
Deliverables:
- Fingerprint storage and matching
- Reference build generation pipeline
- Fingerprint validation corpus
- High-impact CVE coverage (OpenSSL, glibc, zlib, curl)
MVP 4: Full Scanner Integration (Sprint 6000.0004)
Goal: Binary evidence in production scans.
Deliverables:
- Scanner.Worker binary lookup integration
- Findings Ledger binary match records
- Proof segment attestations
- CLI binary match inspection
5b. Fix Evidence Chain
The Fix Evidence Chain provides auditable proof of why a CVE is marked as fixed (or not) for a specific distro/package combination. This is critical for patch-aware backport handling where package versions can be misleading.
5b.1 Evidence Sources
| Source | Confidence | Description |
|---|---|---|
| Security Feed (OVAL) | 0.95-0.99 | Authoritative feed from distro (Debian Security Tracker, Red Hat OVAL) |
| Patch Header (DEP-3) | 0.87-0.95 | CVE reference in Debian/Ubuntu patch metadata |
| Changelog | 0.75-0.85 | CVE mention in debian/changelog or RPM %changelog |
| Upstream Patch Match | 0.90 | Binary diff matches known upstream fix |
5b.2 Evidence Storage
Evidence is stored in two PostgreSQL tables:
-- Fix index: one row per (distro, release, source_pkg, cve_id, architecture)
-- (the architecture column is nullable; the uniqueness key includes it)
CREATE TABLE binaries.cve_fix_index (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id TEXT NOT NULL DEFAULT binaries_app.require_current_tenant(),
distro TEXT NOT NULL, -- debian, ubuntu, alpine, rhel
release TEXT NOT NULL, -- bookworm, jammy, v3.19
source_pkg TEXT NOT NULL,
cve_id TEXT NOT NULL,
architecture TEXT, -- arch scope (nullable: NULL = all arches)
state TEXT NOT NULL -- fixed, vulnerable, not_affected, wontfix, unknown
CHECK (state IN ('fixed', 'vulnerable', 'not_affected', 'wontfix', 'unknown')),
fixed_version TEXT,
method TEXT NOT NULL -- security_feed, changelog, patch_header, upstream_match
CHECK (method IN ('security_feed', 'changelog', 'patch_header', 'upstream_match')),
confidence DECIMAL(3,2) NOT NULL CHECK (confidence >= 0.00 AND confidence <= 1.00),
evidence_id UUID REFERENCES binaries.fix_evidence(id),
snapshot_id UUID,
indexed_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT cve_fix_index_unique
UNIQUE (tenant_id, distro, release, source_pkg, cve_id, architecture)
);
-- Evidence blobs: audit trail
CREATE TABLE binaries.fix_evidence (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id TEXT NOT NULL DEFAULT binaries_app.require_current_tenant(),
-- changelog, patch_header, security_feed, upstream_match
evidence_type TEXT NOT NULL
CHECK (evidence_type IN ('changelog', 'patch_header', 'security_feed', 'upstream_match')),
source_file TEXT, -- Path to source file (changelog, patch)
source_sha256 TEXT, -- Hash of source file
excerpt TEXT, -- Relevant snippet (max 1KB)
metadata JSONB NOT NULL DEFAULT '{}', -- Structured metadata
snapshot_id UUID,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
5b.3 Evidence Types
ChangelogEvidence:
{
"evidence_type": "changelog",
"source_file": "debian/changelog",
"excerpt": "* Fix CVE-2024-0727: PKCS12 decoding crash",
"metadata": {
"version": "3.0.11-1~deb12u2",
"line_number": 5
}
}
PatchHeaderEvidence:
{
"evidence_type": "patch_header",
"source_file": "debian/patches/CVE-2024-0727.patch",
"excerpt": "CVE: CVE-2024-0727\nOrigin: upstream, https://github.com/openssl/commit/abc123",
"metadata": {
"patch_sha256": "abc123def456..."
}
}
SecurityFeedEvidence:
{
"evidence_type": "security_feed",
"metadata": {
"feed_id": "debian-security-tracker",
"entry_id": "DSA-5678-1",
"published_at": "2024-01-15T10:00:00Z"
}
}
5b.4 Confidence Resolution
When multiple evidence sources exist for the same CVE, the upsert (FixIndexRepository.UpsertAsync) keeps the highest-confidence method/evidence while always refreshing the latest state/version/snapshot. The actual conflict target is the full six-column uniqueness key (including the nullable architecture), and the references use the EXCLUDED pseudo-row and the fully-qualified existing row, not existing/new aliases:
ON CONFLICT (tenant_id, distro, release, source_pkg, cve_id, architecture)
DO UPDATE SET
state = EXCLUDED.state,
fixed_version = EXCLUDED.fixed_version,
method = CASE
WHEN binaries.cve_fix_index.confidence < EXCLUDED.confidence THEN EXCLUDED.method
ELSE binaries.cve_fix_index.method
END,
confidence = GREATEST(binaries.cve_fix_index.confidence, EXCLUDED.confidence),
evidence_id = CASE
WHEN binaries.cve_fix_index.confidence < EXCLUDED.confidence THEN EXCLUDED.evidence_id
ELSE binaries.cve_fix_index.evidence_id
END,
snapshot_id = EXCLUDED.snapshot_id,
updated_at = now()
5b.5 Parsers
The following parsers extract CVE fix information:
| Parser | Distros | Input | Confidence |
|---|---|---|---|
DebianChangelogParser | Debian, Ubuntu | debian/changelog | 0.80 |
PatchHeaderParser | Debian, Ubuntu | debian/patches/*.patch (DEP-3) | 0.87 |
AlpineSecfixesParser | Alpine | APKBUILD secfixes block | 0.95 |
RpmChangelogParser | RHEL, Fedora, CentOS | RPM spec %changelog | 0.75 |
5b.6 Query Flow
sequenceDiagram
participant SW as Scanner.Worker
participant BVS as BinaryVulnerabilityService
participant FIR as FixIndexRepository
participant PG as PostgreSQL
SW->>BVS: GetFixStatusAsync(debian, bookworm, openssl, CVE-2024-0727)
BVS->>FIR: GetFixStatusAsync(...)
FIR->>PG: SELECT FROM cve_fix_index WHERE ...
PG-->>FIR: FixIndexEntry (state=fixed, confidence=0.87)
FIR-->>BVS: FixStatusResult
BVS-->>SW: {state: Fixed, confidence: 0.87, method: PatchHeader}
6. Security Considerations
6.1 Trust Boundaries
- Corpus Ingestion - Packages are untrusted; extraction runs in sandboxed workers
- Fingerprint Generation - Reference builds compiled in isolated environments
- Query API - Tenant-isolated via RLS; no cross-tenant data leakage
6.2 Signing & Provenance
- All corpus snapshots are signed (DSSE)
- Fingerprint sets are versioned and signed
- Every match result references evidence digests
6.3 Sandbox Requirements
Binary extraction and fingerprint generation MUST run with:
- Seccomp profile restricting syscalls
- Read-only root filesystem
- No network access during analysis
- Memory/CPU limits
7. Observability
7.1 Metrics
| Metric | Type | Labels |
|---|---|---|
binaryindex_lookup_total | Counter | method, result |
binaryindex_lookup_latency_ms | Histogram | method |
binaryindex_corpus_packages_total | Gauge | distro, release |
binaryindex_fingerprints_indexed | Gauge | algorithm, component |
binaryindex_match_confidence | Histogram | method |
7.2 Traces
binaryindex.lookup- Full lookup spanbinaryindex.corpus.ingest- Corpus ingestionbinaryindex.fingerprint.generate- Fingerprint generation
7.3 Ops Endpoints
Sprint: SPRINT_20260112_007_BINIDX_binaryindex_user_config
BinaryIndex exposes read-only ops endpoints for health, bench, cache, and effective configuration:
All four endpoints are served by BinaryIndexOpsController ([Route("api/v1/ops/binaryindex")]):
| Endpoint | Method | Response Schema | Description |
|---|---|---|---|
/api/v1/ops/binaryindex/health | GET | BinaryIndexOpsHealthResponse | Lifter warmness, per-ISA pool counts, cache availability |
/api/v1/ops/binaryindex/bench/run | POST | BinaryIndexBenchResponse | Run latency benchmark (lifter-acquire + cache-lookup), return min/max/mean/p50/p95/p99 |
/api/v1/ops/binaryindex/cache | GET | BinaryIndexFunctionCacheStats | Function IR cache hit/miss/eviction statistics |
/api/v1/ops/binaryindex/config | GET | BinaryIndexEffectiveConfig | Effective lifter-pool + function-cache configuration |
Auth. These endpoints carry no per-endpoint
[Authorize]/scope attribute. The WebService registers a fallback authorization policy requiring an authenticated bearer token (Authority resource-server scheme) for every route except those that opt out with[AllowAnonymous](/health,/healthz,/readyz, OpenAPI, build-info). No dedicatedbinaryindex:*scope is defined inStellaOpsScopes; onlysymbols:read/symbols:writeexist (consumed by the Symbols.Server, §Symbols).
Startup safety. When PostgreSQL persistence is intentionally absent in local/offline boot, the host fails closed rather than fabricating results for the authoritative resolution path (see §3.1, §8.1.4). The optional B2R2 lifter is no longer compiled into
StellaOps.BinaryIndex.WebService; until a signed mounted analyzer is admitted,/healthreportsstatus: "degraded"withlifterStatus: "not-mounted". ThecacheServiceconstructor dependency remains nullable and reportscacheStatus: "unavailable"when unregistered.
7.3.1 Response Schemas
The records below are defined in BinaryIndexOpsController.cs. Field names are flat (System.Text.Json camel-cases the C# record properties); there is no nested components / lifterWarmness health envelope.
BinaryIndexOpsHealthResponse: (status is healthy only when both lifter is warm and cache is enabled, else degraded)
{
"status": "healthy",
"timestamp": "2026-01-16T12:00:00.0000000+00:00",
"lifterStatus": "warm",
"lifterWarm": true,
"lifterPoolStats": { "intel-64": 4, "armv8-64": 2 },
"cacheStatus": "enabled",
"cacheEnabled": true
}
BinaryIndexBenchResponse: (iterations defaults to 10, range 1–1000; two fixed latency-stat blocks)
{
"timestamp": "2026-01-16T12:00:00.0000000+00:00",
"iterations": 100,
"lifterAcquireLatencyMs": { "min": 5.2, "max": 142.8, "mean": 28.4, "p50": 22.1, "p95": 78.3, "p99": 121.5 },
"cacheLookupLatencyMs": { "min": 0.4, "max": 3.1, "mean": 1.2, "p50": 1.0, "p95": 2.6, "p99": 2.9 }
}
BinaryIndexFunctionCacheStats:
{
"enabled": true,
"hits": 15234,
"misses": 892,
"evictions": 45,
"hitRate": 0.944,
"keyPrefix": "stellaops:binidx:funccache:",
"cacheTtlSeconds": 14400
}
BinaryIndexEffectiveConfig: (flat lifter-pool + function-cache view; no semanticLifting/persistence/backendVersions blocks)
{
"lifterPoolMaxSizePerIsa": 4,
"lifterPoolWarmPreloadEnabled": true,
"lifterPoolWarmPreloadIsas": ["intel-64", "intel-32", "armv8-64", "armv7-32"],
"lifterPoolAcquireTimeoutSeconds": 5,
"cacheEnabled": true,
"cacheKeyPrefix": "stellaops:binidx:funccache:",
"cacheTtlSeconds": 14400,
"cacheMaxTtlSeconds": 86400,
"b2r2Version": "0.9.1",
"normalizationRecipeVersion": "v1"
}
Note. The config endpoint returns only the host-owned
B2R2PoolOptions(StellaOps.BinaryIndex.Core.Configuration) andFunctionIrCacheOptionsvalues; it does not currently perform recursive secret redaction, and the connection string is not echoed by this endpoint at all.
8. Configuration
Sprint: SPRINT_20260112_007_BINIDX_binaryindex_user_config
8.1 Configuration Sections
The bindable option classes use the section paths declared by their SectionName constants. Note the StellaOps:BinaryIndex: prefix on the analysis options (not a bare BinaryIndex: prefix). The WebService also binds top-level Resolution, ResolutionCache, and VexBridge sections plus ConnectionStrings.
8.1.1 B2R2 Lifter Pool (StellaOps:BinaryIndex:B2R2Pool)
Bound to the host-owned B2R2PoolOptions (StellaOps.BinaryIndex.Core.Configuration). This section describes the sanitized operator config exposed by /api/v1/ops/binaryindex/config; it does not imply that the optional B2R2 analyzer implementation is compiled into the WebService.
| Key | Type | Default | Description |
|---|---|---|---|
MaxPoolSizePerIsa | int | 4 | Maximum pooled lifters per ISA |
EnableWarmPreload | bool | true | Warm-preload lifters at startup |
WarmPreloadIsas | string[] | ["intel-64","intel-32","armv8-64","armv7-32"] | ISAs to warm on startup |
AcquireTimeout | TimeSpan | 00:00:05 | Timeout for acquiring a lifter |
EnableMetrics | bool | true | Enable lifter-pool metric collection when a mounted analyzer supplies a lifter |
StellaOps:
BinaryIndex:
B2R2Pool:
MaxPoolSizePerIsa: 4
EnableWarmPreload: true
WarmPreloadIsas: [intel-64, intel-32, armv8-64, armv7-32]
AcquireTimeout: "00:00:05"
EnableMetrics: true
8.1.2 Function IR Cache (StellaOps:BinaryIndex:FunctionIrCache)
Bound to FunctionIrCacheOptions (StellaOps.BinaryIndex.Cache).
| Key | Type | Default | Description |
|---|---|---|---|
KeyPrefix | string | stellaops:binidx:funccache: | Valkey cache key prefix |
CacheTtl | TimeSpan | 04:00:00 | Default cache TTL (4 h) |
MaxTtl | TimeSpan | 24:00:00 | Maximum TTL (24 h) |
Enabled | bool | true | Whether caching is enabled |
B2R2Version | string | 0.9.1 | B2R2 version included in cache keys |
NormalizationRecipeVersion | string | v1 | Recipe version for cache-key stability |
StellaOps:
BinaryIndex:
FunctionIrCache:
Enabled: true
KeyPrefix: "stellaops:binidx:funccache:"
CacheTtl: "04:00:00"
MaxTtl: "24:00:00"
B2R2Version: "0.9.1"
NormalizationRecipeVersion: "v1"
8.1.3 Resolution + Cache + VexBridge (WebService runtime)
The WebService binds these top-level sections (see appsettings.json):
ConnectionStrings:
Default: "Host=postgres;Database=stellaops;..." # required (auth. PG state)
Redis: "valkey:6379" # required (resolution cache)
Resolution: # ResolutionServiceOptions (SectionName: "Resolution")
DefaultCacheTtl: "04:00:00"
MaxBatchSize: 500
EnableDsseByDefault: true
MinConfidenceThreshold: 0.70
ResolutionCache: # ResolutionCacheOptions
FixedTtl: "24:00:00"
VulnerableTtl: "04:00:00"
UnknownTtl: "01:00:00"
KeyPrefix: "resolution"
EnableEarlyExpiry: true
EarlyExpiryFactor: 0.1
VexBridge:
SignWithDsse: true
DefaultProviderId: "stellaops.binaryindex"
DefaultStreamId: "binary_resolution"
MinConfidenceThreshold: 0.70
MaxBatchSize: 1000
RateLimiting: { } # bound by AddResolutionRateLimiting (limits /resolve)
For BinaryIndex.WebService, the live host contract is:
ConnectionStrings:Defaultis required for authoritative PostgreSQL-backed golden-set, vulnerability, and delta-signature state.ConnectionStrings:Redisis required for the authoritative resolution cache (Program.cscallsRequireRedisConnectionStringand fails closed if absent).- Live runtime does not degrade to in-memory storage when either connection string is absent. Golden-set and runtime persistence wire
AddStartupMigrationsto apply embeddedMigrations/*.sqlon boot. - Restart-survival proof is covered by
BinaryIndexDurableRuntimeTests, which writes golden-set state through the HTTP API, persists resolution cache data in Valkey, restarts the host, and reads both back from the durable services.
The previously-documented
Postgres:BinaryIndex,BinaryIndex:SemanticLifting, andBinaryIndex:Opssections (withEnableHealthEndpoint/BenchRateLimitPerMinute/RedactedKeyskeys) do not correspond to bound option classes in the current WebService and are not consumed. Rate limiting is applied to/api/v1/resolvevia theRateLimitingsection, not to the opsbench/runendpoint.
8.2 Legacy / Corpus Configuration
Status (roadmap): the
binaryindex.yamlshape below describes the corpus-connector and fingerprinting configuration envisioned for the corpus ingestion MVP (§5). It is the connector-layer config model and is not bound by theBinaryIndex.WebServicehost today; it is retained as the forward design for the Corpus/Worker ingestion path.
# binaryindex.yaml (corpus configuration)
binaryindex:
enabled: true
corpus:
connectors:
- type: debian
enabled: true
mirror: http://deb.debian.org/debian
releases: [bookworm, bullseye]
architectures: [amd64, arm64]
- type: ubuntu
enabled: true
mirror: http://archive.ubuntu.com/ubuntu
releases: [jammy, noble]
fingerprinting:
enabled: true
algorithms: [basic_block, cfg]
target_components:
- openssl
- glibc
- zlib
- curl
- sqlite
min_function_size: 16 # bytes
max_functions_per_binary: 10000
lookup:
cache_ttl: 3600
batch_size: 100
timeout_ms: 5000
storage:
postgres_schema: binaries
rustfs_bucket: stellaops/binaryindex
9. Testing Strategy
9.1 Unit Tests
- Identity extraction (Build-ID, hashes)
- Fingerprint generation determinism
- Fix index parsing (changelog, patch headers)
9.2 Integration Tests
- PostgreSQL schema validation
- Full corpus ingestion flow
- Scanner.Worker lookup integration
9.3 Regression Tests
- Known CVE detection (golden corpus)
- Backport handling (Debian libssl example)
- False positive rate validation
10. Golden Corpus for Patch Provenance
Sprint: SPRINT_20260121_034/035/036 - Golden Corpus Implementation
The BinaryIndex module supports a golden corpus of patch-paired artifacts that enables offline SBOM reproducibility and binary-level patch provenance verification.
10.1 Corpus Purpose
The golden corpus provides:
- Auditor-ready evidence bundles for air-gapped customers
- Regression testing for binary matching accuracy
- Proof of patch status independent of package metadata
10.2 Corpus Sources
| Source | Type | Purpose |
|---|---|---|
| Debian Security Tracker / DSAs | Advisory | Primary advisory linkage |
| Debian Snapshot | Binary archive | Pre/post patch binary pairs |
| Ubuntu Security Notices | Advisory | Ubuntu-specific advisories |
| Alpine secdb | Advisory | Alpine YAML advisories |
| OSV dump | Unified schema | Cross-reference and commit ranges |
10.2.1 Symbol Source Connectors
Sprint: SPRINT_20260121_035_BinaryIndex_golden_corpus_connectors_cli
The corpus ingestion layer uses pluggable connectors to retrieve symbols and metadata from upstream sources:
| Connector ID | Implementation | Protocol | Data Retrieved |
|---|---|---|---|
debuginfod-fedora | DebuginfodConnector | debuginfod HTTP | ELF debug symbols by Build-ID |
debuginfod-ubuntu | DebuginfodConnector | debuginfod HTTP | ELF debug symbols by Build-ID |
ddeb-ubuntu | DdebConnector | APT/HTTP | .ddeb debug packages |
buildinfo-debian | BuildinfoConnector | HTTP | .buildinfo reproducibility records |
secdb-alpine | AlpineSecDbConnector | Git/HTTP | secfixes YAML from APKBUILD |
Connector Interface:
public interface ISymbolSourceConnector
{
string ConnectorId { get; }
string DisplayName { get; }
string[] SupportedDistros { get; }
Task<ConnectorStatus> GetStatusAsync(CancellationToken ct);
Task SyncAsync(SyncOptions options, CancellationToken ct);
Task<SymbolLookupResult?> LookupByBuildIdAsync(string buildId, CancellationToken ct);
Task<IAsyncEnumerable<SymbolRecord>> SearchAsync(SymbolSearchQuery query, CancellationToken ct);
}
Debuginfod Connector:
The DebuginfodConnector implements the debuginfod protocol for retrieving debug symbols:
- Endpoint:
GET /buildid/<build-id>/debuginfo - Supports federated queries across multiple debuginfod servers
- Caches retrieved symbols in RustFS blob storage
- Rate-limited to respect upstream server policies
Ubuntu ddeb Connector:
The DdebConnector retrieves Ubuntu debug symbol packages (.ddeb):
- Sources:
ddebs.ubuntu.commirror - Indexes: Reads
Packages.xzfor package metadata - Extraction: Unpacks
.ddebAR archives to extract DWARF symbols - Mapping: Links debug symbols to binary packages via Build-ID
Debian Buildinfo Connector:
The BuildinfoConnector retrieves Debian buildinfo files for reproducibility verification:
- Source:
buildinfos.debian.netand snapshot archives - Purpose: Provides build environment metadata for reproducible builds
- Fields extracted:
Build-Date,Build-Architecture,Checksums-Sha256 - Integration: Cross-references with binary packages for provenance
Alpine SecDB Connector:
The AlpineSecDbConnector parses Alpine’s security database:
- Source:
secfixesblocks in APKBUILD files - Repository:
alpine/aportsGit repository - Format: YAML blocks mapping CVEs to fixed versions
- Example:
secfixes: 3.0.11-r0: - CVE-2024-0727 - CVE-2024-0728
OSV Dump Parser:
The OsvDumpParser processes Google OSV database dumps for advisory cross-correlation:
- Source:
osv.devbulk exports (JSON) - Purpose: CVE → commit range extraction for patch identification
- Cross-reference: Correlates OSV entries with distribution advisories
- Inconsistency detection: Identifies discrepancies between OSV and distro advisories
public interface IOsvDumpParser
{
IAsyncEnumerable<OsvParsedEntry> ParseDumpAsync(Stream osvDumpStream, CancellationToken ct);
OsvCveIndex BuildCveIndex(IEnumerable<OsvParsedEntry> entries);
IEnumerable<AdvisoryCorrelation> CrossReferenceWithExternal(
OsvCveIndex osvIndex,
IEnumerable<ExternalAdvisory> externalAdvisories);
IEnumerable<AdvisoryInconsistency> DetectInconsistencies(
IEnumerable<AdvisoryCorrelation> correlations);
}
CLI Access:
All connectors are manageable via the stella groundtruth sources CLI commands:
# List all connectors
stella groundtruth sources list
# Sync specific connector
stella groundtruth sources sync --source buildinfo-debian --full
# Enable/disable connectors
stella groundtruth sources enable ddeb-ubuntu
stella groundtruth sources disable debuginfod-fedora
See Ground-Truth CLI Guide for complete CLI documentation
10.3 Key Performance Indicators
| KPI | Target | Description |
|---|---|---|
| Per-function match rate | >= 90% | Functions matched in post-patch binary |
| False-negative patch detection | <= 5% | Patched functions incorrectly classified |
| SBOM canonical-hash stability | 3/3 | Determinism across independent runs |
| Binary reconstruction equivalence | Trend | Rebuilt binary matches original |
| End-to-end verify time (p95, cold) | Trend | Offline verification performance |
10.4 Validation Harness
The validation harness (IValidationHarness) orchestrates end-to-end verification:
Binary Pair (pre/post) → Symbol Recovery → IR Lifting → Fingerprinting → Matching → Metrics
10.5 Evidence Bundle Format
Evidence bundles follow OCI/ORAS conventions:
<pkg>-<advisory>-bundle.oci.tar
├── manifest.json # OCI manifest
└── blobs/
├── sha256:<sbom> # Canonical SBOM
├── sha256:<pre-bin> # Pre-fix binary
├── sha256:<post-bin> # Post-fix binary
├── sha256:<delta-sig> # DSSE delta-sig predicate
└── sha256:<timestamp> # RFC 3161 timestamp
10.6 Two-Tier Bundle Design and Large Blob References
Sprint: SPRINT_20260122_040_Platform_oci_delta_attestation_pipeline (040-04)
Evidence bundles support two export modes to balance transfer speed with auditability:
| Mode | Export Flag | Contents | Use Case |
|---|---|---|---|
| Light | (default) | Manifest + attestation envelopes + metadata | Quick transfer, metadata-only audit |
| Full | --full | Light + embedded binary blobs in blobs/ | Air-gap replay, full provenance verification |
10.6.1 largeBlobs[] Field
The DeltaSigPredicate includes a largeBlobs array referencing binary artifacts that may be too large to embed in attestation payloads:
{
"schemaVersion": "1.0.0",
"subject": [...],
"delta": [...],
"largeBlobs": [
{
"kind": "preBinary",
"digest": "sha256:a1b2c3...",
"mediaType": "application/octet-stream",
"sizeBytes": 1048576
},
{
"kind": "debugSymbols",
"digest": "sha256:d4e5f6...",
"mediaType": "application/octet-stream",
"sizeBytes": 32768
}
],
"sbomDigest": "sha256:789abc..."
}
Field Definitions (LargeBlobReference in StellaOps.BinaryIndex.DeltaSig/Attestation/DeltaSigPredicate.cs):
| Field | Type | Description |
|---|---|---|
largeBlobs[].kind | string (required) | Blob category. Source-documented kinds: preBinary, postBinary, debugSymbols, irDiff, etc. |
largeBlobs[].digest | string (required) | Content-addressable digest (sha256:<hex>, sha384:<hex>, sha512:<hex>) |
largeBlobs[].mediaType | string (optional) | IANA media type of the blob (nullable in LargeBlobReference) |
largeBlobs[].sizeBytes | long (optional) | Blob size in bytes (nullable in LargeBlobReference) |
sbomDigest | string (optional) | Digest of the canonical SBOM associated with this delta (nullable) |
10.6.2 Blob Fetch Strategy
During stella bundle verify --replay, blobs are resolved in priority order:
- Embedded (full bundles): Read from
blobs/<digest-with-dash>in bundle directory - Local source (
--blob-source /path/): Read from specified local directory - Registry (
--blob-source https://...): HTTP GET from OCI registry (blocked in--offlinemode)
10.6.3 Digest Verification
Fetched blobs are verified against their declared digest using the algorithm prefix:
sha256:<hex> → SHA-256
sha384:<hex> → SHA-384
sha512:<hex> → SHA-512
A mismatch fails the blob replay verification step.
10.7 Related Documentation
11. References
- Advisory:
docs/product/advisories/21-Dec-2025 - Mapping Evidence Within Compiled Binaries.md - Scanner Native Analysis:
src/Scanner/StellaOps.Scanner.Analyzers.Native/ - Existing Fingerprinting:
src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/Binary/ - Build-ID Index:
src/Scanner/StellaOps.Scanner.Analyzers.Native/Index/ - Semantic Diffing Sprint:
docs/implplan/SPRINT_20260105_001_001_BINDEX_semdiff_ir_semantics.md - Semantic Library:
src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.Semantic/ - Semantic Tests:
src/BinaryIndex/__Tests/StellaOps.BinaryIndex.Semantic.Tests/ - Golden Corpus Sprints:
docs/implplan/SPRINT_20260121_034_BinaryIndex_golden_corpus_foundation.md
12. Binary Micro-Witnesses
Binary micro-witnesses provide cryptographic proof of patch status at the binary level. They formalize the output of BinaryIndex’s semantic diffing capabilities into an auditor-friendly, portable format.
12.1 Overview
A micro-witness is a DSSE (Dead Simple Signing Envelope) predicate that captures:
- Subject binary digest (SHA-256)
- CVE/patch reference
- Function-level evidence with confidence scores
- Delta-Sig fingerprint hash
- Tool versions and analysis metadata
- Optional SBOM component mapping
12.2 Predicate Schema
Predicate Type: https://stellaops.dev/predicates/binary-micro-witness@v1
{
"schemaVersion": "1.0.0",
"binary": {
"digest": "sha256:...",
"purl": "pkg:deb/debian/openssl@3.0.11",
"arch": "linux-amd64",
"filename": "libssl.so.3"
},
"cve": {
"id": "CVE-2024-0567",
"advisory": "https://...",
"patchCommit": "abc123"
},
"verdict": "patched",
"confidence": 0.95,
"evidence": [
{
"function": "SSL_CTX_new",
"state": "patched",
"score": 0.97,
"method": "semantic_ksg",
"hash": "sha256:..."
}
],
"deltaSigDigest": "sha256:...",
"sbomRef": {
"sbomDigest": "sha256:...",
"purl": "pkg:...",
"bomRef": "component-ref"
},
"tooling": {
"binaryIndexVersion": "2.1.0",
"lifter": "b2r2",
"matchAlgorithm": "semantic_ksg"
},
"computedAt": "2026-01-28T12:00:00Z"
}
12.3 Verdicts
| Verdict | Meaning |
|---|---|
patched | Binary matches patched version signature |
vulnerable | Binary matches vulnerable version signature |
inconclusive | Unable to determine (insufficient evidence) |
partial | Some functions patched, others not |
12.4 CLI Commands
# Generate a micro-witness
stella witness generate /path/to/binary --cve CVE-2024-0567 --sbom sbom.json --output witness.json
# Verify a micro-witness
stella witness verify witness.json --offline
# Create portable bundle for air-gapped verification
stella witness bundle witness.json --output ./audit-bundle
12.5 Integration with Rekor
When --rekor is specified during generation, witnesses are logged to the Rekor transparency log using v2 tile-based inclusion proofs. This provides tamper-evidence and enables auditors to verify witnesses weren’t backdated.
Offline verification bundles include tile proofs for air-gapped environments.
12.6 Related Documentation
- Auditor Guide:
docs/guides/binary-micro-witness-verification.md - Predicate Schema:
src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-binary-micro-witness.v1.schema.json - CLI Commands:
src/Cli/StellaOps.Cli/Commands/Witness/ - Demo Bundle:
docs/examples/binary-micro-witness/ - Sprint:
docs-archive/implplan/SPRINT_0128_001_BinaryIndex_binary_micro_witness.md
13. Cross-Distro Coverage Matrix for Backport Validation
Manages a curated set of high-impact CVEs with per-distribution backport status tracking, enabling systematic validation of backport detection accuracy across Alpine, Debian, and RHEL.
13.1 Architecture
- CuratedCveEntry — One row per CVE (e.g., Heartbleed, Baron Samedit) with cross-distro
DistroCoverageEntryarray tracking backport status per distro-version - CrossDistroCoverageService — In-memory coverage matrix with upsert, query, summary, and validation marking operations
- SeedBuiltInEntries — Idempotent seeding of 5 curated high-impact CVEs (CVE-2014-0160, CVE-2021-3156, CVE-2015-0235, CVE-2023-38545, CVE-2024-6387) with pre-populated backport status across Alpine, Debian, and RHEL versions
13.2 Distro Families & Backport Status
| Enum | Values |
|---|---|
DistroFamily | Alpine, Debian, Rhel |
BackportStatus | NotPatched, Backported, NotApplicable, Unknown |
13.3 Models
| Type | Description |
|---|---|
DistroCoverageEntry | Per distro-version: package name/version, backport status, validated flag |
CuratedCveEntry | CVE with CommonName, CvssScore, CweIds, Coverage array, computed CoverageRatio |
CrossDistroCoverageSummary | Aggregated counts: TotalCves, TotalEntries, ValidatedEntries, ByDistro breakdown |
DistroBreakdown | Per-distro EntryCount, ValidatedCount, BackportedCount |
CuratedCveQuery | Component/Distro/Status/OnlyUnvalidated filters with Limit/Offset paging |
13.4 Built-in Curated CVEs
| CVE | Component | Common Name | CVSS |
|---|---|---|---|
| CVE-2014-0160 | openssl | Heartbleed | 7.5 |
| CVE-2021-3156 | sudo | Baron Samedit | 7.8 |
| CVE-2015-0235 | glibc | GHOST | 10.0 |
| CVE-2023-38545 | curl | SOCKS5 heap overflow | 9.8 |
| CVE-2024-6387 | openssh | regreSSHion | 8.1 |
13.5 DI Registration
ICrossDistroCoverageService → CrossDistroCoverageService registered via TryAddSingleton in GoldenSetServiceCollectionExtensions.AddGoldenSetServices().
13.6 OTel Metrics
Meter: StellaOps.BinaryIndex.GoldenSet.CrossDistro
| Counter | Description |
|---|---|
crossdistro.upsert.total | CVE entries upserted |
crossdistro.query.total | Coverage queries executed |
crossdistro.seed.total | Built-in entries seeded |
crossdistro.validated.total | Entries marked as validated |
13.7 Source Files
- Models:
src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GoldenSet/Models/CrossDistroCoverageModels.cs - Interface:
src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GoldenSet/Services/ICrossDistroCoverageService.cs - Implementation:
src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GoldenSet/Services/CrossDistroCoverageService.cs
13.8 Test Coverage (37 tests)
- Models: DistroFamily/BackportStatus enum counts, DistroCoverageEntry roundtrips/defaults, CuratedCveEntry coverage ratio/empty, CuratedCveQuery defaults, Summary coverage/empty
- Service: SeedBuiltInEntries population/idempotency/heartbleed/baron-samedit/distro coverage, UpsertAsync store-retrieve/overwrite/null/empty, GetByCveIdAsync unknown/case-insensitive/null, QueryAsync all/component/distro/status/unvalidated/limit-offset/ordering, GetSummaryAsync counts/empty, SetValidatedAsync mark/unknown-cve/unknown-version/summary/null, CreateBuiltInEntries deterministic/distro-coverage
14. ELF Segment Normalization for Delta Hashing
14.1 Purpose
The existing instruction-level normalization (X64/Arm64 pipelines) operates on disassembled instruction streams. ELF Segment Normalization fills the gap for raw binary bytes — zeroing position-dependent data (relocation entries, GOT/PLT displacements, alignment padding) and canonicalizing NOP sleds before disassembly, enabling deterministic delta hashing across builds compiled at different base addresses or link orders.
14.2 Key Types
| Type | Location | Purpose |
|---|---|---|
ElfNormalizationStep | Normalization/ElfSegmentNormalizer.cs | Enum of normalization passes (RelocationZeroing, GotPltCanonicalization, NopCanonicalization, JumpTableRewriting, PaddingZeroing) |
ElfSegmentNormalizationOptions | same | Options record with Default and Minimal presets |
ElfSegmentNormalizationResult | same | Result with NormalizedBytes, DeltaHash (SHA-256), ModifiedBytes, AppliedSteps, StepCounts, computed ModificationRatio |
IElfSegmentNormalizer | same | Interface: Normalize, ComputeDeltaHash |
ElfSegmentNormalizer | same | Implementation with 5 internal passes and 2 OTel counters |
14.3 Normalization Passes
- RelocationZeroing — Scans for ELF64 RELA-shaped entries (heuristic: info field encodes valid x86-64 relocation types 1–42 with symbol index ≤100 000); zeros the offset and addend fields (16 bytes per entry).
- GotPltCanonicalization — Detects
FF 25(JMP [rip+disp32]) andFF 35(PUSH [rip+disp32]) PLT stub patterns; zeros the 4-byte displacement to remove position-dependent indirect jump targets. - NopCanonicalization — Matches 7 multi-byte x86-64 NOP variants (2–7 bytes each, per Intel SDM) and replaces with canonical single-byte NOPs (0x90).
- JumpTableRewriting — Identifies sequences of 4+ consecutive 8-byte entries sharing the same upper 32 bits (switch-statement jump tables); zeros the entries.
- PaddingZeroing — Detects runs of 4+ alignment padding bytes (0xCC or 0x00) between code regions and zeros them.
14.4 Delta Hashing
ComputeDeltaHash produces a lowercase SHA-256 hex string of the normalized byte buffer. Two builds of the same source compiled at different addresses will produce the same delta hash after normalization.
14.5 OTel Instrumentation
Meter: StellaOps.BinaryIndex.Normalization.ElfSegment
| Counter | Description |
|---|---|
elfsegment.normalize.total | Segments normalized |
elfsegment.bytes.modified | Total bytes modified across all passes |
14.6 DI Registration
IElfSegmentNormalizer is registered as TryAddSingleton<ElfSegmentNormalizer> inside AddNormalizationPipelines() in ServiceCollectionExtensions.cs.
14.7 Test Coverage (35 tests)
- Models: DefaultOptions (all enabled), MinimalOptions (relocations only), ModificationRatio zero/computed, enum values
- Service: Constructor null guard, empty input result + SHA-256, ComputeDeltaHash determinism/distinct, NOP canonicalization (3-byte, 2-byte, 4-byte, no-NOP, 7-byte, single-byte), GOT/PLT (JMP disp32, PUSH disp32), alignment padding (INT3 run, zero run, short run), relocation zeroing (valid RELA, invalid entry), jump table (consecutive addresses, random data), full pipeline (deterministic hash, default vs minimal, all-disabled, step-count consistency)
Symbols (Debug Symbol Resolution)
Absorbed from
src/Symbols/intosrc/BinaryIndex/per Sprint 202 (2026-03-04). Project names and namespaces remainStellaOps.Symbols.*to avoid serialized type name breakage.
Overview
The Symbols subsystem provides debug symbol storage, resolution, and marketplace functionality. It is the primary data source for BinaryIndex.DeltaSig when resolving function-level identifiers in stripped binaries.
Project Structure
| Project | Location | Role |
|---|---|---|
StellaOps.Symbols.Core | __Libraries/StellaOps.Symbols.Core/ | Leaf library: models, abstractions (ISymbolRepository, ISymbolResolver), hashing |
StellaOps.Symbols.Client | __Libraries/StellaOps.Symbols.Client/ | HTTP client for Symbols.Server API (depends on Core) |
StellaOps.Symbols.Infrastructure | __Libraries/StellaOps.Symbols.Infrastructure/ | Manifest/blob/runtime plumbing and Blake3 hashing (depends on Core) |
StellaOps.Symbols.Marketplace | __Libraries/StellaOps.Symbols.Marketplace/ | Marketplace scoring and catalog (leaf) |
StellaOps.Symbols.Bundle | __Libraries/StellaOps.Symbols.Bundle/ | Deterministic symbol bundles for air-gapped installs with DSSE manifests (depends on Core) |
StellaOps.Symbols.Server | StellaOps.Symbols.Server/ | Deployable ASP.NET Core WebService with PostgreSQL-backed source/catalog runtime and startup migrations (depends on Core, Infrastructure, Marketplace) |
StellaOps.Symbols.Tests | __Tests/StellaOps.Symbols.Tests/ | Test project covering all Symbols libraries |
Symbols.Server API Surface
The server exposes REST endpoints on port 8080 (mapped to 127.1.0.38:80 in compose):
| Method | Path | Auth | Description |
|---|---|---|---|
| GET | /health | Anonymous | Health check |
| POST | /v1/symbols/manifests | symbols:write | Upload symbol manifest |
| GET | /v1/symbols/manifests/{manifestId} | symbols:read | Get manifest by ID |
| GET | /v1/symbols/manifests | symbols:read | Query manifests (filter by debugId, codeId, binaryName, platform) |
| POST | /v1/symbols/resolve | symbols:read | Batch-resolve symbol addresses |
| GET | /v1/symbols/by-debug-id/{debugId} | symbols:read | Get manifests by debug ID |
Additional marketplace endpoints are mapped via app.MapSymbolSourceEndpoints(). StellaOps.Symbols.Server now has a durable runtime contract:
- Source/catalog endpoints wire
AddSymbolsRuntimePersistence(...), which loads embeddedMigrations/*.sqlat startup and requiresConnectionStrings:Default. - Manifest/blob/resolve endpoints bind to
PostgresSymbolManifestStore; manifests and CAS blobs persist in thesymbolsschema, and address resolution reads the durable manifest rows. - Test projects that need process-local Symbols storage carry their own fixtures under
src/BinaryIndex/__Tests/**;StellaOps.Symbols.Serverdoes not register a runtime fallback when the connection string is absent.
Consumers
- BinaryIndex.DeltaSig (
__Libraries/StellaOps.BinaryIndex.DeltaSig/): ReferencesSymbols.Corefor symbol resolution during delta signature generation. - Cli.Plugins.Symbols (
src/Cli/__Libraries/StellaOps.Cli.Plugins.Symbols/): ReferencesSymbols.CoreandSymbols.Clientfor CLI symbol ingestion commands.
Document Version: 1.7.1 Last Updated: 2026-05-30 (doc↔code reconciliation pass 2: corrected §5b.2 cve_fix_index DDL to include the architecture column + updated_at + the six-column uniqueness key and CHECK constraints; corrected §5b.2 fix_evidence evidence_type to include upstream_match; rewrote §5b.4 ON CONFLICT to match FixIndexRepository.UpsertAsync (6-column conflict target, EXCLUDED references, unconditional state/version/snapshot refresh); fixed BinaryFormat enum casing to Macho; corrected §10.6.1 largeBlobs[].kind example values to source-documented kinds and marked mediaType/sizeBytes/sbomDigest optional) Prior pass (1.7.0, 2026-05-29): HTTP API surface, IBinaryVulnerabilityService / IFixIndexBuilder / IBinaryFeatureExtractor / IBinaryCorpusConnector / IVulnFingerprintGenerator signatures, ops response schemas, config section names, migration/schema inventory, auth model.
