BinaryIndex

Status: Implemented Source: src/BinaryIndex/ Owner: BinaryIndex Guild (Scanner-integrated)

Purpose

BinaryIndex detects vulnerable binaries independently of package metadata. Package version strings can lie — backports, custom builds, and stripped metadata all defeat name/version matching — so BinaryIndex identifies vulnerabilities binary-first, using Build-IDs, hash catalogs, function fingerprints, and delta (patch) signatures. It adds backport-aware fix resolution and emits signed (DSSE) evidence plus VEX candidates describing patch status.

Audience: Scanner integrators and operators who need defensible vulnerable-binary detection where package metadata is untrustworthy, and curators maintaining the golden-set and ground-truth corpora.

Services

BinaryIndex ships as three deployable services plus a large library set (router service names binaryindex and symbols; default port 8080).

• StellaOps.BinaryIndex.WebService — REST API for vulnerability resolution, golden-set curation, patch-coverage statistics, and ops/health (Program.cs). Backed by PostgreSQL and a Valkey/Redis resolution cache. Registers Authority resource-server authentication with a fallback policy that requires every endpoint to be authenticated unless it opts out with [AllowAnonymous] (health endpoints only).
• StellaOps.Symbols.Server — symbol manifest upload/query/resolve and a symbol-source/marketplace catalog (Program.cs, Endpoints/SymbolSourceEndpoints.cs). Authorized by the symbols:read / symbols:write scopes.
• StellaOps.BinaryIndex.Worker — background processing (e.g. the reproducible-build job in Jobs/ReproducibleBuildJob.cs).

API Surface

BinaryIndex WebService (binaryindex)

Vulnerability resolution (Controllers/ResolutionController.cs, route api/v1/resolve):

Golden-set curation (Controllers/GoldenSetController.cs, route api/v1/golden-sets):

Patch coverage / Patch Map Explorer (Controllers/PatchCoverageController.cs, route api/v1/stats/patch-coverage):

Ops / observability (Controllers/BinaryIndexOpsController.cs, route api/v1/ops/binaryindex):

Health/liveness/readiness (/health, /healthz, /readyz) and BuildInfo are mapped anonymously for container probes.

Symbols Server (symbols)

Symbol manifests (Program.cs): POST /v1/symbols/manifests, GET /v1/symbols/manifests/{manifestId}, GET /v1/symbols/manifests, POST /v1/symbols/resolve, GET /v1/symbols/by-debug-id/{debugId}. Symbol sources & marketplace (Endpoints/SymbolSourceEndpoints.cs): GET/POST/PUT/DELETE /api/v1/symbols/sources*, GET /api/v1/symbols/marketplace*, POST /api/v1/symbols/marketplace/{entryId}/install|uninstall, POST /api/v1/symbols/marketplace/sync.

Key DTOs (StellaOps.BinaryIndex.Contracts/Resolution/VulnResolutionContracts.cs)

• VulnResolutionRequest — Package (required PURL/CPE), FilePath, BuildId, Hashes (FileSha256/TextSha256/Blake3), Fingerprint + FingerprintAlgorithm, CveId, DistroRelease. At least one identifier (BuildId, Fingerprint, or one of the hashes) is required.
• VulnResolutionResponse — Package, Status, FixedVersion, Evidence, AttestationDsse, ResolvedAt, FromCache, CveId.
• ResolutionStatus enum — Fixed, Vulnerable, NotAffected, Unknown.
• ResolutionEvidence — MatchType, Confidence, DistroAdvisoryId, PatchHash, MatchedFingerprintIds, SourcePackage, FixMethod, FixConfidence, ChangedFunctions, and a HybridDiff evidence chain (semantic edit script + symbol patch plan + patch manifest digests).
• Match-type constants (ResolutionMatchTypes): build_id, fingerprint, hash_exact, package, range_match, delta_signature, fix_status, unknown. Fix-method constants (ResolutionFixMethods): security_feed, changelog, patch_header, delta_signature, upstream_patch_match, unknown.

Components

The module comprises ~40 libraries under src/BinaryIndex/__Libraries/ plus the three services above. Highlights:

Identity & resolution:

• StellaOps.BinaryIndex.Core — binary identity models, feature extractors (ElfFeatureExtractor, PeFeatureExtractor, MachoFeatureExtractor), and ResolutionService.
• StellaOps.BinaryIndex.Contracts — API/resolution contracts and DTOs.
• StellaOps.BinaryIndex.Cache — Valkey/Redis resolution cache and function-IR cache.
• StellaOps.BinaryIndex.Persistence — PostgreSQL storage adapters and EF-style repositories for the binaries schema.
• StellaOps.BinaryIndex.FixIndex — patch-aware backport / fix-status handling.
• StellaOps.BinaryIndex.Fingerprints — function fingerprint storage and matching.

Delta-signature (backport detection) stack:

• StellaOps.BinaryIndex.Disassembly (+ .Abstractions, .Iced, .B2R2) — pluggable disassembly (Iced for x86/x64; B2R2 for ARM/MIPS/RISC-V/multi-ISA lifting).
• StellaOps.BinaryIndex.Normalization — deterministic instruction normalization.
• StellaOps.BinaryIndex.DeltaSig — delta-signature generation/matching, hybrid-diff composition, attestation predicates, and VEX bridging.
• StellaOps.BinaryIndex.Diff — byte-range / function diffing and rename detection.

Semantic-diffing stack (implemented):

• StellaOps.BinaryIndex.Semantic — IR-level semantic graph fingerprints.
• StellaOps.BinaryIndex.Decompiler — decompiled-code AST parsing and comparison.
• StellaOps.BinaryIndex.Ghidra — Ghidra Headless integration.
• StellaOps.BinaryIndex.ML — function-embedding similarity.
• StellaOps.BinaryIndex.Ensemble — multi-signal decision fusion (EnsembleDecisionEngine, WeightTuningService).

Corpus & ground truth:

• StellaOps.BinaryIndex.Corpus (+ .Debian, .Rpm, .Alpine) — binary-to-advisory corpus ingestion, plus library connectors (glibc, openssl, zlib, curl).
• StellaOps.BinaryIndex.GroundTruth.* — Buildinfo, Ddeb, Debuginfod, Mirror, Reproducible, and SecDb ground-truth sources.
• StellaOps.BinaryIndex.GoldenSet — golden-set definitions, validation, and review workflow.
• StellaOps.BinaryIndex.Validation (+ .Abstractions) — validation harness and KPIs.
• StellaOps.BinaryIndex.VexBridge — VEX candidate emission.

Symbols stack: StellaOps.Symbols.Core, .Infrastructure, .Client, .Bundle, .Marketplace.

Configuration

Each service binds its own configuration; BinaryIndex is not embedded in Scanner/Concelier settings.

BinaryIndex WebService (appsettings.json):

• ConnectionStrings:Redis — required Valkey/Redis resolution cache (startup fails if absent).
• Resolution — DefaultCacheTtl, MaxBatchSize (500), EnableDsseByDefault, MinConfidenceThreshold.
• ResolutionCache — FixedTtl, VulnerableTtl, UnknownTtl, KeyPrefix, EnableEarlyExpiry, EarlyExpiryFactor.
• VexBridge — SignWithDsse, DefaultProviderId, DefaultStreamId, MinConfidenceThreshold, MaxBatchSize.
• Authority:ResourceServer, Router, RateLimiting sections (resource-server auth, router messaging transport, rate limiting).

Core library options (StellaOps:BinaryIndex section, BinaryIndexOptions): B2R2Pool (per-ISA lifter pool, warm-preload ISAs intel-64/intel-32/armv8-64/armv7-32), SemanticLifting (B2R2Version 0.9.1, NormalizationRecipeVersion v1), FunctionCache, Persistence (schema, pooling, retries), and Ops (endpoint toggles, redacted keys).

Key capabilities:

• Multi-tier binary identification (package/version range, Build-ID/hash catalog, function fingerprints, delta signatures).
• Binary identity extraction across ELF (GNU Build-ID), PE (CodeView GUID), and Mach-O (UUID).
• Backport-aware fix resolution with confidence-weighted evidence and DSSE attestation.
• Offline-first design with deterministic outputs.

Pluginized Runtime Composition

BinaryIndex uses the shared plugin probe surface for analyzer/data-pack classification:

• GET /internal/plugins/status
• POST /internal/plugins/probe

Compose plugin overlays mount the canonical roots for binaryindex-web:

Base compose also mounts Router/Messaging bundle roots read-only into binaryindex-web for shared Router registration transport. Those mounts use /app/plugins/router/base, /app/plugins/messaging/base, and /app/trust-roots/plugins/router; they are not BinaryIndex analyzer bundles.

devops/etc/plugins/binaryindex/registry.yaml mirrors the current analyzer surface catalog. Core host entries (binaryindex.identity.build-id, binaryindex.parser.*, and binaryindex.fingerprint.stable-hash) must work in the base image and are not represented by a mounted base bundle. The packaging script can produce signed executable analyzer bundles for binaryindex.disassembly.iced and binaryindex.disassembly.b2r2:

.\devops\build\package-runtime-plugins.ps1 -Module binaryindex -Profile recommended -SignBinaryIndexBundles -UseOfflineDevSigner
.\devops\build\package-runtime-plugins.ps1 -Module binaryindex -Profile harness -SignBinaryIndexBundles -UseOfflineDevSigner

binaryindex.disassembly.ghidra and binaryindex.corpus.builder remain future executable analyzer/corpus-builder producers. All optional executable analyzers are non-required and report notMounted until runtime loader admission validates the mounted manifest, checksums, payload, and signature before code load. binaryindex.datapack.* entries remain non-executable data packs, report disabled, and must not be loaded as plugins.

The WebService no longer compiles the optional B2R2 analyzer implementation or Router/Messaging transport implementation payload directly. Its ops controller reports the lifter as not-mounted until a signed analyzer bundle is admitted. The base mounted-runtime proof on 2026-06-07 rebuilt stellaops/binaryindex-web:dev in mounted-plugin mode, passed image audit, started healthy, and returned POST /internal/plugins/probe with isComplete=true for the five required host-core rows. The Worker still references StellaOps.BinaryIndex.Builders for the current reproducible-build job, so corpus-builder externalization remains future loader work.

StellaOps.Symbols.Server is de-scoped from BinaryIndex analyzer bundle mounts until it exposes its own executable plugin/probe contract. Symbols continues to serve the symbols:* API surface without /internal/plugins/* endpoints.

CLI

Delta-signature commands ship in the CLI (src/Cli/StellaOps.Cli/Commands/DeltaSig/, plugin StellaOps.Cli.Plugins.DeltaSig):

stella deltasig extract    # Extract signatures from a binary
stella deltasig author     # Author a vuln/patched signature pair
stella deltasig sign       # Sign a signature as a DSSE envelope
stella deltasig verify     # Verify a signed signature
stella deltasig match      # Match a binary against signatures
stella deltasig pack       # Create a signature pack (ZIP)
stella deltasig inspect    # Inspect a signature or envelope

Ground-truth validation commands ship under StellaOps.Cli.Plugins.GroundTruth.

Persistence

PostgreSQL schema binaries (with binaries_app helper schema and forced row-level security per tenant_id). One live embedded migration — StellaOps.BinaryIndex.Persistence/Migrations/001_v1_binaryindex_baseline.sql — carries the collapsed pre-1.0 chain (the old 001_initial_schema.sql … 005_validation_kpis.sql files are archived under Migrations/_archived/pre_1.0/mig061/ and are not embedded); the Symbols server has its own baseline under StellaOps.Symbols.Server/Migrations/001_v1_symbols_server_baseline.sql. Core tables (baseline) include:

• binaries.binary_identity — file/text/BLAKE3 hashes, build_id + build_id_type (gnu-build-id/pe-cv/macho-uuid), format (elf/pe/macho), architecture, strip state.
• binaries.corpus_snapshots, binaries.binary_package_map — distribution corpus snapshots and binary↔package mappings.
• binaries.vulnerable_buildids, binaries.binary_vuln_assertion — known-vulnerable Build-IDs and per-binary assertions (status affected/not_affected/fixed/unknown; method range_match/buildid_catalog/fingerprint_match/fix_index).
• binaries.cve_fix_index, binaries.fix_evidence, binaries.fix_index_priority — patch-aware fix index with evidence trail.
• binaries.vulnerable_fingerprints, binaries.fingerprint_corpus_metadata, binaries.fingerprint_matches — function-level fingerprint catalog and match results.

Valkey/Redis holds the authoritative resolution cache. The WebService no longer falls back to in-memory stores for golden sets, vulnerability data, delta signatures, or the resolution cache.

Dependencies

• PostgreSQL (owns the binaries schema; auto-migrated on startup).
• Valkey/Redis (resolution and function-IR caches).
• Authority (resource-server authentication; symbols:read/symbols:write scopes for the Symbols server).
• Scanner (consumes BinaryIndex during native binary analysis).
• Concelier (advisory data feeding corpus/fix-index correlation).

Related Documentation

• Architecture: ./architecture.md
• API reference: ./api-reference.md
• Hybrid Diff Stack: ./hybrid-diff-stack.md
• Semantic Diffing: ./semantic-diffing.md
• Ground-Truth Corpus: ./ground-truth-corpus.md
• DeltaSig v2 Schema: ./deltasig-v2-schema.md
• Golden-Set Schema: ./golden-set-schema.md
• High-Level Architecture: ../../ARCHITECTURE_OVERVIEW.md
• Scanner Architecture: ../scanner/architecture.md
• Concelier Architecture: ../concelier/architecture.md

Current Status

Library and service implementation is complete: ELF (Build-ID), PE (CodeView GUID), and Mach-O (UUID) identity extraction; multi-architecture disassembly (Iced + B2R2); deterministic normalization; delta-signature generation/matching; backport-aware fix index; golden-set curation; and a VEX bridge. The semantic-diffing stack (IR semantics, decompiler/AST comparison, Ghidra integration, ML similarity, ensemble fusion) is implemented as shipped libraries — see the note below. BinaryIndex is integrated into Scanner’s native binary analysis pipeline.

Appendix — Semantic Diffing (shipped; original phasing retained for context)

Reconciliation note. Earlier revisions of this document described semantic-level binary diffing as a future roadmap. The work has since shipped: the four SPRINT_20260105_001_*_BINDEX_semdiff_* sprints are archived under docs-archive/implplan/, and the corresponding libraries (StellaOps.BinaryIndex.Semantic, .Decompiler, .Ghidra, .ML, .Ensemble) exist in src/BinaryIndex/__Libraries/. The phase tables and metrics below are retained for historical context and reflect the original design phasing and targets, not re-validated current status.

Semantic-level binary diffing detects function equivalence based on behavior rather than syntax, addressing limitations in byte/symbol-based matching when dealing with:

• Compiler optimizations (same source, different instructions)
• Stripped binaries (no symbols)
• Cross-compiler builds (GCC vs Clang)
• Obfuscated code

Phases (delivered)

Libraries (shipped)

• StellaOps.BinaryIndex.Semantic — IR lifting and semantic graph fingerprints
• StellaOps.BinaryIndex.Corpus — function behavior database (corpus ingestion)
• StellaOps.BinaryIndex.Ghidra — Ghidra Headless integration
• StellaOps.BinaryIndex.Decompiler — decompiled code AST comparison
• StellaOps.BinaryIndex.ML — function-embedding similarity
• StellaOps.BinaryIndex.Ensemble — multi-signal decision fusion

Target Outcomes (original goals)

The accuracy/coverage percentages above are the original design targets; they are not re-validated KPIs. Live validation KPIs are tracked via StellaOps.BinaryIndex.Validation and the groundtruth.validation_kpis / validation_pair_kpis / kpi_baselines / regression_checks tables in the persistence baseline 001_v1_binaryindex_baseline.sql.

Sprint Files (archived)

• docs-archive/implplan/SPRINT_20260105_001_001_BINDEX_semdiff_ir_semantics.md
• docs-archive/implplan/SPRINT_20260105_001_002_BINDEX_semdiff_corpus.md
• docs-archive/implplan/SPRINT_20260105_001_003_BINDEX_semdiff_ghidra.md
• docs-archive/implplan/SPRINT_20260105_001_004_BINDEX_semdiff_decompiler_ml.md

Architecture Documentation

See ./semantic-diffing.md for comprehensive architecture documentation.