BinaryIndex fold-into-hub — S0 target design (SPRINT_20260722_014 BIN-1)

Status: APPROVED (owner, 2026-08-03 — ledger round 88 ruling C, “all ten S0 drafts approved”). Partially EXECUTED: the hub-side homes landed 2026-08-04 (BIN-3, §5 below). This is NOT a standalone-service merge design: the target was re-scoped by DC-33 — the binary-knowledge plane lands INSIDE src/Vulnerabilities/ (no stellaops_binaryindex, no new connection env), and the binaryindex-web + symbols deployables RETIRE. Verification base: the sprint’s recorded rounds (12/13/14/15/18/21), the live baselines (re-enumerated this pass), and the 2026-08-03 boundary register. ⚠ marks remaining verification.

1. Target shape (DC-33)

VULN-B1 source retirement, 2026-09-11: StellaOps.BinaryIndex.VexBridge and its tests are frozen under src/__Obsoleted/BinaryIndex/. BIN-5 removed the resolution controller; the bridge had no remaining caller beyond an unused host DI registration. That registration, project edge and solution entries are removed. This source change does not retire the running BinaryIndex host or resolve the remaining BIN-4/BIN-8/BIN-9/BIN-10 acceptance criteria.

2. Table inventory + dispositions (P13)

Re-enumerated this pass: binaries 15, binary_index 4, groundtruth 11, symbols 4, golden_sets 4 (in BinaryIndex.GoldenSet, D12-pinned to this program’s wave).

3. Build-boundary dispositions (2026-08-03 register)

binaryindex-web pins: concelier + excititor (become Contracts-only per DC-25 at the fold — the VexBridge repoint IS the DSSE push path onto the issuer API), attestor (011 Evidence SDK), policy, scanner (DC-15/017 seams), symbols (intra-plane; dies by the fold). The fold RETIRES both deployable keys, which deletes every pin row where a retiring key is the CONSUMER — that is every binaryindex-web → * row plus binaryindex-web → symbols, and it does make this program a large pin-count reducer.

CORRECTED 2026-08-06 (MBI-4 §6 G4) — “wholesale” was too strong, and the residue is the largest part of this program’s boundary work. Retiring a host removes the pins where that host is the consumer; it cannot remove a library from another key’s build. Of 014’s four register rows, one (binaryindex-web → symbols) has a retiring key as the consumer and dies as described. The other three have scanner as the consumer and 23 measured foreign projects between them: scanner-web → binaryindex (6: Contracts, Decompiler, Disassembly+.Abstractions, Ghidra, Semantic, via Scanner.Reachability), scanner-worker → binaryindex (16: the analysis stack incl. Core, Corpus, DeltaSig, Fingerprints, FixIndex, ML, Normalization, Persistence, via a direct Scanner.Worker reference) and scanner-worker → symbols (1: Symbols.Core, through BinaryIndex.DeltaSig). Under DC-33 those libraries move into src/Vulnerabilities/— out of one foreign family and into another — so the artifact seam the register records for all three (corpus artifact + hub API) is real work that the fold does not do for free. It is gated in SPRINT_20260722_014 BIN-5, coordinated with 017 SCN-3 which carries the Scanner side.

Q-4 measured disposition, corrected 2026-08-25. The subsequent severance/classification waves reduced scanner-worker → binaryindex to three projects: ML, GroundTruth.Reproducible, and GroundTruth.Abstractions (generate-build-boundary-report.ps1 -Check: 59 pins / 64 maximal pairs, the row’s foreignProjectCount = 3). The owner recorded a conditional answer: ML is Scanner; reproducibility is neutral only if it is algorithms/primitives without persistent service-owned data, otherwise Scanner. Source selects the fallback. .Abstractions defines KPI, symbol observation/raw-document/source-state, and security-pair repositories plus SecurityPairService; Reproducible.BundleExportService reads IKpiRepository and ValidationHarnessService consumes the security-pair service. Its project closure has no persistence package, but its source owns data contracts and ground-truth workflow. The reproducibility capability therefore moves to Scanner.

The physical move could not be a directory rename of all three projects. .Abstractions has 13 direct csproj consumers across CLI, DeltaSig, Validation, connector libraries, Reproducible, and tests. BIN-4(c2)/BIN-5 therefore extracted the Scanner-owned implementation without moving that surface or creating a new cross-family edge. The groundtruth tables remain the bench/QA plane approved in §4 item 2; Q-4 does not transfer that data to Scanner or the Vulnerabilities hub. ML’s production consumers are Scanner Worker/AiMlSecurity; its old-family Ensemble and test/benchmark edges travel or repoint with the implementation slice.

ML physical rehome executed 2026-08-26. Source commit b78fb9c1964b41d87fad3b8333ab7e610e76ec4d moves StellaOps.BinaryIndex.ML byte-for-byte under src/Scanner/__Libraries/, keeps its assembly/namespace, repoints the two production consumers and the retained Ensemble/test/benchmark consumers, and leaves the neutral Decompiler/Semantic foundation in place. At that source boundary, scanner-worker → binaryindex reduced from three projects to the two reproducibility projects. Verified-by authority 1/2: Scanner Worker Release build; AiMlSecurity 13/13; Ensemble 37/37; focused Worker AI/ML stage 4/495; boundary -Check reported 0 unpinned/stale/grown/activation/purity failures and only the intentionally deferred generated-report staleness. Integration commit 869e5aaed09dd5f1bc7e4c9d8cdc0b99232dd100 subsequently refreshed the report and dispositions; re-verify with pwsh ./tools/scripts/build-boundary/generate-build-boundary-report.ps1 -Check.

Reproducibility physical split executed 2026-08-26. Source commit 0f3c71e3eab819ecdffb80dbeaa275b14e986fb3 extracts the stateless rebuild/determinism engine into Scanner.BuildProvenance/Reproducibility; it does not move or duplicate the mixed GroundTruth.Abstractions surface. BinaryIndex keeps repository-backed bundle export/import, KPI regression and validation harness tooling. Scanner BuildProvenance therefore reaches no BinaryIndex project, and its real AddBuildProvenance composition now registers the concrete services its verifier requires. The generator named scanner-worker|binaryindex stale before pin deletion and, after deletion, reports 0 unpinned/stale/grown/activation/purity failures. BuildProvenance 37/37, retained ground-truth tooling 92/92, Scanner Worker Release 117 projects / 0 errors. The integrated report/dispositions are current in 869e5aaed09dd5f1bc7e4c9d8cdc0b99232dd100: 56 pins / 61 maximal pairs (59 default + 2 opt-in), with boundary conformance 40/40.

4. What S1 must decide

1. DC-33 execution confirmed (the re-scope is already owner-ratified; S1 records the fold order: connector family first, fingerprint tables second, VexBridge repoint third, Symbols CAS last).
2. Ground-truth + golden-set plane disposition — sweeps done (§2): both are bench/QA planes with CLI-tooling + test consumers only. OWNER RULING (2026-08-03, review item A6): approved — retire to test fixtures + an explicit bench store; nothing rides runtime migrations into the hub. The generic /api/v1/stats route DIES at the fold unless a named consumer appears.
3. symbols.blobs object-store prefix + budget row (matrix §6.2).
4. Sequencing vs 008: the fold adds schemas to stellaops_vuln — land AFTER the G1 cutover window or coordinate inside it (the hub baseline is pre-deployment-amendable only until G1).

Remaining verification CLOSED (2026-08-03):

• Scanner consumer calls: NONE — the pipeline consumes the offline NDJSON build-id index artifact (§1); the CLI is the HTTP consumer.
• Routes (4): /api/v1/resolve, /api/v1/ops/binaryindex, /api/v1/golden-sets, and the suspiciously generic /api/v1/stats— all to binaryindex.stella-ops.local. The fold’s route change must decide whether /api/v1/stats survives as a hub route or dies (a binaryindex-claimed generic path is a collision risk either way).
• Deploy reality: BOTH retiring hosts run in the live main stack (binaryindex-web, symbols) — the fold retires two live containers; S7+ needs deploy windows.
• Ground-truth + golden-set reader sweeps: §2 (bench/QA planes; no runtime service consumers).

5. BIN-3 execution record (2026-08-04) — hub-side homes

Landed inside src/Vulnerabilities/ while the hub baseline was still pre-deployment-amendable (verified before acting: docker-compose.vulnerabilities.yml is not in the standing stack’s config_files list and no hub container runs on it — the g0full-project rehearsal containers are scratch and do not count). This is the last window in which amending 001_v1_vulnerabilities_baseline.sql in place is correct; after the 008 G1 cutover every change here is a forward migration.

Connector family. DistroBinaryCorpusSourceConnector (distro-binary-corpus) on the ordinary hub SPI, registered in AddVulnerabilitiesIngestScheduling — so completion, tombstoning and capacity accounting come from runtime.source_generation like every other source and the hub core never learns that binaries are special. It streams repository index documents (deb-packages / rpm-repomd / apk-index) under a new binary-corpus doc kind, Full scope by construction, and fails closed on an unconfigured fleet or an empty index body. Package payloads are never fetched here — that split is the capacity invariant behind design §9.1.

Baseline amendments — 15 tables, all tenant-free (P12), all P13-headed, zero seed rows:

Deviations from the §2 per-table list (each is a “never port a layer this merge exists to fix” call, not an omission):

1. vulnerable_buildids and fingerprint_claims do not become tables. Both assert “this build-id / this fingerprint is affected by this CVE” — which is a fact under the fact class. They collapse into facts.fact_current + binaries.binary_fact, so identity (D4), DSSE provenance and generation tombstoning have exactly one home.
2. fingerprint_matches and delta_sig_match do not enter the hub. They are keyed by tenant + scan id: per-scan results are scan evidence, not global knowledge, and the hub is tenant-free (P12).
3. fix_index_priority does not become a table. Its only content would be seed rows, which §2.11 forbids in a migration; method precedence is a deterministic ranking applied at read time from code — the same “rank at read, never at write” rule facts.exploit_evidence already follows.
4. binary_vuln_assertion does not become a table — same reason as (1); it is the fact itself.
5. symbols.blobs splits: catalog row in symbols.blob_catalog, bytes to the object store (P16). The catalog carries no payload column at all, guarded by test.

golden_sets disposition executed. Owner ruling A6 (retire to test fixtures + an explicit bench store; nothing rides runtime migrations into the hub) is now enforced mechanically: HubSchemaConventionTests.BenchAndQaPlanes_NeverEnterARuntimeMigration fails if a groundtruth or golden_sets schema ever appears in the hub, and the baseline header states the rule. The live golden_sets schema is deliberately left in place: BinaryIndex.WebService still migrates and serves it, so removing it before the deployable retires would break a working surface rather than clean one up. Its retirement rides the deployable; the schema drop is BIN-10’s separate destructive approval. What BIN-3 guarantees is that the retirement cannot orphan the plane into the hub.

Correction (2026-08-11, BIN-5). The sentence above used to add “and the CLI golden-set commands still read it”. They do not. Measured: every stella golden subcommand (init, validate, import, list, show) is local-file tooling over a ./golden-corpus directory of YAML, and no CLI code path calls /api/v1/golden-sets/* — GoldenSetController has no in-repo consumer at all. That makes BIN-8’s retirement gate smaller than it was written, not larger: the golden-set API is not a live surface a compose-key retirement would take down.

The CLI-side end named here is done: stella golden build-index was REMOVED rather than wired up, because the extraction it lacked (BinaryIndex.Analysis) runs hub-side under DC-33, the artifact it would emit now has a producer (the hub’s BuildIdIndexExporter), and golden sets are bench material by ruling A6. The remaining subcommands survive as local bench tooling and touch neither the deployable nor the schema.

§2.11 finding in the retiring plane (not fixed here). src/BinaryIndex/StellaOps.Symbols.Server/Migrations/001_v1_symbols_server_baseline.sql seeds fabricated demo rows into a runtime migration: two symbol sources (“Microsoft Public Symbols”, “Ubuntu Debuginfod”) with invented sync counters, and two catalog packs with placeholder digests (sha256:aabbccdd). The symbols deployable runs live, so the migration is applied and removal is forward-only (ADR-004) — it is recorded for the retirement window rather than edited in place. None of it is carried into the hub; symbols.source/catalog_entry are structure only.

6. BIN-11 execution record (2026-08-11) — the write path

§5 landed the schema and the connector; BIN-4 landed the domain layer and BIN-5 the read routes and the artifact exporter. None of that wrote a row. BinaryCorpusWriter had no caller outside its own tests, and the connector fetches repository indexes only — by the explicit contract quoted in §5, because unpacking is “the hub-side analysis stage”, which did not exist. This section records that stage.

Where it lives. src/Vulnerabilities/__Libraries/StellaOps.Vulnerabilities.BinaryAnalysis/, depending one-directionally on …Binaries. Deliberately NOT inside …Binaries: the hub WebService references that library for the read routes and the doctor checks, and the stage carries an archive/object-format stack (SharpCompress, System.Formats.Tar) plus an HttpClient. A host that only reads the binary plane must not ship a decompression surface it never calls. Only vulnerabilities-worker composes the stage.

The chain, end to end.

DistroBinaryCorpusSourceConnector      (§5) → ingest.raw_document  [INDEX documents only]
  └─ BinaryCorpusAnalysisStage         (§6) → reads them past ingest.normalize_checkpoint
       ├─ RepositoryIndexParser              → CorpusPackageRef per package
       ├─ IPackagePayloadSource              → package bytes, TRANSIENT, digest-verified
       ├─ PackageBinaryExtractor             → ar / tar / gz|xz|bz2 walk, bounded
       ├─ NativeBuildIdReader                → (build-id, file digest, format, arch, type)
       └─ BinaryCorpusWriter          (BIN-4) → binaries.binary_identity + binary_package_map
            └─ BuildIdIndexExporter    (BIN-5) → the NDJSON artifact Scanner already consumes

Invariants it holds, and how.

• Payloads are transient. IPackagePayloadSource spools to a FileOptions.DeleteOnClose temp file, so a crash or an abandoned run leaves nothing to sweep, and verifies the sha256 the index declared — a mismatch is refused as a supply-chain event, not parsed anyway. Asserted three ways by test: the ingest store still holds only the index document, no stored payload equals a fetched archive, and total stored bytes stay strictly below the fetched bytes.
• Generation marking is closed-only. Rows carry the newest completed runtime.source_generation; with only an active generation the stage writes nothing at all, because rows marked with a generation that never closes can never be tombstoned (D4).
• Idempotence is measured, not assumed. A second pass over an unchanged generation writes zero row versions, proved by xmin across both tables after deleting the checkpoint so the stage genuinely re-reads and re-writes.
• No new table, so no forward migration. The stage checkpoints in the existing ingest.normalize_checkpoint, which is keyed by source_key alone; the fact pump refuses distro-binary-corpus by name, so the two can never contend for the row.

Coverage, stated honestly. Debian Packages and Alpine APKINDEX are enumerated and their .deb/.apk payloads unpacked. The index’s transport container is stripped first, by RepositoryIndexPayloadReader, and it is chosen by SNIFFING THE BYTES rather than by reading the URL: gzip, xz, bzip2 and tar (APKINDEX.tar.gz included, signature member and all) are peeled, a plain index passes through untouched for air-gapped estates, and .zst refuses by name. The reader sits on the read side because ingest.raw_document is content-addressed on the bytes the mirror served — unwrapping before storage would make the stored digest disagree with the upstream file. Before 2026-09-04 that step existed only as a comment claiming another component owned it, so apk-index enumerated zero packages from every real Alpine mirror (D-BIN11-7 / D-BIN11-8).

Where a package’s bytes are fetched from is composed by two components and the seam between them is load-bearing, so it is stated here rather than left to each side’s comments. HttpPackagePayloadSource.ResolvePackageUri picks the BASE from the index URI’s shape — never from its host, so any mirror resolves identically — and RepositoryIndexParser must state each package’s path against that base:

An Alpine repository publishes APKINDEX.tar.gz and its .apk files in one directory, so the architecture is already the last segment of the index URI. Re-stating it in the path asks the mirror for …/main/x86_64/x86_64/<name>-<version>.apk, which is what every package fetch of the 2026-09-04 live run received a 404 for while enumeration itself succeeded (D-BIN11-9, fixed 2026-09-05). PackageUriResolutionTests pins the seam end to end — real index bytes through the real parser into the resolver, asserted against the URL a mirror serves — because each half was independently green while the composed URL was wrong.

rpm-repomd refuses loudly: repomd.xml enumerates no packages at all — it only locates primary.xml.gz — so RPM support is a second transient fetch, a capability rather than a parser. Refusing rather than returning an empty list is load-bearing, not fastidiousness: an index kind that silently yields zero packages lets the generation close as a complete Full inventory, and a completed Full generation is what authorizes D4 absence-tombstones. Empty-but-clean would therefore not be a missing answer — it would be positive authority to delete knowledge the hub already holds. Filed as D-BIN11-3, which names this “silently empty reads as clean” shape across its three instances in this plane and states the standing rule: a component that cannot do its job throws or reports unhealthy, and never returns the empty set. .zst-compressed data.tar likewise refuses by name: SharpCompress 0.38 carries no zstd decoder, so adding it is a dependency decision with a BUSL-1.1 license gate.

Refusal recovery (2026-08-24). The stage preserves the immutable index for retry: an unsupported or malformed index raises a contextual failure naming the source, URI and indexKind, writes no binary rows, and does not advance ingest.normalize_checkpoint. The operator action is explicit in the failure — disable the unsupported repository or add parser support, then rerun the source. A valid parsed document advances the checkpoint only after all analysed package transactions commit. This closes a source defect where RepositoryIndexFormatException was logged and converted to an empty success, despite the refusal contract above.

The stage extracts build-ids only. Fingerprint and delta-signature generation is still absent hub-side, which is why the disassembly libraries have no hub caller yet and why their relocation is gated on the D-BIN5-6 ruling rather than on more implementation.

Capacity. Vulnerabilities:Binaries:Analysis:MaxPackagesPerDocument defaults to 250 against a Debian index of ~60k packages. Unpacking a whole repository is the largest disk and CPU cost in the fold and §9.1’s budget was written before any of it ran, so the default is a bounded slice an operator raises deliberately once a real run has been measured. Per-package, per-entry, total-uncompressed and entry-count limits bound a hostile or malformed archive.

Two defects in existing code, fixed here rather than inherited. BuildIdIndexExporter used WriteLineAsync, which emits CRLF on Windows — in a stream its own header calls digest-verified downstream, so one corpus produced different bytes depending on the exporting host; it now writes a bare LF. And the ingest scheduler sent every completed source to the normalization pump, which throws on a source key it has no normalizer for, so an enabled binary corpus reported FAILED on every cycle after a successful ingest; dispatch is now explicit.

For reference when reading the retiring plane: BinaryIndex.Corpus.Debian.DebianPackageExtractor opens a .deb with SharpCompress.ArchiveFactory, which does not implement ar — the resulting InvalidFormatException goes to its own catch-and-log and returns an empty result, so that path has always reported “no binaries” for every Debian package. The ar reader in PackageBinaryExtractor is written in-tree for exactly this reason.

7. BIN-6 execution record (2026-08-24) — the live Console repoint

The Patch Map was the one reachable Console consumer of the retiring BinaryIndex HTTP plane. It now consumes the two already-deployed Vulnerabilities hub reads without adding a gateway route:

• GET /api/vulnerabilities/v1/binaries/patch-coverage, with the hub’s vulnerability/component filters and signature-count response vocabulary;
• GET /api/vulnerabilities/v1/binaries/patch-coverage/{vulnerabilityId}, whose empty symbols array renders as “No delta signatures held” rather than a failed read.

There is deliberately no third /matches call. The old affected-image payload is tenant + scan keyed; publishing it from this tenant-free corpus would break D-BIN3-2’s boundary. The UI branch and client method retired, and an HTTP contract test negatively pins that PatchCoverageHttpClient cannot emit /matches. Labels now say vulnerable/patched signatures, never images, fleet state, “unknown”, or “safe”. Because the list response carries no total count, paging requests one look-ahead row and displays only the requested page; it does not invent a corpus total.

The client preserves raw HttpErrorResponse objects. The page passes them to the canonical error panel, keeping a human message and the HTTP/body detail together; retry repeats the failed list or detail operation, so a recovered detail read cannot fall into an empty details view.

Execution was source-only. Before the repoint, the running OpenAPI and direct read probes confirmed both hub routes are present/auth-gated and /matches is absent. No image, gateway, route, container, configuration, corpus ingest or other live mutation was performed.

8. BIN-5 decision preflight (2026-08-24) — the CLI resolve POST is not a mechanical repoint

The remaining HTTP consumer is exactly stella binary resolve: it sends one VulnResolutionRequest to POST /api/v1/resolve/vuln, using a pseudo-PURL, a build-id and an optional distro/release hint, and consumes one VulnResolutionResponse. The carried contract is materially wider than that call: it also admits file/text/BLAKE3 hashes and base64 fingerprint bytes, defines a batch POST, and returns one aggregate Fixed/Vulnerable/NotAffected/Unknown verdict with confidence, cache, DSSE and hybrid-diff evidence fields.

The hub does not implement that aggregate. Its authenticated advisory:read binary routes return an ordered list of current facts for a build-id or fingerprint id, or distro fix-index rows. A BinaryFactView can preserve issuer, vulnerability, product, status, severity and fixed-version truth, but it neither chooses one winner nor supplies the retiring plane’s confidence, cache, DSSE, hybrid-diff or hash semantics. This is deliberate: BIN-4 recorded BinaryVulnerabilityService as not ported because it is tenant/RLS bound to the dissolved binary_vuln_assertion model. The old resolver also contains semantics that cannot become compatibility policy by accident: no matches becomes NotAffected, the highest-confidence row wins, and a build-id-catalog match becomes Fixed.

No adapter was implemented. DC-33 and D-BIN5-7 say the CLI consumer and VulnResolutionContracts.cs must leave BinaryIndex; they do not decide the wire version or how many hub facts collapse into one legacy verdict. The owner/API ruling must choose one of these explicit contracts before BIN-5 can close:

1. Versioned legacy adapter: choose its hub path and lifetime, define fact aggregation and status precedence, state which identifier modes remain supported, and define 400/empty/unknown behavior; also decide the batch, cache, DSSE and hybrid-diff fields rather than emitting fabricated defaults.
2. Hub-native facts contract: publish a producer-owned versioned contract from the Vulnerabilities family, return the fact list without aggregation, repoint the CLI, and explicitly accept or stage the compatibility break for the retiring POST.

Either option rides the existing hub gateway prefix and advisory:read; neither needs a new scope or gateway route. Deployment and removal of the old route remain BIN-8 live-window work after the source contract is decided and validated.