Clean-room Trivy-compatible BoltDB writer — design memo
Status: Implemented (Sprint 20260513_001 E2, Path B) Owner: Concelier / Implementer Counsel basis: docs/legal/decisions/trivy-db-builder-counsel-thread-airgap-export-reply.md
Purpose
Concelier emits a Trivy-compatible vulnerability DB (trivy.db + metadata.json) directly from its own canonical advisory store, with no dependency on the upstream aquasecurity/trivy-db builder. This memo records how the on-disk schema was derived and states the clean-room boundary that was observed, so the artifact’s provenance is auditable.
“Trivy-compatible” is used deliberately. No Aqua endorsement is implied.
Clean-room boundary (what was and was NOT used)
Per counsel (section 3, “code/expression versus interface/format/function”): copyright does not protect an idea, procedure, process, method of operation, or the format of data files used to exploit program functions. The schema was derived only from:
- Public Trivy documentation — the published descriptions of the Trivy DB (a BoltDB file containing vulnerability information aggregated from feeds such as NVD, Red Hat, and Debian) and the self-hosting / air-gap guidance.
- The documented BoltDB (bbolt) file format — a single-file, memory-mapped B+tree key/value store with a publicly documented page layout (meta pages, freelist page, branch/leaf pages, bucket headers).
- Black-box round-trip inspection — writing a DB with our own writer and parsing it back with an independently written reader (
__Tests/.../Bolt/TrivyDbReader.cs) that does not share code with the writer under test, confirming the observable structure parses and a known advisory round-trips.
The following were NOT used and NOT copied: Aqua trivy-db builder source code, comments, documentation prose, table-generation logic, or non-essential naming. Only the observable interface/format (bucket names, key shapes, JSON field names, the integer severity scale) was reproduced — these are functional facts required for a Trivy reader to interpret the file, not protected expression.
BoltDB file layout produced
(All multi-byte integers little-endian; page size 4096.)
| Page(s) | Contents |
|---|---|
| 0, 1 | Meta pages (magic=0xED0CDAED, version=2, pageSize, root bucket pgid, freelist pgid, high-water pgid, txid, FNV-1a checksum over the meta payload). |
| 2 | Freelist page (empty — the writer never frees pages). |
| 3… | Branch/leaf pages, allocated bottom-up so child bucket page chains exist before parents reference them. |
- Leaf page: header + N
leafPageElement{flags, pos, ksize, vsize}+ packed key/value bytes. Sub-buckets are leaf elements flaggedbucketLeafFlagwhose value is a 16-byte bucket header (root pgid+sequence) pointing at the child’s page chain (non-inline; the bbolt reader treats this identically to an inline bucket when traversing). - Branch page: header + N
branchPageElement{pos, ksize, pgid}+ first-key bytes; emitted only when a bucket exceeds one page. - Keys within a bucket are byte-ordinal sorted, matching bbolt’s on-disk order, which makes the output deterministic and reader-traversable.
Implementation: src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/Bolt/ (BoltDbFormat, Fnv64a, BoltBucket, BoltDbWriter).
Trivy-compatible schema (bucket map)
| Bucket path | Key | Value |
|---|---|---|
trivy | metadata | TrivyMetadata JSON (Version, NextUpdate, UpdatedAt, DownloadedAt). |
vulnerability | <vulnID> (CVE id preferred) | Vulnerability detail JSON (Title, Description, Severity, References, CVSS, dates). |
<data-source> → <package> | <vulnID> | Advisory detail JSON (FixedVersion, VulnerableVersions, Severity, DataSource). |
<data-source>is the OS platform ("<distro> <release>") for OS packages, otherwise the source id (e.g.ghsa).- Severity uses the Trivy integer scale: UNKNOWN=0, LOW=1, MEDIUM=2, HIGH=3, CRITICAL=4.
Implementation: TrivyDbSchema.cs.
Source-license allowlist + provenance
Each canonical record carries its source via Concelier provenance. The SourceLicenseAllowlist resolves each source to a disposition (see counsel section 4):
- Include (CC0 / MIT / BSD / Apache / CC-BY / public-domain): NVD, CVE/MITRE, GHSA, OSV, Red Hat, Alpine, Wolfi, Chainguard, Debian, Photon, Oracle.
- Segregate (CC-BY-SA share-alike): Ubuntu. Excluded from the main DB and routed to a separate
trivy-by-sa.dbshard so the share-alike obligations are never folded into the BUSL-restricted artifact. - Facts-only (no clear license / restrictive vendor terms — the default for unknown sources): only uncopyrightable facts (CVE id, package, version, fixed version, source URL) are emitted.
- Exclude: dropped entirely.
Each emitted record embeds a DataSource object (ID, Name, URL, License) for downstream provenance/audit.
Data artifact, not BUSL
The produced DB is a data artifact carrying the composite third-party data notice in third-party-licenses/DATA-NOTICES.md. It is not licensed under Stella Ops’ BUSL-1.1 software license. CC-BY / CC-BY-SA content must not be placed under additional downstream restrictions.
Verification
TrivyDbBoltBuilderRoundTripTests (black-box): build trivy.db, parse it with the independent TrivyDbReader, and assert (a) the structure parses as a valid BoltDB with the expected buckets, (b) a known advisory round-trips (write → read → same FixedVersion/Severity/DataSource), © Ubuntu CC-BY-SA is absent from the main DB and present in the BY-SA shard, and (d) reruns are byte-identical (deterministic).