Developer Tools Architecture

Audience: Stella Ops developers and operators working in local and CI workflows. For the module overview and quick links, see ./README.md.

Standalone CLI utilities, benchmarks, SDK generators, and the developer portal that support building, testing, and shipping Stella Ops.

Overview

The Tools directory (src/Tools/) contains a set of independent utilities. Most are .NET CLI applications with their own Program.cs entry point (several Policy tools are thin wrappers whose logic lives in the shared StellaOps.Policy.Tools library); WorkflowGenerator is a C# library consumed by other tools/tests rather than a standalone exe, and several Bench harnesses are Python scripts. These tools are not deployed as services – they are invoked locally by developers or executed in CI pipelines. Each tool is narrowly scoped to a single responsibility, from fixture management to workflow generation.

Components

src/Tools/
  FixtureUpdater/           # Deterministic Concelier OSV/GHSA/NVD fixture rewriter
    Program.cs
  GoldenPairs/              # SBOM/advisory corpus management (StellaOps.Tools.GoldenPairs)
    Program.cs
  PolicyDslValidator/       # Policy DSL file validation (thin wrapper over StellaOps.Policy.Tools)
    Program.cs
  PolicySchemaExporter/     # JSON schema export for IDE support (wrapper over StellaOps.Policy.Tools)
    Program.cs
  PolicySimulationSmoke/    # Policy simulation smoke tests (wrapper over StellaOps.Policy.Tools)
    Program.cs
  LanguageAnalyzerSmoke/    # Language analyzer golden-snapshot smoke tests (python, rust)
    Program.cs
  NotifySmokeCheck/         # Notify pipeline smoke check (Valkey/Redis via StackExchange.Redis)
    Program.cs
  RustFsMigrator/           # RustFS data migration between schema versions
    Program.cs
  DebianTrackerImport/      # Debian security-tracker JSON -> LIVE Concelier vuln store bulk importer
    Program.cs              #   (the only tool that writes to a production datastore)
  StellaOps.OpenApi.DocGen/ # OpenAPI spec -> API-reference markdown generator
    Program.cs
  StellaOps.Tools.WorkflowGenerator/  # CI workflow generation library (C# + YamlDotNet)
    GitHubActionsGenerator.cs  GitLabCiGenerator.cs  AzureDevOpsGenerator.cs
  StellaOps.Bench/          # Performance benchmarks (absorbed from src/Bench/)
    LinkNotMerge/            #   Link-not-merge linkset benchmarks (.NET)
    LinkNotMerge.Vex/        #   VEX-enriched linkset benchmarks (.NET)
    Notify/                  #   Notification pipeline benchmarks (.NET)
    PolicyEngine/            #   Policy engine evaluation benchmarks (.NET)
    Scanner.Analyzers/       #   Language analyzer benchmarks (.NET)
    ImpactIndex/             #   Impact-index benchmark (Python)
    PolicyCache/             #   Policy cache benchmark (Python)
    PolicyDelta/             #   Policy delta benchmark (Python)
    Signals/                 #   Reachability signals benchmark (Python)
    Determinism/             #   Determinism fixture inputs/configs (Python)
    Graph/                   #   Graph benchmark scenarios and results (Python + UI driver)
  Bench/                    # Developer-only C# benchmark quality/compute layer for stella-vs-trivy
    StellaOps.Bench.Oracle/  #   Rule-based benchmark oracle and scorer
    StellaOps.Bench.Oracle.Cli/       #   Executable entry point for the oracle
    StellaOps.Bench.Compute/ #   Canonical JSON, structural JSON equivalence, golden fixture helpers
    StellaOps.Bench.Compute.Cli/      #   Executable entry point — exposes the `stella-bench-compute run` verb
    StellaOps.Bench.Compute.Orchestration/ # Docker/Git process wrappers for benchmark orchestration
    StellaOps.Bench.{Compute,Oracle}.Tests/ # Per-project test suites
  StellaOps.Verifier/       # Standalone offline bundle verifier (absorbed from src/Verifier/)
    __Tests/
  StellaOps.Tools.TlptScoper/ # Deterministic DORA TLPT scope generator
    fixtures/
    __Tests/
  StellaOps.Sdk.Generator/  # Multi-language SDK code generation (absorbed from src/Sdk/)
    go/  java/  python/  ts/
    postprocess/
  StellaOps.Sdk.Release/    # SDK release guild charter (AGENTS.md only; pipeline not yet implemented in-tree)
  StellaOps.DevPortal.Site/  # Developer portal static site (absorbed from src/DevPortal/)
    src/  public/  scripts/  snippets/
  __Tests/                  # Per-tool xUnit test projects (FixtureUpdater, GoldenPairs, etc.)
  plugins/concelier/        # Vendored Concelier connector DLLs used by FixtureUpdater fixtures
  openssl/                  # Vendored OpenSSL binaries (linux-x64)
  certbund_offline_snapshot.py  # Offline CERT-Bund advisory snapshot helper
src/__Tests/supply-chain/   # Deterministic supply-chain hardening lanes
  01-jcs-property/
  02-schema-fuzz/
  03-rekor-neg/
  04-big-dsse-referrers/
  05-corpus/

Tool Descriptions

FixtureUpdater

Rewrites the Concelier OSV/GHSA/NVD golden fixtures deterministically (the RootCommand description is “Rewrites Concelier OSV/GHSA/NVD fixtures deterministically.”). It is fully offline: it references the Concelier connector libraries (StellaOps.Concelier.Connector.{Osv,Ghsa,Nvd,Common}) and StellaOps.Concelier.Testing directly and rewrites the raw/snapshot fixture files in-process – it does not call any live service. A --fixed-time option (FixtureUpdaterDefaults.DefaultFixedTime) drives deterministic timestamps. CLI options: --repo-root, --osv-fixtures, --ghsa-fixtures, --nvd-fixtures, --fixed-time.

GoldenPairs

Project StellaOps.Tools.GoldenPairs. Manages the golden-pairs SBOM/advisory test corpus under src/__Tests/__Datasets/golden-pairs. Exposes three subcommands: mirror, diff, and validate (BuildMirrorCommand/BuildDiffCommand/BuildValidateCommand), with --repo-root and --dataset-root options. The diff command surfaces changes to upstream advisory formats so they are detected and accommodated.

PolicyDslValidator

Thin entry point (Program.cs -> StellaOps.Policy.Tools.PolicyDslValidatorApp) that validates policy DSL files against the current schema. Used in CI gates to catch policy syntax errors before merge. The validation logic itself lives in the shared StellaOps.Policy.Tools library.

PolicySchemaExporter

Thin entry point (Program.cs -> StellaOps.Policy.Tools.PolicySchemaExporterApp). Exports the Policy DSL schema to JSON format for documentation and IDE autocomplete support. Enables policy authors to get inline validation and completion in their editors.

PolicySimulationSmoke

Thin entry point (Program.cs -> StellaOps.Policy.Tools.PolicySimulationSmokeApp). Runs policy simulation smoke tests in-process against the StellaOps.Policy evaluation library – it does not connect to a running Policy Engine service. It enumerates frozen scenario.json fixtures under a scenario root (default docs/samples/policy/simulations), evaluates each, and validates that policy evaluation produces the expected verdicts. Uses a fixed time (DateTimeOffset.UnixEpoch by default) for determinism.

LanguageAnalyzerSmoke

Exercises the Scanner language analyzers (StellaOps.Scanner.Analyzers.Lang*) in-process against curated fixtures and compares each scenario’s serialized output to a repository golden snapshot (expected.json) for byte-equality. It does not compute precision/recall metrics. Supported analyzers are selected via --analyzer (python, rust); a --allow-golden-drift flag downgrades snapshot mismatch from a failure to a warning, and --fixed-time (default 2025-01-01 UTC) drives deterministic analyzer output.

NotifySmokeCheck

Smoke check for the Notify delivery pipeline. Connects to a Valkey/Redis instance (StackExchange.Redis) and exercises notification publish/consume behaviour end-to-end. Used to validate that the messaging transport is wired correctly in a deployed environment.

RustFsMigrator

Migrates data stored in RustFS (S3-compatible object storage) between schema versions via AWSSDK.S3. Handles object key transformations and metadata updates required during platform upgrades.

DebianTrackerImport (StellaOps.Tools.DebianTrackerImport)

The only tool that mutates a production datastore. Added to this dossier 2026-07-12 (it shipped undocumented). Source: src/Tools/DebianTrackerImport/Program.cs.

A one-off operational/benchmark bulk importer that drains the full Debian security-tracker JSON corpus into the live Concelier vuln store. It writes through Concelier’s production canonical write pathDebianTrackerJsonParserDebianMapperIAdvisoryStore.UpsertAsync (PostgresAdvisoryStoreAdvisoryRepository) — producing vuln.advisories, vuln.advisory_affected (ecosystem='deb') and alias rows. Source attribution resolves to the existing debian source via the distro-debiandebian SourceKeyAlias.

StellaOps.Tools.DebianTrackerImport <path-to-tracker.json> [options]
  --connection "<npgsql conn string>"    (or env DEBIMPORT_CONNECTION)
  --limit <N>                            cap advisories upserted (0 = all)
  --releases bookworm,bullseye,trixie    restrict to release codenames
  --parallel <N>                         concurrent upsert workers (default 8)
  --dry-run                              parse + map only, no DB writes

Safety properties (verified, not assumed): it does not fabricate data — every row is mapped from the tracker JSON the operator supplied; the parser is offline unit-tested (DebianTrackerJsonParserTests); and --dry-run gates all DB writes. Operator caution: it takes a raw Npgsql connection string and writes at --parallel 8 by default into whatever database that string points at — it has no tenant guard of its own and no dry-run default. Point it at the intended store, and prefer --dry-run first.

StellaOps.OpenApi.DocGen

Generates an API-reference markdown document from an OpenAPI spec (JSON or YAML, via YamlDotNet). Invoked as StellaOps.OpenApi.DocGen --input <openapi.json|yaml> --output <api-reference.md> [--title ...] [--source-label ...]. Output is deterministic (UTF-8 without BOM); the tool reports the number of operations rendered and returns non-zero on usage/IO errors. Used to keep human-readable API docs in sync with the generated spec.

WorkflowGenerator (StellaOps.Tools.WorkflowGenerator)

A C# library (not F#) that generates CI/CD workflow definitions, using YamlDotNet for YAML emission. It exposes an IWorkflowGenerator abstraction and a WorkflowGeneratorFactory, with concrete generators for four platforms (CiPlatform enum): GitHub Actions, GitLab CI, Azure DevOps, and Gitea Actions. The library has no Program entry point of its own; it is consumed by other tools/tests. It ensures CI workflow files are consistent, auditable, and derived from a single source of truth rather than hand-edited YAML.

WorkflowGenerator output is offline-first and pinned/preinstalled by default:

StellaOps.Bench (Performance Benchmarks)

Collected from the former src/Bench/ directory. Contains performance benchmarks for multiple subsystems, a mix of .NET console harnesses and Python harnesses:

The .NET benchmark subprojects are plain Exe console projects (not packaged as dotnet tool); the Python harnesses are run directly via their *_bench.py scripts / shell wrappers.

Bench (stella-vs-trivy compute rewrite)

The newer src/Tools/Bench/ area contains developer-only C# libraries and tests used by the staged stella-vs-trivy benchmark rewrite. StellaOps.Bench.Compute provides deterministic canonical JSON serialization, structural JSON equivalence, hermetic golden fixture loading for byte-equivalence gates over the frozen benchmark corpus, the DiscoveryTruth lockfile/manifest discovery oracle for the stellaops.bench.discovery-truth/v1 schema, DiscoveryHarcaseProbe for the BP-11 stellaops.bench.hardcase-result/v1 manifest-only hardcase discovery result, the EvidenceScorer for the existing stellaops.bench.evidence/v2 evidence-richness contract, CompareMetrics for the stellaops.bench.compare/v5 metric document, ReportBuilder for the aggregate stellaops.bench.report/v5 JSON/Markdown report, and CapsuleBuilder for the BP-13 stellaops.bench.capsule/v1 audit capsule. Discovery truth recursively walks a supplied root, prunes installed/vendor/build directories, parses ecosystem lockfiles without extra NuGet dependencies, tolerates malformed inputs by skipping them, and emits sorted expected components, counts, and source paths. DiscoveryHarcaseProbe reads a BP-11 expected.json and a compare JSON, checks the per-tool SBOM PURL arrays for the expected package name and version, and reports which tools found the hardcase PURL. EvidenceScorer scores comparable evidence fields across Stella, Trivy, and Grype from StellaOps.Scanner.Benchmark.Harness.NormalizedFinding lists, carries tool-specific evidence facts through the existing metadata bag, and reports Stella-only match/reachability richness separately so competitors are not penalized for fields they cannot emit. CompareMetrics.Build consumes the already-produced benchmark findings, CycloneDX/Syft SBOMs, optional discovery truth, perf, evidence, determinism, interop, replay, KEV, VEX, remediation, and reachability inputs to emit the compare/v5 shape without jq; bug-prone jq helpers such as PURL bucketing, discovery recall, Jaccard, semver upgrade buckets, VEX/breaking-risk normalization, KEV aggregates, and reachability passthrough are now typed and unit-tested. ReportBuilder preserves the bash report totals, per-cohort rollups, reportByType, oracle recall block, and Markdown projection over the four-project golden corpus. CapsuleBuilder lifts replay rows from the aggregate report and computes capsuleDigest from canonical JSON with top-level generatedAt excluded from hashed bytes; when BENCH_CAPSULE_SIGNING_KEY is present it emits an hmac-sha256 capsule signature, otherwise unsigned. StellaOps.Bench.Compute.Orchestration is the Phase 2 foundation for the same benchmark rewrite: it wraps the Docker and Git CLIs through System.Diagnostics.Process only (no Docker.DotNet), captures stdout/stderr/exit codes, applies bounded retry and timeouts, translates Windows drive-letter host paths such as C:\repo to Docker mount paths such as /c/repo, forces MSYS path-conversion guards for Docker CLI calls, extracts exported rootfs tarballs with symlink-tolerant Windows behavior, and treats osv-scanner exit code 1 as a successful scan-with-vulnerabilities result. These Bench projects are not added to release solution files or packaging artifacts.

ProjectPreparerService is the C# prepare-phase orchestrator for the same developer-only lane. It uses the existing process-backed GitClient and DockerClient to materialize cached/sparse source checkouts, select project/common/repo/crafted Dockerfiles, run docker build, docker save, and rootfs export/extract, invoke stella sbom generate --directory, run the C# DiscoveryTruth, and emit stellaops.bench.prepare/v2. The service keeps the bash prepare contract’s fail-soft behavior for Git/Docker/Stella failures, CRLF-tolerant project.env loading, ecosystem source_include_patterns, symlink-tolerant tar extraction, repo=local-fixture metadata-only fixture materialization, and the BENCH_KEEP_SOURCE_FOR_REACH=1 retained reach-src snapshot path. Its generated benchmark .dockerignore keeps broad dependency-cache pruning while re-including vendor/node_modules/**/package.json so manifest-only BP-11 npm fixtures remain buildable without committing package source. The Docker export helper creates containers with a placeholder true command, allowing rootfs export for scratch images that intentionally have no default command.

BenchRunOrchestrator (P2-T4/T5) is the full C# pipeline orchestrator: it selects projects from projects.json, resolves enabled scanner tools to immutable digests with prewarmed caches before compare, runs bounded-parallel prepare + compare + oracle, aggregates report.json + capsule.json via the C# ReportBuilder/CapsuleBuilder, and writes results-current.csv. The stella-bench-compute run verb exposes the orchestrator. Two shell-level switches control how the pipeline is invoked:

When neither env var is set, the full bash path (run-benchmark.sh + compare-project.sh + prepare-project.sh + inline jq/python) runs unchanged — this is always the fallback. Phase 3 decommission (removal of the bash/python compute layer) is a future step tracked separately after a full-cohort parity run.

Parity is proven by xUnit tests in StellaOps.Bench.Compute.Tests (BenchRunOrchestratorTests.CsharpRunReport_MatchesBashGoldenReport_*): for golang/caddy and javascript/ghost, the tests seed a temp run-dir with the frozen golden compare.json + prepare.json, run the C# orchestrator with --skip-prepare --skip-compare (report aggregation only), build a single-project expected report from those committed bash fixtures, and assert the produced report.json with JsonEquivalence.AssertEquivalentOnExpectedKeys.

StellaOps.Verifier (Standalone Bundle Verifier)

Absorbed from the former src/Verifier/ directory. A standalone CLI tool for offline verification of Stella Ops evidence bundles in air-gapped environments. Publishes as a single-file, self-contained executable (stella-verifier) targeting multiple RIDs: win-x64, linux-x64, linux-musl-x64, osx-x64, osx-arm64. Has no framework dependencies – designed for minimal footprint in restricted environments.

StellaOps.Tools.TlptScoper (DORA TLPT Scope Generator)

Generates canonical tlpt-scope.v1 JSON for DORA TLPT white-team scoping. The local implementation reads an Asset Registry v1 JSON input, filters active tenant assets by criticality threshold, references a ReachGraph subgraph, and records criticality coverage warnings. The live implementation uses Graph’s public Asset Registry client contract to query active tenant assets without a local --asset-registry file, then applies the same deterministic mapping, filtering, and hashing rules.

StellaOps.Sdk.Generator (SDK Code Generation)

Absorbed from the former src/Sdk/StellaOps.Sdk.Generator/ directory. Generates typed SDK clients for Go, Java, Python, and TypeScript from the Stella Ops OpenAPI specification. Includes per-language config.yaml, generation scripts, postprocessing templates, and deterministic toolchain lockfile (toolchain.lock.yaml).

StellaOps.Sdk.Release (SDK Release Automation)

Absorbed from the former src/Sdk/StellaOps.Sdk.Release/ directory. Intended to automate the release pipeline (packaging, signing, publishing, changelog) for generated SDK packages and dev-portal bundles. Status: charter only – the in-tree directory currently contains just the guild charter (AGENTS.md); the release-automation implementation does not yet live here.

StellaOps.DevPortal.Site (Developer Portal)

Absorbed from the former src/DevPortal/ directory. An Astro-based static site providing interactive API documentation, SDK quickstarts, code examples, and a “Try It” console backed by RapiDoc. Includes offline build support (build-offline.mjs), link checking, accessibility audits, and performance checks. Not a .NET project – uses Node.js/npm.

Supply-Chain Hardening Suite

Deterministic, offline-safe multi-lane harness (src/__Tests/supply-chain/) validating canonicalization invariants (01-jcs-property), mutation-fuzz crash resistance (02-schema-fuzz), Rekor negative-path diagnostics (03-rekor-neg), oversized DSSE/referrer reject behavior (04-big-dsse-referrers), and a fixture corpus/manifest builder (05-corpus). Driven by run.sh / run.ps1 / run_suite.py with smoke (PR gate) and nightly profiles; each lane emits junit.xml, report.json/summary.json, and replay-ready diagnostics under out/supply-chain/.

Data Flow

Tools are consumers and producers of artifacts:

  1. FixtureUpdater reads local raw Concelier inputs and rewrites OSV/GHSA/NVD golden fixture files to the repository deterministically (in-process, no live service). GoldenPairs mirrors/diffs/validates the golden-pairs corpus under src/__Tests/__Datasets/golden-pairs.
  2. PolicyDslValidator and PolicySchemaExporter read policy definitions and produce validation results or schema files (logic in StellaOps.Policy.Tools).
  3. PolicySimulationSmoke and LanguageAnalyzerSmoke execute tests in-process against the StellaOps.Policy and StellaOps.Scanner.Analyzers.Lang* libraries over frozen fixtures, comparing output to expected verdicts / golden snapshots and producing pass/fail reports.
  4. NotifySmokeCheck connects to a Valkey/Redis instance and exercises the Notify publish/consume pipeline, producing a pass/fail result.
  5. RustFsMigrator reads from and writes to S3-compatible storage (RustFS) via the AWS S3 SDK.
  6. StellaOps.OpenApi.DocGen reads an OpenAPI spec (JSON/YAML) and writes a deterministic API-reference markdown file.
  7. WorkflowGenerator takes a typed WorkflowOptions (C# record) and emits offline-safe CI workflow YAML for GitHub Actions, GitLab CI, Azure DevOps, or Gitea Actions.
  8. Supply-chain hardening suite reads frozen corpus fixtures, executes deterministic lanes, and emits replay-ready artifacts for CI and local triage.
  9. StellaOps.Bench reads frozen inputs (SBOMs, VEX, graphs) and produces benchmark reports with latency/throughput metrics (mix of .NET and Python harnesses).
  10. StellaOps.Verifier reads evidence bundles from disk and produces pass/fail verification results for offline audit.
  11. StellaOps.Tools.TlptScoper reads local Asset Registry v1 or live Graph Asset Registry inputs plus ReachGraph references and produces deterministic TLPT scope JSON.
  12. StellaOps.Sdk.Generator reads the OpenAPI spec and produces typed SDK source code for Go, Java, Python, and TypeScript.
  13. StellaOps.DevPortal.Site reads the OpenAPI spec and SDK documentation, producing a static developer portal site.

Database Schema

No tool owns a schema — none of them define migrations, and the §2.7 auto-migration invariant does not apply to this module.

One tool does write to a live database: DebianTrackerImport bulk-writes vuln.advisories + vuln.advisory_affected + aliases into the Concelier vuln store through Concelier’s own canonical write path (IAdvisoryStore.UpsertAsync). It uses Concelier’s schema; it does not own one. See its entry under Tool Descriptions. (Corrected 2026-07-12: this section previously read “Not applicable … no persistent database”, which was false for that tool.)

Endpoints

Not applicable. Tools are client-side CLI applications with no HTTP endpoints.

Dependencies

Library/ToolPurpose
StellaOps.Policy.ToolsShared library backing PolicyDslValidator, PolicySchemaExporter, PolicySimulationSmoke
StellaOps.Concelier.Connector.{Osv,Ghsa,Nvd,Common} + TestingFixtureUpdater fixture rewriting
StellaOps.Scanner.Analyzers.Lang*LanguageAnalyzerSmoke analyzer execution
YamlDotNetWorkflowGenerator YAML emission, OpenApi.DocGen spec parsing
StackExchange.RedisNotifySmokeCheck Valkey/Redis transport
AWSSDK.S3RustFsMigrator object storage access
StellaOps.Graph.ApiTlptScoper live Asset Registry client contract
StellaOps.PolicyBench.PolicyEngine benchmark target
StellaOps.Scanner.BenchmarkShared normalized finding adapters and benchmark metric primitives used by developer-only Bench compute/oracle projects
Docker CLI + Git CLIExternal process tools used by developer-only Bench orchestration wrappers
StellaOps.Scanner.Analyzers.Lang.*Bench.ScannerAnalyzers benchmark targets
StellaOps.Notify.ModelsBench.Notify benchmark target
StellaOps.TestKitShared test infrastructure for benchmark tests
System.CommandLineVerifier, TLPT Scoper, FixtureUpdater, GoldenPairs, LanguageAnalyzerSmoke CLI parsing
Astro + Starlight + RapiDocDevPortal static site generation and “Try It” console
openapi-generator-cliSdk.Generator multi-language code generation

Security Considerations