Scanner Rust Analyzer Benchmarks (Sprint 130)
Summary
- New fixtures under
src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/rust/capture heuristic and fallback behaviours. RustLanguageAnalyzerBenchmark(seesrc/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks) exercises the new fixtures and emits heuristic coverage metrics.- March 2025 snapshot: heuristic detection covers 3 crates (
reqwest,serde,tokio) vs competitor baseline (1 crate), yielding 300% relative coverage.
Running the benchmarks
# Build once in Release for accurate metrics
DOTNET_CLI_UI_LANGUAGE=en dotnet build src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj -c Release
# Dump heuristic output
DOTNET_CLI_UI_LANGUAGE=en dotnet run --no-build --project src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj -- --dump-heuristics
# Dump fallback binary output (bin provenance)
DOTNET_CLI_UI_LANGUAGE=en dotnet run --no-build --project src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj -- --dump-fallback
# Run benchmarks (optional)
DOTNET_CLI_UI_LANGUAGE=en dotnet run --project src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj -c Release
BenchmarkDotNet emits warnings when running against Debug builds. Use
-c Releaseor setconfig.WithOptions(ConfigOptions.DisableOptimizationsValidator)if you need debugging traces.
Heuristic coverage comparison
| Fixture | Our detector | Competitor baseline | Relative gain |
|---|---|---|---|
lang/rust/heuristics | reqwest, serde, tokio | serde | +300% (3/1) |
Competitor baseline data lives in competitor-baseline.json alongside the heuristics fixture and is used by RustHeuristicCoverageComparisonTests.
Fallback binary snapshot
The opaque fixture is a stripped Rust binary: it carries no crate symbols, only the rustc standard-library source path (/rustc/<40 hex>/library/...). Since PCD-816 an ELF file without Rust evidence (a well-formed Rust legacy-mangled symbol or that path) is not reported at all, so the fallback applies only to Rust binaries whose crates cannot be recovered.
componentKey: bin::sha256:f2af833346fdb3e282786c37ee4592233624d9d355a49a687d2b9950c1b86935
metadata:
binary.path -> usr/local/bin/opaque_bin
binary.sha256 -> f2af833346fdb3e282786c37ee4592233624d9d355a49a687d2b9950c1b86935
provenance -> binary
evidence:
source=binary locator=usr/local/bin/opaque_bin sha256=f2af833346fdb3e282786c37ee4592233624d9d355a49a687d2b9950c1b86935
This snapshot is persisted in expected.json for repeatable regression tests.
