Scanner Rust Analyzer Benchmarks (Sprint 130)
Summary
- New fixtures under
src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/rust/capture heuristic and fallback behaviours. RustLanguageAnalyzerBenchmark(seesrc/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks) exercises the new fixtures and emits heuristic coverage metrics.- March 2025 snapshot: heuristic detection covers 3 crates (
reqwest,serde,tokio) vs competitor baseline (1 crate), yielding 300% relative coverage.
Running the benchmarks
# Build once in Release for accurate metrics
DOTNET_CLI_UI_LANGUAGE=en dotnet build src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj -c Release
# Dump heuristic output
DOTNET_CLI_UI_LANGUAGE=en dotnet run --no-build --project src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj -- --dump-heuristics
# Dump fallback binary output (bin provenance)
DOTNET_CLI_UI_LANGUAGE=en dotnet run --no-build --project src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj -- --dump-fallback
# Run benchmarks (optional)
DOTNET_CLI_UI_LANGUAGE=en dotnet run --project src/Scanner/__Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks/StellaOps.Scanner.Analyzers.Lang.Rust.Benchmarks.csproj -c Release
BenchmarkDotNet emits warnings when running against Debug builds. Use
-c Releaseor setconfig.WithOptions(ConfigOptions.DisableOptimizationsValidator)if you need debugging traces.
Heuristic coverage comparison
| Fixture | Our detector | Competitor baseline | Relative gain |
|---|---|---|---|
lang/rust/heuristics | reqwest, serde, tokio | serde | +300% (3/1) |
Competitor baseline data lives in competitor-baseline.json alongside the heuristics fixture and is used by RustHeuristicCoverageComparisonTests.
Fallback binary snapshot
componentKey: bin::sha256:10f3c03766e4403be40add0467a2b2d07fd7006e4b8515ab88740ffa327ea775
metadata:
binary.path -> usr/local/bin/opaque_bin
binary.sha256 -> 10f3c03766e4403be40add0467a2b2d07fd7006e4b8515ab88740ffa327ea775
provenance -> binary
evidence:
source=binary locator=usr/local/bin/opaque_bin sha256=10f3c03766e4403be40add0467a2b2d07fd7006e4b8515ab88740ffa327ea775
This snapshot is persisted in expected.json for repeatable regression tests.
