ADR — Findings Runtime Score Formula

Formerly docs/architecture/ADR-FINDINGS-RUNTIME-SCORE-FORMULA.md.

In one line: Stella Ops reduces a finding’s runtime evidence to one comparable 0–100 runtime score — a deterministic, explainable weighted sum of four normalized components (hits 0.40, entry-point 0.30, coverage 0.20, recency 0.10) — so operators can rank otherwise-equivalent findings by what is actually being exercised in production.

Audience: Findings Ledger engineers implementing or tuning the score, and operators interpreting it. The four component values are surfaced for explainability; the weights are config-tunable without redeploying the library.

Context

Findings Ledger persists runtime instrumentation traces (eBPF / syscall / APM probes) correlated to findings. Operators need a single comparable number — the runtime score — to answer the question “which of these otherwise-equivalent findings is actually being exercised in production?”

Wire shape (see Models/RuntimeModels.cs):

RuntimeScore {
    string TenantId;
    Guid   FindingId;
    double Score;             // 0..100
    DateTimeOffset ComputedAt;
    TimeSpan Window;
    IReadOnlyDictionary<string, double> Components;  // hits / entryPoint / coverage / recency
}

The persisted domain score is 0..100. The HTTP runtime-score endpoint adapts that value to a normalized 0..1 wire value for the existing UI contract; clients that render a gauge multiply by 100.

Constraints we are designing under:

• The score must be explainable — operators must see why a finding scored what it did. We expose the four pre-weight component values in RuntimeScore.Components.
• The score must be deterministic — same trace stream + same now → same score, byte-for-byte. No floating-point non-determinism, no wall-clock dependencies inside the calculator (the caller supplies now).
• Weights must be tunable post-deployment — we expect to recalibrate after we have a few months of operational data. RuntimeScoreCalculatorOptions is the tuning surface.
• The implementation is a pure library — no DI registration, no Postgres, no I/O. It must be reusable in both the live ingest pipeline and offline backfill jobs.

Decision

The runtime score is a four-component weighted sum, each component normalized to [0, 1], then clamped and scaled to [0, 100]:

score = clamp01(
      W_hits     * normalizedHits(aggregates)
    + W_entry    * entryPointPresence(aggregates)
    + W_cover    * vulnerableFunctionCoverage(recentTraces)
    + W_recency  * recencyWeight(aggregates, window, now)
) * 100

Component definitions

Where:

• S is HitCountSaturation, defaulting to 100. This means a finding observed 100 times within the window contributes a maxed-out hits component; further hits saturate. The log-scale captures the intuition that “10 hits is much more than 1, but 1000 is not much more than 100.”
• “distinct functions” in the coverage component is computed over the trace stream after applying RuntimeSymbolRedactor.RedactSymbol, so per-process address differences do not inflate the denominator.
• lastSeen for the recency component is max(aggregate.LastSeen) — the most recent observation across all aggregates.
• entryPointPresence is read from persisted aggregate has_entry_point after Sprint 023. Existing aggregate rows created before that corrective migration are backfilled from retained runtime_traces.frames where possible; rows without recoverable source traces remain false until rebuilt or updated by new runtime ingests.
• Default weights (0.40, 0.30, 0.20, 0.10) sum to exactly 1.0. RuntimeScoreCalculator validates this at construction time.

Boundary behavior (all asserted by unit tests)

• Empty input (no aggregates, no traces) → score = 0, all components = 0.
• All four components maxed out → score = 100.
• Hit count past saturation → hits component clamps at 1.0, score still ≤ 100.
• lastSeen >= now (clock skew, future timestamp) → recency = 1.0 (treat as fully fresh — caller is responsible for clock sanity).
• lastSeen <= now - window → recency = 0 (outside the window, do not contribute negative recency).

Rationale

Why log-scale hits? Linear scaling makes a 10× more popular finding look 10× more important, which over-weights the most-trafficked path at the expense of less-trafficked but reachable code. Log scaling keeps the score sensitive to “any traffic at all” while flattening at high volume.

Why binary entry-point? Entry-point presence is qualitative — either the vulnerability is reachable from a user-visible surface or it isn’t. Counting the number of entry points would reward apps with bigger surface areas, which is the wrong signal.

Why ratio-based coverage? A vulnerability that’s the only function the trace stream sees is more concerning than one that’s one of fifty functions exercised. The ratio captures “is this the hot path or peripheral noise?”

Why linear recency decay? It maps cleanly to the operational “sliding window of N hours” mental model and is easy to explain. We considered exponential decay; linear was chosen for explainability over mathematical elegance — operators reading “this trace is 12 hours into a 24-hour window, so recency = 0.5” is more useful than reading “recency = e^(-12/τ)”.

Why these specific weights (40/30/20/10)? They reflect the prioritization the product team validated:

1. Hits dominate because actual exercise of the code is the strongest signal we have. 0.40 is the largest weight.
2. Entry-point next because reachability from user-facing surface converts a theoretical risk to a real one.
3. Coverage is a secondary signal: even a hit-heavy / reachable vulnerability that’s drowning in unrelated traffic is lower priority than one that is the trace stream.
4. Recency is the smallest component because once a finding has been exercised at all, its severity does not really decay — we just slightly down-weight stale evidence to favor still-active code paths.

Weights are configurable via RuntimeScoreCalculatorOptions so that post-launch tuning does not require a re-deploy of the runtime library.

Alternatives considered

• Single sigmoid over total hits. Simpler to compute but loses the reachability and coverage signals. Rejected: not explainable.
• Multiplicative model (score = hits * entry * coverage * recency). Any zero collapses the score to zero, which is too brittle — a high-traffic finding without a labeled entry-point would score 0, which is wrong.
• Bayesian network / learned weights. Out of scope for v1; the four hand-tuned weights are easier to audit and the RuntimeScoreCalculatorOptions shape leaves room for a learned-weights feeder later.

Consequences

• The algorithm is online-friendly: aggregates and recent traces are the only inputs; both are cheap to compute on a sliding window.
• The algorithm is cache-friendly: with now as an explicit parameter, callers can time-bucket scores and re-use them across requests.
• Recalibration is operationally cheap — change the weights in config, no schema migration, no recompute of historical traces.

Test coverage

22 unit tests in src/Findings/__Tests/StellaOps.Findings.Ledger.Runtime.Tests/Scoring/RuntimeScoreCalculatorTests.cs, covering:

• Empty input / score=0
• All-maxed input / score=100
• Saturation clamping (hits, score ≤ 100, score ≥ 0)
• Per-component log-scaling, ratios, recency curve
• Boundary timestamps (now == lastSeen, lastSeen <= now - window, half-window)
• Weight validation (must sum to 1.0)
• Saturation validation (must be > 0)
• Determinism (same input → same score across invocations)

References

• ADR-013 Findings Runtime Disabled Mode
• ADR-014 Findings Runtime Persistence — runtime_scores is where this score is stored.
• ADR-015 Findings Runtime Privacy Filter — redaction feeds the coverage denominator.
• ADR-017 Findings Runtime Score Recompute Async-Migration Trigger — when score recompute moves off the ingest hot path.
• docs/modules/findings-ledger/runtime-instrumentation-api.md — runtime-score endpoint contract.
• src/Findings/__Libraries/StellaOps.Findings.Ledger.Runtime/RuntimeScoreCalculator.cs
• src/Findings/__Tests/StellaOps.Findings.Ledger.Runtime.Tests/Scoring/RuntimeScoreCalculatorTests.cs

Architecture-lead sign-off

• Author (Implementer): 2026-04-29.
• Architecture lead: Accepted 2026-04-29 as part of the ADR-FINDINGS-RUNTIME-* set (see the Sign-off Window below for how acceptance was reached). The formula is implemented as described above; weights are config-tunable post-review.
• Reviewed and re-affirmed during SPRINT_20260429_015_FindingsLedger_runtime_documentation task FL-RUNTIME-DOC-003 ADR sweep. No changes to the formula or weights were proposed during this sweep.

Sign-off Window

Accepted on 2026-04-29 by Architecture lead. The 14-day default-accept window from sprint 024 (deadline 2026-05-13) closed early via explicit approval rather than the timeout path. Original window text retained below for history.

Following the project’s RFC default-accept process: if this ADR remains Proposed with no objection logged in this document by 2026-05-13 (14 days from sprint 024 commit), it is automatically promoted to Accepted. Architecture-lead may flip status earlier on review. Objections should be added under “Open Issues” below with name + date.

Open Issues

No objections logged during the sign-off window.