FN-Drift Metrics Reference

Audience: Scanner and Telemetry engineers, SREs, and policy owners tracking scanner accuracy. Scope: The metric families, classification model, SLO thresholds, alert rules, and storage schema for False-Negative Drift.

Overview

False-Negative Drift (FN-Drift) measures how often vulnerability classifications change from “not affected” or “unknown” to “affected” during rescans. It is the primary signal for how often Stella Ops missed a vulnerability on an earlier scan. This metric is critical for:

• Accuracy Assessment: Tracking scanner reliability over time
• SLO Compliance: Meeting false-negative rate targets
• Root Cause Analysis: Stratified analysis by drift cause
• Feed Quality: Identifying problematic vulnerability feeds

Metrics

Gauges (30-day rolling window)

Counters (all-time)

Classification Statuses

Drift Causes

FN-Drift Definition

A False-Negative Transition occurs when:

• Previous status was unaffected or unknown
• New status is affected

This indicates the scanner previously classified a finding as “not vulnerable” but now classifies it as “vulnerable” - a false negative in the earlier scan.

FN-Drift Rate Calculation

FN-Drift % = (FN Transitions / Total Reclassified) × 100

Where:

• FN Transitions: Count of (unaffected|unknown) → affected changes
• Total Reclassified: Count of all status changes (excluding new)

SLO Thresholds

Alerting Rules

# Example Prometheus alerting rules
groups:
  - name: fn-drift
    rules:
      - alert: FnDriftWarning
        expr: scanner_fn_drift_percent > 1.0
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "FN-Drift rate above warning threshold"
          
      - alert: FnDriftCritical
        expr: scanner_fn_drift_percent > 2.5
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "FN-Drift rate above critical threshold"
          
      - alert: EngineDriftDetected
        expr: scanner_fn_drift_cause_engine > 0
        for: 1m
        labels:
          severity: page
        annotations:
          summary: "Engine-caused FN drift detected - determinism violation"

Dashboard Queries

FN-Drift Trend (Grafana)

# 30-day rolling FN-Drift percentage
scanner_fn_drift_percent

# FN transitions by cause
sum by (cause) (rate(scanner_fn_transitions_total[1h]))

# Classification changes rate
sum by (cause) (rate(scanner_classification_changes_total[1h]))

Drift Cause Breakdown

# Pie chart of drift causes
topk(5, 
  sum by (cause) (
    increase(scanner_fn_transitions_total[24h])
  )
)

Database Schema

classification_history Table

CREATE TABLE scanner.classification_history (
    id BIGSERIAL PRIMARY KEY,
    artifact_digest TEXT NOT NULL,
    vuln_id TEXT NOT NULL,
    package_purl TEXT NOT NULL,
    tenant_id UUID NOT NULL,
    manifest_id UUID NOT NULL,
    execution_id UUID NOT NULL,
    previous_status TEXT NOT NULL,
    new_status TEXT NOT NULL,
    is_fn_transition BOOLEAN GENERATED ALWAYS AS (...) STORED,
    cause TEXT NOT NULL,
    cause_detail JSONB,
    changed_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

fn_drift_stats Materialized View

Aggregated daily statistics for efficient dashboard queries:

• Day bucket
• Tenant ID
• Cause breakdown
• FN count and percentage

Related Documentation

• Scanner Architecture
• Telemetry Architecture
• Metrics & SLOs