Reachability Analysis Operations Runbook

Version: 1.1.1
Sprint: 3500.0004.0004
Last Updated: 2026-05-31

This runbook covers operational procedures for Reachability Analysis, including call graph management, analysis troubleshooting, and explain queries.

Source of truth: HTTP routes are defined in src/Scanner/StellaOps.Scanner.WebService/Endpoints/ (CallGraphEndpoints.cs, ReachabilityEndpoints.cs, ReachabilityEvidenceEndpoints.cs, ReachabilityStackEndpoints.cs, ReachabilityDriftEndpoints.cs); CLI verbs in src/Cli/StellaOps.Cli/Commands/ (ReachabilityCommandGroup.cs, ScanGraphCommandGroup.cs); persistence in src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations/ and src/Signals/__Libraries/StellaOps.Signals.Persistence/Migrations/. Where this runbook describes behaviour that is not yet wired end to end, the section is annotated [ROADMAP].

Table of Contents

1. Overview
2. Call Graph Operations
3. Reachability Computation
4. Explain Queries
5. Troubleshooting
6. Monitoring & Alerting
7. Escalation Procedures

1. Overview

What is Reachability Analysis?

Reachability Analysis determines whether vulnerable code is actually reachable from application entrypoints. This reduces false positives by filtering out vulnerabilities in code that cannot be executed.

Reachability Verdicts

The final verdict for a three-layer reachability stack is the ReachabilityVerdict enum (src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Stack/ReachabilityStack.cs). Verdicts are returned by the analyze/result endpoints and the stack endpoints.

Confidence within each layer is the ConfidenceLevel enum (StellaOps.Scanner.Explainability.Assumptions.ConfidenceLevel: Low, Medium, High, Verified). Layer-3 runtime gating uses GatingOutcome (NotGated, Blocked, Conditional, Unknown); per-condition status uses GatingStatus (Enabled, Disabled, Unknown, RuntimeConfigurable).

The component/finding query endpoints (GET /scans/{scanId}/reachability/{components,findings}) return a free-form status string (e.g. confirmed_reachable, unreachable) plus a latticeState (e.g. callable_path_confirmed) supplied by the reachability backend — these are separate from the ReachabilityVerdict enum above. There is no REACHABLE_STATIC / REACHABLE_PROVEN / POSSIBLY_REACHABLE status; those values do not exist in code.

Key Components

Prerequisites

• Access to Scanner WebService API (/api/v1 base path; behind the gateway, the Scanner surface is mounted under /api/v1/scanner).
• OAuth scopes (the runbook examples need read + write + ingest):
  • Read endpoints (components, findings, explain, stack, drift, trace export) require policy scanner.scans.read, satisfied by scope scanner.scans.read or the canonical scanner:read (StellaOpsScopes.ScannerRead).
  • Write endpoints (compute-reachability, analyze, VEX generation) require policy scanner.scans.write, satisfied by scanner.scans.write or scanner:write (StellaOpsScopes.ScannerWrite).
  • Call graph upload requires policy scanner.callgraph.ingest, satisfied only by scope scanner.callgraph.ingest (no canonical-scope alias).
  • There is no scanner.reachability scope.
• CLI access with stella configured (stella reachability ..., stella scan graph ...).
• For source-side call-graph extraction, the per-language extractor binary (stella-callgraph-dotnet, stella-callgraph-java, …) must be on PATH.

2. Call Graph Operations

2.1 Call Graph Upload

The upload endpoint is POST /scans/{scanId}/callgraphs (handler CallGraphEndpoints.HandleSubmitCallGraphAsync). It requires:

• A Content-Digest header (any non-empty value) — used for idempotency. A missing or blank Content-Digest returns 400 Bad Request; a repeat upload with the same digest returns 409 Conflict (the conflict body echoes the existing callgraphId and digest).
• A body matching schema stella.callgraph.v1 (schema, scanKey, language, nodes, edges). At least one node and at least one edge are required; every node needs a unique non-empty nodeId and a non-empty symbolKey; and every edge from/to must reference a declared node id. Supported language values (server upload schema): dotnet, java, node, python, go, rust, binary, ruby, php.

On success the endpoint returns 202 Accepted with { callgraphId, nodeCount, edgeCount, digest }.

# Upload via API (RFC 9530 Content-Digest format).
# Behind the gateway the Scanner surface is under /api/v1/scanner.
DIGEST="sha-256=:$(openssl dgst -sha256 -binary callgraph.json | base64):"
curl -X POST "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/callgraphs" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Content-Digest: $DIGEST" \
  -d @callgraph.json

The CLI extracts a graph from source and (optionally) uploads it in one step. It posts to /api/v1/scanner/scans/{scanId}/callgraphs with a computed Content-Digest (see ScanGraphCommandGroup):

# Extract a .NET call graph and upload it to an existing scan.
# Requires the per-language extractor (stella-callgraph-dotnet) on PATH.
stella scan graph --lang dotnet --target ./src --upload --scan-id $SCAN_ID

# .NET extraction can be pointed at a specific solution file with --sln:
stella scan graph --lang dotnet --sln ./MyApp.sln --target ./src --output callgraph.json

# Extract only, write JSON to a file (no upload):
stella scan graph --lang java --target ./build --output callgraph.json --format json

Note: the CLI scan graph verb supports the languages dotnet, java, node, python, go, rust, ruby, php (ScanGraphCommandGroup); it does not accept binary. The server upload schema additionally accepts binary (binary call graphs are produced by the worker-side binary analyzer, not by stella scan graph). The CLI --format accepts json (default), dot, and summary for the local extraction output; summary here is a --format value, not a stella scan graph summary subcommand.

Not implemented: there is no stella scan graph upload, --format ndjson, --streaming, or server-side NDJSON streaming ingestion. Uploads are a single JSON POST. Large graphs are handled by the body-size limits in 5.1, not by a streaming verb.

2.2 Call Graph Inspection

The server-side call graph is stored in signals.cg_nodes / signals.cg_edges (populated by the Signals call-graph sync, see src/Signals/__Libraries/StellaOps.Signals.Persistence/Migrations/001_initial_schema.sql). The CLI keeps the planned graph-query names discoverable, but they are not wired to the Signals/ReachGraph store and fail closed with exit code 9:

# Discoverable roadmap commands; no graph data is emitted
stella reachability graph list --scan $SCAN_ID

# Show graph metadata
stella reachability graph show <graph-digest>

# Slice a graph by CVE/PURL
stella reachability graph slice --digest <graph-digest> --cve CVE-2024-1234 --depth 3

To render a subgraph from a previously exported subgraph JSON file (DOT/Mermaid/SVG):

stella reachability show   --input subgraph.json --format mermaid
stella reachability export --input subgraph.json --output subgraph.dot --format dot

Not implemented: stella scan graph summary, stella scan graph entrypoints, stella scan graph export --scan-id, and stella scan graph visualize do not exist. stella reachability graph {replay,verify} exist as verbs but return a “not implemented” exit code (the deterministic replay / Attestor verification services are not wired here).

2.3 Call Graph Validation

Validation happens server-side at upload time (see the validation rules in CallGraphEndpoints.ValidateCallGraph): schema must equal stella.callgraph.v1; scanKey and language are required; language must be supported; each node’s nodeId must be unique and non-empty and each node’s symbolKey must be non-empty; at least one node and one edge are required; and every edge endpoint must reference a declared node. A failing body returns 400 Bad Request with an errors array.

Not implemented: there is no stella scan graph validate CLI verb and no separate /validate endpoint. Validate by attempting the upload (use a disposable scan) and inspecting the 400 error list.

2.4 Multi-language graphs

A scan can receive one call graph per language (each upload carries its own language field and is keyed by (scan_id, language, graph_digest) in the drift snapshot table — see call_graph_snapshots, migration 009). The Cartographer assembler (src/Scanner/StellaOps.Scanner.Cartographer/) is responsible for assembling graphs.

Not implemented: there is no stella scan graph merges / stella scan graph merge CLI verb and no operator-facing “force re-merge” command.

3. Reachability Computation

3.1 Triggering Computation

There are two compute surfaces:

1. Per-scan recompute — POST /scans/{scanId}/compute-reachability (handler ReachabilityEndpoints.HandleComputeReachabilityAsync). Body is optional (ComputeReachabilityRequestDto: forceRecompute, entrypoints, targets). Returns 202 Accepted with { jobId, status, estimatedDuration }. If a computation is already running it returns 409 Conflict. If no reachability runtime backend is configured the endpoint returns 501 Not Implemented.
2. CVE-scoped analyze — POST /reachability/analyze (handler ReachabilityEvidenceEndpoints.AnalyzeAsync). Body ReachabilityAnalyzeRequest: imageDigest, cveId, purl (all required), plus sourceCommit, includeBinaryAnalysis (L2), includeRuntimeAnalysis (L3), maxPaths (default 5), maxDepth (default 256). Returns 200 OK with { jobId, status, verdict, evidenceUri, durationMs, error } (note: analyze returns 200, not 202). Returns 400 when imageDigest/cveId/purl are missing, and 404 when the CVE has no sink mappings.

# Per-scan recompute via API
curl -X POST "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/compute-reachability" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"forceRecompute": true}'

# CVE-scoped analyze via API
curl -X POST "https://scanner.example.com/api/v1/scanner/reachability/analyze" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"imageDigest":"sha256:...","cveId":"CVE-2024-1234","purl":"pkg:npm/lodash@4.17.20","includeBinaryAnalysis":true,"includeRuntimeAnalysis":true}'

Not implemented: there is no stella reachability compute CLI verb. The compute trigger is API-only today. The route is /compute-reachability, not /reachability/compute.

3.2 Analyze options

The CVE-scoped analyze body accepts the following (defaults from ReachabilityEvidenceEndpoints / ReachabilityJobOptions):

Not implemented: indirect-resolution, timeout, parallel, and include-runtime are not parameters of either compute endpoint. Runtime evidence participation is controlled by includeRuntimeAnalysis on the analyze body.

3.3 Retrieving Job Results

The async result for a CVE-scoped analyze is fetched by job id: GET /reachability/result/{jobId} (handler ReachabilityEvidenceEndpoints.GetResultAsync). The job id is deterministic — ReachabilityEvidenceJob.ComputeJobId(imageDigest, cveId, purl) — so you can re-derive it instead of remembering it.

curl "https://scanner.example.com/api/v1/scanner/reachability/result/$JOB_ID" \
  -H "Authorization: Bearer $TOKEN"

# Response shape (ReachabilityResultResponse):
# { "jobId": "...", "status": "Completed", "verdict": "LikelyExploitable",
#   "verdictExplanation": "...", "isReachable": true, "pathCount": 1,
#   "entrypointCount": 2, "evidenceBundleId": "...", "evidenceUri": "...",
#   "completedAt": "...", "durationMs": 1234, "error": null }

Not implemented: stella reachability job-status, job-logs, and job-cancel do not exist; there is no progress/log-streaming or cancellation surface. The analyze executor runs synchronously and the result is stored under the deterministic job id.

3.4 Checking CVE Sink Mappings

analyze (3.1) returns 404 when a CVE has no sink mappings. To check the CVE→symbol mappings directly (e.g. before an analyze run), use GET /reachability/mapping/{cveId} (handler ReachabilityEvidenceEndpoints.GetCveMappingAsync, inherits the group default policy scanner.scans.read). An optional ?purl= query parameter restricts the lookup to one package. Returns 404 when no mappings exist for the CVE.

curl "https://scanner.example.com/api/v1/scanner/reachability/mapping/CVE-2024-1234?purl=pkg:npm/lodash@4.17.20" \
  -H "Authorization: Bearer $TOKEN"

# Response shape (CveMappingResponse):
# { "cveId": "CVE-2024-1234", "mappingCount": 1,
#   "mappings": [ { "symbolName": "...", "canonicalId": "...", "purl": "...",
#                   "filePath": "...", "vulnType": "...", "confidence": 0.9,
#                   "source": "..." } ] }

3.5 Querying Results per Scan

For scan-level results use the read endpoints (all require scanner.scans.read):

# Components with reachability status
curl "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/reachability/components" \
  -H "Authorization: Bearer $TOKEN"

# Findings (CVE-level) with reachability status; optional ?cve= and ?status= filters
curl "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/reachability/findings?status=confirmed_reachable" \
  -H "Authorization: Bearer $TOKEN"

# Deterministic trace export (path proof / sensor-gap report)
curl "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/reachability/traces/export?format=json" \
  -H "Authorization: Bearer $TOKEN"

The findings/components status filter takes the backend’s free-form status strings (e.g. confirmed_reachable, unreachable), not the ReachabilityVerdict enum.

The CLI surfaces the trace export (stella reachability trace, see ReachabilityCommandGroup.BuildTraceExportCommand):

# Export a deterministic reachability trace for a scan
stella reachability trace --scan-id $SCAN_ID --format json-lines --output trace.jsonl

Not implemented: stella reachability summary, stella reachability findings, and stella reachability explain-all do not exist. There is no SARIF export of reachability findings from this surface. The trace export supports formats json-lines (default), json, graphson, and sarif on the API; the CLI trace verb advertises json-lines and graphson.

4. Explain Queries

4.1 Explain a Single Finding (API)

The explain endpoint is GET /scans/{scanId}/reachability/explain (handler ReachabilityEndpoints.HandleExplainAsync). Both cve and purl are required query parameters; omitting either returns 400 Bad Request. A missing CVE/PURL combination returns 404 Not Found.

curl "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/reachability/explain?cve=CVE-2024-1234&purl=pkg:npm/lodash@4.17.20" \
  -H "Authorization: Bearer $TOKEN"

The response is a ReachabilityExplanationDto:

{
  "cveId": "CVE-2024-1234",
  "purl": "pkg:npm/lodash@4.17.20",
  "status": "confirmed_reachable",
  "confidence": 0.91,
  "latticeState": "callable_path_confirmed",
  "pathWitness": [
    "Demo.Api.Controllers.UploadController.Post",
    "Demo.App.Services.Parser.Parse",
    "Demo.VulnerableSink.Invoke"
  ],
  "why": [ { "code": "...", "description": "...", "impact": "..." } ],
  "evidence": {
    "staticAnalysis": { "callgraphDigest": "sha256:...", "pathLength": 3, "edgeTypes": ["static-call"] },
    "runtimeEvidence": { "observed": false, "hitCount": 0, "lastObserved": null },
    "policyEvaluation": { "policyDigest": "...", "verdict": "...", "verdictReason": "..." }
  },
  "spineId": "..."
}

status and latticeState are free-form strings from the reachability backend. The returned confidence is a 0.0–1.0 double, and pathWitness is the ordered symbol path.

4.2 Explain via CLI

The CLI explain verb takes an image digest argument (not --scan-id/--purl) and an optional --vuln:

stella reachability explain <image-digest> --vuln CVE-2024-1234 --format text

Not implemented (2026-07-29): the CLI reachability explain, witness, and guards verbs have no authoritative query client. They return exit code 9 and emit no verdict, path, guard, confidence, or count. Use the API explain endpoint (4.1) for authoritative results.

Not implemented: there is no --all-paths flag (the explain endpoint returns one path witness), no stella reachability explain --scan-id ... --purl ... form, and no stella reachability explain-all batch verb. The status/confidence values shown above come from the backend; the legacy REACHABLE_STATIC / UNREACHABLE literal statuses do not exist.

4.3 Reachability Stack (per-finding 3-layer breakdown)

To inspect the full three-layer stack for a finding, use the stack endpoints (ReachabilityStackEndpoints, all require scanner.scans.read):

# Full 3-layer breakdown for a finding
curl "https://scanner.example.com/api/v1/scanner/reachability/$FINDING_ID/stack" \
  -H "Authorization: Bearer $TOKEN"

# A single layer (1=static call graph, 2=binary resolution, 3=runtime gating)
curl "https://scanner.example.com/api/v1/scanner/reachability/$FINDING_ID/stack/layer/1" \
  -H "Authorization: Bearer $TOKEN"

# Ordered deterministic stack frames for a scan (paged: ?offset=&limit=, max 500)
curl "https://scanner.example.com/api/v1/scanner/reachability/$SCAN_GUID/stack/frames?limit=100" \
  -H "Authorization: Bearer $TOKEN"

When no stack repository is registered for the deployment, the /stack and /stack/layer/{n} endpoints return 501 Not Implemented. The /stack/frames endpoint does not gate on the stack repository — it returns 404 when the scan id is unknown and 400 for a non-GUID scan id, empty GUID, negative offset, or limit < 1.

4.4 Reachability Drift (between two scans)

Drift compares the head scan’s call graph against a baseline scan to surface sinks that became reachable or unreachable (ReachabilityDriftEndpoints, both require scanner.scans.read):

# Compute (or fetch the latest stored) drift for a scan.
# With ?baseScanId= a fresh comparison is computed and stored; without it, the latest
# stored drift result for (scan, language) is returned (404 if none recorded).
curl "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/drift?baseScanId=$BASE_SCAN_ID&language=dotnet&includeFullPath=false" \
  -H "Authorization: Bearer $TOKEN"

# List the drifted sinks for a stored drift result (paged; default limit 100, max 500)
curl "https://scanner.example.com/api/v1/scanner/drift/$DRIFT_ID/sinks?direction=became_reachable&offset=0&limit=100" \
  -H "Authorization: Bearer $TOKEN"

• language is normalized through IDriftLanguageRouter; an unsupported (non-tier-1) language returns 400 Bad Request listing the supported languages. A missing or unknown head/base scan, or a missing call-graph snapshot for either side, returns 404.
• direction accepts became_reachable (also newly_reachable/reachable/up) or became_unreachable (also newly_unreachable/unreachable/down); it defaults to became_reachable. driftId must be a non-empty GUID, offset must be >= 0, and limit must be 1..500.
• Drift results and drifted sinks persist to scanner.reachability_drift_results / scanner.drifted_sinks (migration 010).

Not implemented: there is no stella reachability drift / stella scan graph drift CLI verb on this surface. Drift is API-only today. (A separate top-level stella drift command group exists for a different change-impact surface — it does not call these scan-drift endpoints.)

5. Troubleshooting

5.1 Call Graph Too Large

Symptom: Upload fails with 413 Payload Too Large (the upload endpoint declares this status; the limit is governed by the configured request body limits).

Diagnosis:

# Check graph size
du -h callgraph.json

# Count nodes/edges (schema uses top-level "nodes" and "edges" arrays)
jq '.nodes | length' callgraph.json
jq '.edges | length' callgraph.json

Resolution:

• Extract per-language graphs separately and upload one graph per language (each upload is keyed by (scan_id, language, graph_digest)), instead of a single combined body.
• Restrict extraction scope (exclude test projects — the CLI omits them unless --include-tests is passed; trim third-party packages you do not analyze).
• Raise the Scanner request-body limit if the deployment legitimately needs larger graphs.

Not implemented: stella scan graph upload --streaming, stella scan graph convert (NDJSON), and stella scan graph partition do not exist. There is no NDJSON or streaming ingestion path; uploads are a single JSON POST.

5.2 Missing Entrypoints

Symptom: Reachability returns no callable paths / “no entrypoints” for an artifact you expect to be reachable.

Diagnosis:

• Confirm the uploaded graph actually marks entrypoint candidates: in the stella.callgraph.v1 body, nodes set isEntrypointCandidate: true. The store records this as signals.cg_nodes.is_entrypoint_candidate and tracks entrypoint_count in the drift snapshot (call_graph_snapshots, migration 009).
• Verify the correct language extractor ran — entrypoint classification is language-specific (StellaOps.Scanner.CallGraph/Extraction/<lang>/...EntrypointClassifier).

Common causes:

1. Wrong / no language extractor: the artifact’s language was not extracted.
2. Framework entrypoints not classified: framework-specific entrypoints (HTTP routes, message handlers) were not detected by the language classifier.
3. No entrypoint candidates in the uploaded graph: extractor produced an internal-only subgraph.

Resolution:

• Re-extract with the correct --lang and re-upload.
• Ensure entrypoint nodes carry isEntrypointCandidate: true before upload.

Not implemented: stella scan graph entrypoints, stella scan graph detect-framework, stella scan graph upload --framework, and stella scan graph entrypoint add do not exist. There is no operator command to inject custom entrypoints post-upload — encode entrypoints in the extracted graph.

5.3 Reachability Computation Unavailable / Slow

Symptom: POST .../compute-reachability (or a query endpoint) returns 501 Not Implemented, or analyze runs are slow.

Diagnosis:

• A 501 from compute/query endpoints means no reachability runtime backend is configured for the deployment (the handlers translate ReachabilityRuntimeUnavailableException → 501). The trace export endpoint instead emits an explicit sensor_gap payload listing runtime-trace-backend-unavailable / reachability-query-backend-unavailable.
• For analyze cost, the dominant levers are body options: maxDepth (default 256), maxPaths (default 5), and whether L2/L3 are enabled.

Resolution:

# Reduce Layer-1 search cost on the analyze body
curl -X POST "https://scanner.example.com/api/v1/scanner/reachability/analyze" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"imageDigest":"sha256:...","cveId":"CVE-2024-1234","purl":"pkg:npm/lodash@4.17.20","maxDepth":32,"maxPaths":3,"includeBinaryAnalysis":false,"includeRuntimeAnalysis":false}'

Not implemented: stella reachability job-stats, --timeout, --max-depth, --indirect-resolution, and --partition-by flags do not exist. There is no compute timeout knob; tune cost via the analyze body fields above.

5.4 Inconsistent Results

Symptom: Different results between runs.

Diagnosis:

• The trace export is deterministic by construction: it canonicalizes scan + options + sensor gaps + nodes + edges and emits a contentDigest (sha256:...). Two exports of the same scan/options MUST return the same contentDigest — compare those.
• Analyze job ids are deterministic (ComputeJobId(imageDigest, cveId, purl)), so the same inputs map to the same stored result.
• Graph identity is the graph_digest recorded in call_graph_snapshots; compare digests across runs to confirm you are analyzing the same graph.

# Compare the deterministic trace export digest across two runs
curl -s "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/reachability/traces/export?format=json" \
  -H "Authorization: Bearer $TOKEN" | jq -r '.contentDigest'

Not implemented: stella scan manifest, stella scan graph hash, and the --deterministic / --seed / --graph-digest compute flags do not exist. Determinism is enforced by the canonicalized trace-export digest, not by a seed parameter.

5.5 False Positives/Negatives

Symptom: Reachability verdict seems incorrect.

Diagnosis:

# Authoritative explanation (path witness + evidence chain)
curl "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/reachability/explain?cve=CVE-2024-1234&purl=pkg:npm/lodash@4.17.20" \
  -H "Authorization: Bearer $TOKEN"

# Per-layer stack to see which layer drove the verdict
curl "https://scanner.example.com/api/v1/scanner/reachability/$FINDING_ID/stack" \
  -H "Authorization: Bearer $TOKEN"

Inspect evidence.staticAnalysis.edgeTypes and evidence.runtimeEvidence.observed in the explanation, and the L1/L2/L3 breakdown in the stack, to see which layer drove the verdict.

Common causes for false positives:

1. Heuristic edges: indirect call edges (signals.cg_edges.kind is a SMALLINT where 0=static, 1=heuristic, 2=runtime; kind=1 heuristic edges) inflate the path.
2. Reflection/dynamic calls: may create paths that never execute.
3. Dead code not detected: code exists but is never invoked.

Common causes for false negatives:

1. Missing edges: the uploaded call graph is incomplete.
2. Cross-language calls: a language boundary is not bridged in the graph.
3. Runtime layer disabled: a path confirmable only at runtime (L3) was not analyzed because includeRuntimeAnalysis was false.

Resolution:

• Re-extract and re-upload a more complete call graph (the path quality is bounded by the graph you submit).
• Re-run analyze with includeBinaryAnalysis/includeRuntimeAnalysis enabled to add L2/L3 evidence.
• Generate a VEX statement from a completed reachability job to record the determination: POST /reachability/vex (ReachabilityEvidenceEndpoints.GenerateVexAsync, body { jobId, productId }, requires scanner.scans.write).

Not implemented: there is no stella scan graph edge inspector, no stella scan evidence upload runtime-trace ingestion verb on this surface, and no stella reachability feedback false-positive reporting / ML-training loop. Record determinations via the VEX endpoint above.

6. Monitoring & Alerting

[ROADMAP] The metric names, Grafana dashboard, and Prometheus alert rules in this section are a target monitoring design, not the metrics emitted today. The reachability meters currently wired in code live in the StellaOps.Scanner.Reachability library (cache / PR-gate paths), not the WebService HTTP endpoints:

• StellaOps.Scanner.Reachability.Surfaces — counters stellaops.surface_query.cache_hits / .cache_misses / .surface_hits / .surface_misses (see SurfaceQueryService).
• StellaOps.Scanner.Reachability.Cache — counters stellaops.reachability_cache.hits / .misses / .full_recomputes / .incremental_computes and histogram stellaops.reachability_cache.analysis_duration_ms (see IncrementalReachabilityService).
• StellaOps.Scanner.Reachability.PrGate — counters stellaops.reachability_prgate.passed / .blocked (see PrReachabilityGate).

None of the callgraph_* / reachability_computation_* / reachability_nodes_visited / reachability_job_failures_total / entrypoint_detection_rate series below are wired — treat them as the intended SLO shape, not as queryable metrics.

6.1 Key Metrics (target)

6.2 Grafana Dashboard

Dashboard: Reachability Operations
Panels:
- Call graph upload throughput
- Graph size distribution
- Computation duration (p50, p95, p99)
- Reachability verdict distribution
- Job queue depth
- Entrypoint detection rate

6.3 Alerting Rules

groups:
  - name: reachability
    rules:
      - alert: ReachabilityComputationSlow
        expr: histogram_quantile(0.95, reachability_computation_duration_seconds) > 300
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "Reachability computation is slow"
          
      - alert: ReachabilityJobFailures
        expr: increase(reachability_job_failures_total[1h]) > 5
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Multiple reachability job failures"
          
      - alert: LowEntrypointDetectionRate
        expr: entrypoint_detection_rate < 0.8
        for: 1h
        labels:
          severity: warning
        annotations:
          summary: "Entrypoint detection rate is low"

7. Escalation Procedures

7.1 Escalation Matrix

7.2 P1 Response Procedure

1. Acknowledge alert
2. Triage:

   # Inspect / adjust the scanner worker configuration (get|set are the only subcommands)
   stella scanner workers get
   
   # Probe platform health (queries the live platform-readiness backend API)
   stella admin diagnostics health --detail
   
   # A 501 from compute/query endpoints == no reachability runtime backend configured
   curl -i -X POST "https://scanner.example.com/api/v1/scanner/scans/$SCAN_ID/compute-reachability" \
     -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{}'
   

3. Mitigate:

   # Adjust worker count via the supported config (then redeploy the worker)
   stella scanner workers set --count 10 --pool default
   
   # Re-derive a stuck analyze job's deterministic id to re-inspect / re-run it
   # (analyze runs synchronously; there is no job-cancel verb — re-POST /reachability/analyze
   #  with forceRecompute semantics via the compute endpoint, or restart the worker)
   

4. Communicate: Update status page
5. Resolve: Fix root cause (most often: provision the reachability runtime backend, or re-upload a complete call graph — see 5.2/5.3)
6. Postmortem: Document within 48 hours

Not implemented: stella scanner workers status/scale, stella health check, and stella reachability jobs [cancel] do not exist. The scanner worker group exposes only get/set (--count required, --pool optional); health lives at stella admin diagnostics health (a live platform-readiness backend call via GetPlatformReadinessAsync; flags are --detail and --format table|json, no --service filter — --service is on admin diagnostics logs); and analyze jobs have no list/cancel surface (see 3.3).

Related Documentation

• Reachability API Reference
• Scanner Architecture
• Call Graph Schema (stella.callgraph.v1)
• Entrypoint Detection

Last Updated: 2026-05-31
Version: 1.1.1
Sprint: 3500.0004.0004