AI Runs Framework

Sprint: SPRINT_20260109_011_003_BE_ai_runs_framework Status: Active Last Updated: 2026-01-10

The AI Runs Framework provides an auditable container for AI-assisted investigations, capturing the complete lifecycle from initial query through tool calls, artifact generation, and approvals.

Overview

“Chat is not auditable, repeatable, actionable with guardrails, or collaborative.”

The Run concept transforms ephemeral chat into:

• Auditable: Every interaction logged with timestamps and content digests
• Repeatable: Deterministic replay possible for verification
• Actionable: Artifacts produced (evidence packs, VEX statements, decisions)
• Collaborative: Handoffs, approvals, shared context across team members

Run Lifecycle

Runs progress through defined states with explicit transitions:

Created -> Active -> PendingApproval -> Completed
                  \-> Cancelled
                  \-> Failed

States

Lifecycle Diagram

+----------+    +---------+    +------------+    +----------+
| Created  | -> | Active  | -> | Pending    | -> | Complete |
|          |    |         |    | Approval   |    |          |
+----------+    +---------+    +------------+    +----------+
     |              |               |                 |
     v              v               v                 v
+--------------------------------------------------+
|                 Run Timeline                      |
|  +-------+ +------+ +---------+ +--------+ +---+ |
|  |Created| | User | |Assistant| | Action | |...| |
|  | Event | | Turn | |  Turn   | |Proposed| |   | |
|  +-------+ +------+ +---------+ +--------+ +---+ |
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
|              Artifacts Produced                   |
|  +---------+ +--------+ +------+ +-----+         |
|  | Evidence| |Decision| |Action| | VEX |         |
|  |   Pack  | | Record | |Result| |Stmt |         |
|  +---------+ +--------+ +------+ +-----+         |
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
|           Run Attestation (DSSE)                  |
|  - Content digest of all turns                   |
|  - Evidence references                           |
|  - Artifact digests                              |
|  - Signed by platform key                        |
+--------------------------------------------------+

Current API Reference

PCD-456 source review at ad9edbcef5 (2026-10-08). The host composes RunEndpoints and then clones its endpoint metadata under /api/advisoryai/v1/advisory-ai/runs. This is the run family, not the separate /api/advisoryai/v1/runs/... task-result family.

View policy accepts advisory-ai:view, advisory-ai:operate or advisory-ai:admin; write operations additionally require operate/admin. Tenant and actor are resolved from envelope claims, not request-body/header identity options. The current resolver still has default/anonymous fallbacks; this review does not certify missing-claim isolation. Runtime store/provider and deployment acceptance remain separate.

The current create response’s Location uses /api/v1/runs/{id}; that header is a source-owner residual, not an alias introduced by this guide. Use the explicitly mapped group above after reviewing its current owner contract.

Unavailable reads and replay

No GET artifact-list or POST run-replay operation is composed. POST /{runId}/artifacts attaches an artifact; it is not a list operation. The library’s stored metadata and run digest do not establish a replay endpoint or deterministic-comparison response.

Attestation boundary

POST /{runId}/attest is the separate signing-dependent operation. RunService.AttestAsync requires IRunAttestationSigner or IAiAttestationEnvelopeSigner, rejecting missing, empty, unsigned or placeholder signatures. The current attestation verification operation is global to the AI family, with a runId request body; see below.

Reverify Program.cs, Endpoints/RunEndpoints.cs, Endpoints/AttestationEndpoints.cs, run DTOs/service and the consolidation mapper under src/AdvisoryAI/. No action, signing key enrollment, runtime query or deployment is performed by this review.

Evidence pack create requests retain truthful evidence digests: valid caller hashes or hashes computed from present normalized snapshot data; empty data with no digest is rejected. This separate owner contract is unchanged.

Timeline Event Types

The timeline captures all significant events during a Run.

Event Details

Each event type may include additional details:

AssistantTurn:

{
  "contentDigest": "sha256:...",
  "groundingScore": 0.92,
  "tokenCount": 847
}

ActionProposed:

{
  "actionType": "approve",
  "label": "Accept Risk",
  "parameters": {
    "cveId": "CVE-2024-1234",
    "rationale": "Not exploitable in our environment"
  }
}

ArtifactCreated:

{
  "artifactId": "art-001",
  "artifactType": "EvidencePack",
  "contentDigest": "sha256:..."
}

Artifact Types

Runs can produce various artifact types:

Artifact Structure

{
  "artifactId": "art-001",
  "type": "EvidencePack",
  "name": "CVE-2024-1234 Evidence Pack",
  "contentDigest": "sha256:def456...",
  "uri": "evidence://packs/art-001",
  "createdAt": "2026-01-10T14:35:00Z",
  "metadata": {
    "cveId": "CVE-2024-1234",
    "component": "pkg:npm/lodash@4.17.21",
    "scanId": "scan-789",
    "format": "evidence-pack/v1"
  }
}

Replay Verification

The replay procedure in this section is design history. No run-replay endpoint is composed, and the following comparison fields are not a current response contract. The proposed goals are:

• Compliance audits: Proving AI outputs are reproducible
• Debugging: Understanding how AI reached conclusions
• Trust: Demonstrating consistent behavior

Replay Requirements

For successful replay:

1. Temperature = 0: No randomness in token selection
2. Fixed seed: Same seed across replays
3. Model match: Same model weights (verified by digest)
4. Prompt match: Identical prompts (verified by hash)
5. Context match: Same input context

Replay Results

Common Divergence Causes

Run Attestation

Completed Runs produce DSSE-signed attestations containing:

Signing is mandatory for this path. If no signer is configured, RunService.AttestAsync fails and leaves the run unattested; operators must wire a real signing implementation before using run-level attestations in production-like hosts.

{
  "_type": "https://stellaops.org/attestation/ai-run/v1",
  "runId": "run-abc123",
  "tenantId": "tenant-xyz",
  "userId": "user@example.com",
  "conversationId": "conv-abc123",
  "startedAt": "2026-01-10T14:30:00Z",
  "completedAt": "2026-01-10T14:40:00Z",
  "model": {
    "modelId": "gpt-4-turbo",
    "modelVersion": "2024-04-09",
    "provider": "azure-openai"
  },
  "promptTemplate": {
    "templateId": "security-investigate",
    "version": "1.2.0",
    "digest": "sha256:..."
  },
  "context": {
    "findingId": "f-456",
    "cveId": "CVE-2024-1234",
    "policyId": "policy-001",
    "evidenceUris": [
      "sbom://scan-789/lodash",
      "reach://api-gateway:lodash.get"
    ]
  },
  "turns": [
    {
      "turnId": "turn-001",
      "role": "User",
      "contentDigest": "sha256:...",
      "timestamp": "2026-01-10T14:30:05Z"
    },
    {
      "turnId": "turn-002",
      "role": "Assistant",
      "contentDigest": "sha256:...",
      "timestamp": "2026-01-10T14:30:08Z",
      "claims": [
        {
          "text": "CVE-2024-1234 is reachable through...",
          "groundingScore": 0.92,
          "groundedBy": ["reach://api-gateway:lodash.get"],
          "verified": true
        }
      ]
    }
  ],
  "overallGroundingScore": 0.89,
  "artifacts": [
    {
      "artifactId": "art-001",
      "type": "EvidencePack",
      "contentDigest": "sha256:..."
    }
  ]
}

Attestation Verification

POST /api/advisoryai/v1/advisory-ai/attestations/verify uses VerifyAttestationRequest with runId, not a runId path segment. Operate/admin authorization and the stored record’s claim tenant are checked. The result is AttestationVerificationResponse with isValid/digestValid/signatureValid and record/key/error/time metadata. Invalid, absent or foreign records return 400; no configured key or live verification is inferred from publication.

UI Guide

Run Timeline View

The Run Timeline component provides a visual representation of all events:

+------------------------------------------------------------------+
| Run: run-abc123                                     [Active]      |
| Started: Jan 10, 2026 14:30                                       |
+------------------------------------------------------------------+
|                                                                   |
|  o  Run Created                                        14:30:00   |
|  |  Run initialized from conversation conv-abc123                 |
|  |                                                                |
|  o  User Turn                                          14:30:05   |
|  |  user@example.com                                              |
|  |  "Is CVE-2024-1234 exploitable in our environment?"            |
|  |                                                                |
|  o  Assistant Turn                                     14:30:08   |
|  |  assistant                                                     |
|  |  "Based on the reachability analysis [reach:api-gateway:...]"  |
|  |  Grounding: 92%                                                |
|  |                                                                |
|  o  Action Proposed                                    14:30:09   |
|  |  [Accept Risk] [Create VEX] [Escalate]                         |
|  |                                                                |
|  o  Artifact Created                                   14:35:00   |
|     Evidence Pack: CVE-2024-1234                                  |
|                                                                   |
+------------------------------------------------------------------+
| Artifacts (1)                                                     |
| +----------------------------+                                    |
| | Evidence Pack              |                                    |
| | CVE-2024-1234 Evidence Pack|                                    |
| | sha256:def456...           |                                    |
| +----------------------------+                                    |
+------------------------------------------------------------------+

Key UI Components

Status Colors

Configuration

AdvisoryAI:
  Runs:
    Enabled: true
    AutoCreate: true          # Auto-create Run from first conversation turn
    RetentionDays: 90         # How long to keep completed Runs
    AttestOnComplete: false   # Attestation requires an explicit signer-backed call
    ReplayEnabled: true       # Allow replay verification

  Timeline:
    MaxEventsPerRun: 1000     # Maximum timeline events per Run
    ContentDigestAlgorithm: sha256

  Artifacts:
    MaxPerRun: 50             # Maximum artifacts per Run
    MaxSizeBytes: 10485760    # 10 MB max artifact size

Error Handling

See Also

• AdvisoryAI Architecture
• Chat Interface
• AI Attestations
• Evidence Locker
• Attestor Module

Last updated: 10-Jan-2026