AI Runs Framework

Sprint: SPRINT_20260109_011_003_BE_ai_runs_framework Status: Active Last Updated: 2026-01-10

The AI Runs Framework provides an auditable container for AI-assisted investigations, capturing the complete lifecycle from initial query through tool calls, artifact generation, and approvals.

Overview

“Chat is not auditable, repeatable, actionable with guardrails, or collaborative.”

The Run concept transforms ephemeral chat into:

• Auditable: Every interaction logged with timestamps and content digests
• Repeatable: Deterministic replay possible for verification
• Actionable: Artifacts produced (evidence packs, VEX statements, decisions)
• Collaborative: Handoffs, approvals, shared context across team members

Run Lifecycle

Runs progress through defined states with explicit transitions:

Created -> Active -> PendingApproval -> Completed
                  \-> Cancelled
                  \-> Failed

States

Lifecycle Diagram

+----------+    +---------+    +------------+    +----------+
| Created  | -> | Active  | -> | Pending    | -> | Complete |
|          |    |         |    | Approval   |    |          |
+----------+    +---------+    +------------+    +----------+
     |              |               |                 |
     v              v               v                 v
+--------------------------------------------------+
|                 Run Timeline                      |
|  +-------+ +------+ +---------+ +--------+ +---+ |
|  |Created| | User | |Assistant| | Action | |...| |
|  | Event | | Turn | |  Turn   | |Proposed| |   | |
|  +-------+ +------+ +---------+ +--------+ +---+ |
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
|              Artifacts Produced                   |
|  +---------+ +--------+ +------+ +-----+         |
|  | Evidence| |Decision| |Action| | VEX |         |
|  |   Pack  | | Record | |Result| |Stmt |         |
|  +---------+ +--------+ +------+ +-----+         |
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
|           Run Attestation (DSSE)                  |
|  - Content digest of all turns                   |
|  - Evidence references                           |
|  - Artifact digests                              |
|  - Signed by platform key                        |
+--------------------------------------------------+

API Reference

Create Run

Creates a new Run from an existing conversation.

POST /api/v1/advisory-ai/runs
Content-Type: application/json
Authorization: Bearer <token>
X-StellaOps-TenantId: <tenant-id>

{
  "conversationId": "conv-abc123",
  "context": {
    "findingId": "f-456",
    "cveId": "CVE-2024-1234",
    "component": "pkg:npm/lodash@4.17.21",
    "scanId": "scan-789",
    "sbomId": "sbom-xyz"
  }
}

Response (201 Created):

{
  "runId": "run-abc123",
  "tenantId": "tenant-xyz",
  "userId": "user@example.com",
  "conversationId": "conv-abc123",
  "status": "Created",
  "createdAt": "2026-01-10T14:30:00Z",
  "context": {
    "findingId": "f-456",
    "cveId": "CVE-2024-1234",
    "component": "pkg:npm/lodash@4.17.21",
    "scanId": "scan-789",
    "sbomId": "sbom-xyz"
  },
  "timeline": [],
  "artifacts": []
}

Get Run

Retrieves a Run with its complete timeline and artifacts.

GET /api/v1/advisory-ai/runs/{runId}
Authorization: Bearer <token>
X-StellaOps-TenantId: <tenant-id>

Response (200 OK):

{
  "runId": "run-abc123",
  "tenantId": "tenant-xyz",
  "userId": "user@example.com",
  "conversationId": "conv-abc123",
  "status": "Active",
  "createdAt": "2026-01-10T14:30:00Z",
  "completedAt": null,
  "context": { ... },
  "timeline": [
    {
      "eventId": "evt-001",
      "eventType": "RunCreated",
      "timestamp": "2026-01-10T14:30:00Z",
      "actor": "system",
      "summary": "Run created from conversation conv-abc123"
    },
    {
      "eventId": "evt-002",
      "eventType": "UserTurn",
      "timestamp": "2026-01-10T14:30:05Z",
      "actor": "user:user@example.com",
      "summary": "Is CVE-2024-1234 exploitable in our environment?",
      "relatedTurnId": "turn-001"
    },
    {
      "eventId": "evt-003",
      "eventType": "AssistantTurn",
      "timestamp": "2026-01-10T14:30:08Z",
      "actor": "assistant",
      "summary": "Based on the reachability analysis...",
      "details": {
        "contentDigest": "sha256:abc123...",
        "groundingScore": 0.92
      },
      "relatedTurnId": "turn-002"
    }
  ],
  "artifacts": [],
  "attestationDigest": null
}

Get Timeline

Returns paginated timeline events for a Run.

GET /api/v1/advisory-ai/runs/{runId}/timeline?limit=50&cursor=evt-050
Authorization: Bearer <token>
X-StellaOps-TenantId: <tenant-id>

Response (200 OK):

{
  "runId": "run-abc123",
  "events": [ ... ],
  "cursor": "evt-100",
  "hasMore": true
}

Get Artifacts

Lists all artifacts attached to a Run.

GET /api/v1/advisory-ai/runs/{runId}/artifacts
Authorization: Bearer <token>
X-StellaOps-TenantId: <tenant-id>

Response (200 OK):

{
  "runId": "run-abc123",
  "artifacts": [
    {
      "artifactId": "art-001",
      "type": "EvidencePack",
      "name": "CVE-2024-1234 Evidence Pack",
      "contentDigest": "sha256:def456...",
      "uri": "evidence://packs/art-001",
      "createdAt": "2026-01-10T14:35:00Z",
      "metadata": {
        "cveId": "CVE-2024-1234",
        "component": "pkg:npm/lodash@4.17.21"
      }
    }
  ]
}

Complete Run

Marks a Run as complete and records the deterministic run content digest. Attestation is a separate fail-closed step: RunService.AttestAsync requires a configured IRunAttestationSigner or registered IAiAttestationEnvelopeSigner, and it rejects missing, empty, unsigned, or placeholder signatures instead of persisting a mock attestation.

POST /api/v1/advisory-ai/runs/{runId}/complete
Authorization: Bearer <token>
X-StellaOps-TenantId: <tenant-id>

Response (200 OK):

{
  "runId": "run-abc123",
  "status": "Completed",
  "completedAt": "2026-01-10T14:40:00Z",
  "contentDigest": "sha256:xyz789...",
  "isAttested": false
}

Cancel Run

Cancels an active Run.

POST /api/v1/advisory-ai/runs/{runId}/cancel
Content-Type: application/json
Authorization: Bearer <token>
X-StellaOps-TenantId: <tenant-id>

{
  "reason": "Investigation no longer needed - issue resolved"
}

Response (200 OK):

{
  "runId": "run-abc123",
  "status": "Cancelled",
  "cancelledAt": "2026-01-10T14:35:00Z",
  "reason": "Investigation no longer needed - issue resolved"
}

Replay Run

Replays a Run for verification and determinism checking.

POST /api/v1/advisory-ai/runs/{runId}/replay
Authorization: Bearer <token>
X-StellaOps-TenantId: <tenant-id>

Response (200 OK):

{
  "runId": "run-abc123",
  "deterministic": true,
  "originalDigest": "sha256:abc123...",
  "replayDigest": "sha256:abc123...",
  "differences": []
}

Response (Non-deterministic):

{
  "runId": "run-abc123",
  "deterministic": false,
  "originalDigest": "sha256:abc123...",
  "replayDigest": "sha256:def456...",
  "differences": [
    "Turn 2: original=sha256:111..., replay=sha256:222..."
  ]
}

List Runs

Lists Runs with filtering and pagination.

GET /api/v1/advisory-ai/runs?userId=user@example.com&status=Active&limit=20
Authorization: Bearer <token>
X-StellaOps-TenantId: <tenant-id>

Query Parameters:

Response (200 OK):

{
  "runs": [ ... ],
  "cursor": "run-xyz",
  "hasMore": true
}

Evidence pack API create requests must attach truthful evidence item digests. A caller-provided digest must be a valid sha256:<64 hex> value; sha256:unknown is rejected. If a digest is omitted, the service computes a digest from normalized snapshot data only when snapshot data is present. Empty snapshot data without a caller digest is a bad request.

Timeline Event Types

The timeline captures all significant events during a Run.

Event Details

Each event type may include additional details:

AssistantTurn:

{
  "contentDigest": "sha256:...",
  "groundingScore": 0.92,
  "tokenCount": 847
}

ActionProposed:

{
  "actionType": "approve",
  "label": "Accept Risk",
  "parameters": {
    "cveId": "CVE-2024-1234",
    "rationale": "Not exploitable in our environment"
  }
}

ArtifactCreated:

{
  "artifactId": "art-001",
  "artifactType": "EvidencePack",
  "contentDigest": "sha256:..."
}

Artifact Types

Runs can produce various artifact types:

Artifact Structure

{
  "artifactId": "art-001",
  "type": "EvidencePack",
  "name": "CVE-2024-1234 Evidence Pack",
  "contentDigest": "sha256:def456...",
  "uri": "evidence://packs/art-001",
  "createdAt": "2026-01-10T14:35:00Z",
  "metadata": {
    "cveId": "CVE-2024-1234",
    "component": "pkg:npm/lodash@4.17.21",
    "scanId": "scan-789",
    "format": "evidence-pack/v1"
  }
}

Replay Verification

Runs can be replayed to verify determinism. This is crucial for:

• Compliance audits: Proving AI outputs are reproducible
• Debugging: Understanding how AI reached conclusions
• Trust: Demonstrating consistent behavior

Replay Requirements

For successful replay:

1. Temperature = 0: No randomness in token selection
2. Fixed seed: Same seed across replays
3. Model match: Same model weights (verified by digest)
4. Prompt match: Identical prompts (verified by hash)
5. Context match: Same input context

Replay Results

Common Divergence Causes

Run Attestation

Completed Runs produce DSSE-signed attestations containing:

Signing is mandatory for this path. If no signer is configured, RunService.AttestAsync fails and leaves the run unattested; operators must wire a real signing implementation before using run-level attestations in production-like hosts.

{
  "_type": "https://stellaops.org/attestation/ai-run/v1",
  "runId": "run-abc123",
  "tenantId": "tenant-xyz",
  "userId": "user@example.com",
  "conversationId": "conv-abc123",
  "startedAt": "2026-01-10T14:30:00Z",
  "completedAt": "2026-01-10T14:40:00Z",
  "model": {
    "modelId": "gpt-4-turbo",
    "modelVersion": "2024-04-09",
    "provider": "azure-openai"
  },
  "promptTemplate": {
    "templateId": "security-investigate",
    "version": "1.2.0",
    "digest": "sha256:..."
  },
  "context": {
    "findingId": "f-456",
    "cveId": "CVE-2024-1234",
    "policyId": "policy-001",
    "evidenceUris": [
      "sbom://scan-789/lodash",
      "reach://api-gateway:lodash.get"
    ]
  },
  "turns": [
    {
      "turnId": "turn-001",
      "role": "User",
      "contentDigest": "sha256:...",
      "timestamp": "2026-01-10T14:30:05Z"
    },
    {
      "turnId": "turn-002",
      "role": "Assistant",
      "contentDigest": "sha256:...",
      "timestamp": "2026-01-10T14:30:08Z",
      "claims": [
        {
          "text": "CVE-2024-1234 is reachable through...",
          "groundingScore": 0.92,
          "groundedBy": ["reach://api-gateway:lodash.get"],
          "verified": true
        }
      ]
    }
  ],
  "overallGroundingScore": 0.89,
  "artifacts": [
    {
      "artifactId": "art-001",
      "type": "EvidencePack",
      "contentDigest": "sha256:..."
    }
  ]
}

Attestation Verification

POST /api/v1/advisory-ai/runs/{runId}/attestation/verify
Authorization: Bearer <token>
X-StellaOps-TenantId: <tenant-id>

Response:

{
  "valid": true,
  "runId": "run-abc123",
  "attestationDigest": "sha256:...",
  "signatureValid": true,
  "contentValid": true,
  "verifiedAt": "2026-01-10T15:00:00Z"
}

UI Guide

Run Timeline View

The Run Timeline component provides a visual representation of all events:

+------------------------------------------------------------------+
| Run: run-abc123                                     [Active]      |
| Started: Jan 10, 2026 14:30                                       |
+------------------------------------------------------------------+
|                                                                   |
|  o  Run Created                                        14:30:00   |
|  |  Run initialized from conversation conv-abc123                 |
|  |                                                                |
|  o  User Turn                                          14:30:05   |
|  |  user@example.com                                              |
|  |  "Is CVE-2024-1234 exploitable in our environment?"            |
|  |                                                                |
|  o  Assistant Turn                                     14:30:08   |
|  |  assistant                                                     |
|  |  "Based on the reachability analysis [reach:api-gateway:...]"  |
|  |  Grounding: 92%                                                |
|  |                                                                |
|  o  Action Proposed                                    14:30:09   |
|  |  [Accept Risk] [Create VEX] [Escalate]                         |
|  |                                                                |
|  o  Artifact Created                                   14:35:00   |
|     Evidence Pack: CVE-2024-1234                                  |
|                                                                   |
+------------------------------------------------------------------+
| Artifacts (1)                                                     |
| +----------------------------+                                    |
| | Evidence Pack              |                                    |
| | CVE-2024-1234 Evidence Pack|                                    |
| | sha256:def456...           |                                    |
| +----------------------------+                                    |
+------------------------------------------------------------------+

Key UI Components

Status Colors

Configuration

AdvisoryAI:
  Runs:
    Enabled: true
    AutoCreate: true          # Auto-create Run from first conversation turn
    RetentionDays: 90         # How long to keep completed Runs
    AttestOnComplete: false   # Attestation requires an explicit signer-backed call
    ReplayEnabled: true       # Allow replay verification

  Timeline:
    MaxEventsPerRun: 1000     # Maximum timeline events per Run
    ContentDigestAlgorithm: sha256

  Artifacts:
    MaxPerRun: 50             # Maximum artifacts per Run
    MaxSizeBytes: 10485760    # 10 MB max artifact size

Error Handling

See Also

• AdvisoryAI Architecture
• Chat Interface
• AI Attestations
• Evidence Locker
• Attestor Module

Last updated: 10-Jan-2026