Advisory AI Orchestration Pipeline (Planning Notes)

Status: In progress – orchestration metadata and cache-key wiring underway for AIAI-31-004.
Audience: Advisory AI guild, WebService/Worker guilds, CLI guild, Docs/QA support teams.

1. Goal

Wire the deterministic pipeline (Summary / Conflict / Remediation flows) into the Advisory AI service, workers, and CLI with deterministic caching, prompt preparation, and guardrail fallback. This document captures the pre-integration checklist and task breakdown so each guild understands their responsibilities before coding begins.

2. Prerequisites

Area	Requirement	Owner	Status
Toolset	Deterministic comparators, dependency analyzer (`IDeterministicToolset`, `AdvisoryPipelineOrchestrator`)	Advisory AI	✅ landed (AIAI-31-003)
SBOM context	Real SBOM context client delivering timelines + dependency paths	SBOM Service Guild	✅ typed client and DI helper ready; supply host BaseAddress at integration time
Prompt artifacts	Liquid/Handlebars prompt templates for summary/conflict/remediation	Advisory AI Docs Guild	⏳ authoring needed
Cache strategy	Decision on DSSE or hash-only cache entries, TTLs, and eviction policy	Advisory AI + Platform	⏳ hash-only plan keys implemented; persistence decision outstanding
Auth scopes	Confirm service account scopes for new API endpoints/worker-to-service calls	Authority Guild	🔲 define

Blocking risk: SBOM client and prompt templates must exist (even stubbed) before the orchestrator can produce stable plans.

3. Integration plan (high-level)

Service layer (WebService / Worker)
- Inject IAdvisoryPipelineOrchestrator via AddAdvisoryPipeline.
- Define REST endpoint POST /v1/advisories/{key}/pipeline/{task} (task ∈ summary/conflict/remediation).
- Worker consumes queue messages (advisory.pipeline.execute) -> fetches plan -> executes prompt -> persists output & provenance.
- Add metrics: advisory_pipeline_requests_total, advisory_pipeline_plan_cache_hits_total, advisory_pipeline_latency_seconds.
CLI
- New command stella advise run <task> with flags for artifact id, profile, policy version, --force-refresh.
- Render JSON/Markdown outputs; handle caching hints (print cache key, status).
Caching / storage
- Choose storage (Mongo collection vs existing DSSE output store).
- Persist AdvisoryTaskPlan metadata + generated output keyed by cache key + policy version.
- Expose TTL/force-refresh semantics.
Docs & QA
- Publish API spec (docs/modules/advisory-ai/guides/api.md) + CLI docs.
- Add golden outputs for deterministic runs; property tests for cache key stability (unit coverage landed for cache hashing + option clamps).

4. Task Breakdown

AIAI-31-004A (Service orchestration wiring)

Scope: WebService/Worker injection, REST/queue plumbing, metrics counters, basic cache stub.
Dependencies: AddAdvisoryPipeline, SBOM client stub.
Exit: API responds with plan metadata + queue message; worker logs execution attempt; metrics emitted.

AIAI-31-004B (Prompt assembly & cache persistence)

Scope: Implement prompt assembler, connect to guardrails, persist cache entries w/ DSSE metadata.
Dependencies: Prompt templates, cache storage decision, guardrail interface.
Exit: Deterministic outputs stored; force-refresh honoured; tests cover prompt assembly + caching.

2025-11-03: Prompt assembler now emits deterministic JSON payloads, guardrail pipeline wiring is stubbed for upcoming security hardening, and outputs persist with DSSE-ready provenance metadata plus golden test coverage.

AIAI-31-004C (CLI integration & docs)

Scope: CLI command + output renderer, docs updates, CLI tests (golden outputs).
Dependencies: Service endpoints stable, caching semantics documented.
Exit: CLI command produces deterministic output, docs updated, smoke tests recorded.

AIAI-31-006 (Service API surface)

Scope: Expose REST endpoints for summary/conflict/remediation execution plus cached output retrieval (POST /api/v1/advisory/{task}, GET /api/v1/advisory/outputs/{cacheKey}). Include guardrail execution, provenance hashing, metrics, and stubs for RBAC/rate limits.
Dependencies: Guardrail enforcement (AIAI-31-005), Authority scope wiring (advisory-ai:view / advisory-ai:operate), Offline kit docs.
Exit: Endpoints return sanitized prompts with citations, guardrail metadata, DSSE hash, and plan cache indicators; OpenAPI description updated; rate-limit hooks ready for Authority integration.

2025-11-03: Initial REST surface shipped – direct execution runs through guardrail pipeline, outputs persist with DSSE-ready provenance, metrics advisory_outputs_stored/advisory_guardrail_blocks emit, and cache retrieval endpoint exposes stored artefacts (RBAC/header enforcement pending scope delivery).

Supporting tasks (other guilds)

AUTH-AIAI-31-004 – Update scopes and DSSE policy (Authority guild).
DOCS-AIAI-31-003 – Publish API documentation, CLI guide updates (Docs guild).
QA-AIAI-31-004 – Golden/properties/perf suite for pipeline (QA guild).

5. Acceptance checklist (per task)

Item	Notes
Cache key stability	`AdvisoryPipelineOrchestrator` hash must remain stable under re-run of identical inputs.
Metrics & logging	Request id, cache key, task type, profile, latency; guardrail results logged without sensitive prompt data.
Offline readiness	All prompt templates bundled with Offline Kit; CLI works in air-gapped mode with cached data.
Policy awareness	Plans encode policy version used; outputs reference policy digest for audit.
Testing	Unit tests (plan generation, cache keys, DI), integration (service endpoint, worker, CLI), deterministic golden outputs.

6. Next steps

Finalize SBOM context client (AIAI-31-002) and prompt templates.
Create queue schema spec (docs/modules/advisory-ai/queue-contracts.md) if not already available.
Schedule cross-guild kickoff to agree on cache store & DSSE policy.

7. Recent updates

2025-11-04 — Orchestrator metadata now captures SBOM environment flags, blast-radius metrics, and dependency analysis details; cache-key normalization covers ordering.
2025-11-04 — Unit tests added for SBOM-absent requests, option-limit enforcement, and cache-key stability.
2025-11-04 — AddSbomContext DI helper enforces tenant header + base address wiring for downstream hosts.

Last updated: 2025-11-04