AdvisoryAI AAI-9 maintenance window — execution plan

Prepared 2026-09-04 by the live-deploy operator. This is a PLAN, not authorization to run it. The owner approved planning this window on 2026-09-04. Executing the data move needs a separate, explicit go.

Step authority is advisoryai-aai9-data-move.md— the 13 numbered steps, the copy contract, findings D1-D6 and the rollback/unfreeze conditions live there and are not duplicated here. This document adds what that runbook does not carry: a timed sequence, a preflight NO-GO list with exit codes, the point of no return, per-phase rollback with its artifact, the forcing functions, the operator roles, duration estimates with their basis, and the failure modes with detection and response.

Where this plan and the runbook disagree, this plan’s corrections win and are listed in §10 with the evidence; fold them into the runbook before the window opens.

1. Offline prep executed 2026-09-04 (no live state changed)

Every value below was measured today. Nothing was deployed, copied, revoked, retired or restarted.

1.1 Successor images — BUILT, clean, digest-pinned

Built from an isolated detached worktree at HEAD (tmp/worktrees/aai9-build-9d2e5ef, git status --porcelain = 0 lines) with MSBUILDDISABLENODEREUSE=1 DOTNET_BUILD_SERVER=0 PUBLISH_MAX_CPU_COUNT=2, invoking the worktree’s own relative bash devops/docker/build-service-publish.sh. Confirmed from the build log that every publish path sits under the worktree. Not pushed to any registry — building is not publishing.

Acceptance check is gitSha == the commit you intended to pin, not worktreeState == clean— a mis-invoked build against the main clone produces an honest clean stamp of the wrong commit. Both images stamp 9d2e5ef5, the HEAD at build time.

These digests expire. HEAD moved twice during this prep session (ebc9ff5a → 9d2e5ef5) because roughly seven lanes are landing work. If HEAD has moved past 9d2e5ef5 when the window opens, and any moved commit touches src/AdvisoryAI/**, src/Router/**, src/OpsMemory/** or devops/compose/docker-compose.advisoryai.yml, rebuild and re-record — do not deploy a stale digest against a newer runbook. Verify with git diff --name-only 9d2e5ef54193b01881793dcef35d525e0c3734f0..HEAD -- src/AdvisoryAI src/Router src/OpsMemory devops/compose/docker-compose.advisoryai.yml.

1.2 Rollback artifacts — all three exist

The research lane’s report that “the web and OpsMemory :dev tags are ABSENT” is correct but is not a gap: those two predecessors were never on :dev. Each of the three has both a tag and an ID, and each container’s compose chain already ends in a pin naming the image ID, so rollback does not depend on any tag surviving.

All three are Up (healthy), uptime 28 h at measurement.

The compose chains are heterogeneous and end in ID pins (read from com.docker.compose.project.config_files, not reconstructed):

• advisory-ai-web — 18 files, last pin tmp/kestrel-rotation/20260822T091500Z/overlays/phase2-new-cert-advisory-ai-web.pin.yml → image: "sha256:7433b4ac…".
• advisory-ai-worker — 15 files, last pin …phase2-new-cert-advisory-ai-worker.pin.yml.
• opsmemory-web — 10 files, last pin …phase2-new-cert-opsmemory-web.pin.yml.

All 10/15/18 files were verified present on disk today.

Rollback fragility — act on this before the window. Five of those chain files live under tmp/ (tmp/build/bl2-stage0/*.override.yml, tmp/kestrel-rotation/**). tmp/build/ is swept under LOCATIONS.md, and the pins that name the rollback image IDs are the last entries in every chain. A sweep between now and the window silently destroys the rollback recipe while leaving the images intact. Copy all three chains and every file they name into ${AAI9_ROOT}/rollback/ at step 2 and assert the copies, not the originals.

1.3 Source re-measurement at the boundary — unchanged from 2026-08-28

Per-table non-zero counts (all five equal the copy contract): api_operation 2190, api_spec 1, doctor_search_projection 186, kb_chunk 10154, kb_doc 707. The three excluded ledger rows are advisoryai.schema_migrations, advisoryai.__migration_history, opsmemory.schema_migrations, one row each.

Storage split, which is what sets the restore duration: kb_chunk is 37 MB heap + 88 MB indexes + ~98 MB TOAST = 223 MB. The restore rebuilds those 88 MB of indexes (13 on kb_chunk, including five per-language GIN tsvectors) from 10,154 rows.

Canonical source fingerprint captured today (to_jsonb ordered digest, the step-5 comparator), first 16 hex of each:

advisoryai.api_operation             2190   781a37ddfc060196
advisoryai.api_spec                     1   41edc1d28029ed3c
advisoryai.doctor_search_projection   186   00d7f1f204b2752f
advisoryai.kb_chunk                 10154   baa4e13111cf39f0
advisoryai.kb_doc                     707   cd4806d2204d7b64

These digests are NOT a window baseline. See §1.6 — a live writer rewrites kb_doc/kb_chunk on a timer. parity/source-a.tsv must be taken inside the freeze.

1.4 Target database and isolation

• stellaops_advisoryai exists, owner advisoryai; 0 relations, extensions = plpgsql only.
• Role advisoryai: LOGIN, and NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS all confirmed false in pg_roles.
• datacl on the target is {=T/advisoryai,advisoryai=CTc/advisoryai} — PUBLIC holds TEMP only, no CONNECT. Same shape as stellaops_platform and stellaops_findings.
• probe-database-isolation.sh: 18/18 rows correct for advisoryai— CONNECTED to stellaops_advisoryai and refused on all 17 siblings. Whole-matrix verdict == PASS: every role reaches exactly its own database ==.

But the bare invocation the runbook’s step 3 uses exits 1 today — see NO-GO item P-04 and defect D-3.

1.5 Carried volumes

Measured against the real volume names, which are compose_-prefixed:

Nothing is queued, so the swap carries no in-flight work. Ownership: the three carried volumes are 10001:10001 drwx------, matching the runtime uid (stella, 10001) in both predecessor and successor. compose_advisoryai-plugin-scratch is 0:0 drwxr-xr-xand advisoryai-data-init does not chown it — see risk W-02.

1.6 The source is NOT static — per-source ingest baseline

kb_doc.max(indexed_at) is 2026-09-04 13:37:56Z, minutes before this measurement. The live predecessor runs UnifiedSearchIndexRefreshService on a timer:

Unified search incremental indexing completed: domains=8, seen_chunks=20, changed_chunks=0, removed=0
Unified search periodic indexing run completed: domains=8, chunks=20, duration_ms=149

Baseline by document source, so window damage is separable from pre-existing noise:

Sibling tables: api_spec 1, api_operation 2190, doctor_search_projection 186.

Pre-existing error noise to baseline, so it is not misread as window damage. The predecessor’s VEX adapter fails its Authority token fetch on every unified-index pass and falls back to the snapshot:

StellaOps.Auth.Client.StellaOpsDiscoveryCache.GetAsync … EnsureSuccessStatusCode()
  at StellaOps.AdvisoryAI.UnifiedSearch.Adapters.VexSearchAdapter.FetchFromServiceAsync
info: Unified search refresh domain 'vex' completed: seen_chunks=3, changed_chunks=0, removed=0

This is pre-existing and unrelated to AAI-9. It must appear in the pre-window log snapshot, or the step-12 soak will attribute it to the move.

Two window consequences, both live findings:

1. The live predecessor sets three unified-search adapter base URLs; the successor overlay sets none. Live: ADVISORYAI__KnowledgeSearch__{Findings,Vex,Policy}AdapterBaseUrl = http://scanner.stella-ops.local, http://concelier.stella-ops.local, http://policy-engine.stella-ops.local. grep -n 'AdapterBaseUrl' devops/compose/docker-compose.advisoryai.yml → no match. After the cutover the eight unified domains (20 docs / 20 chunks) stop refreshing from the live services and serve snapshots indefinitely. Step 10’s flip of UnifiedAutoIndexOnStartup=true does not restore it, because UnifiedAutoIndexEnabled defaults false and the early return at UnifiedSearchIndexRefreshService.cs:27-33 needs both conditions. The key shape also changed — the predecessor image binds KnowledgeSearch:* while the successor host strips prefix ADVISORYAI__ and binds section AdvisoryAI:KnowledgeSearch (KnowledgeSearchOptions.SectionName, WebService/Program.cs:58). Copying the predecessor’s literal variable names forward would bind nothing and fail silently. The successor form is ADVISORYAI__AdvisoryAI__KnowledgeSearch__FindingsAdapterBaseUrl. ⇒ Decision required before the window (see P-09): either accept the regression explicitly and record it, or add the three keys in the successor shape to step 10’s activation environment. Do not discover this during the soak.
2. The runbook’s own precision note is confirmed live. The fourth carried-table writer is inert on the overlay only because no adapter URL is set — not because UnifiedAutoIndexOnStartup=false. The step-3 render check would stay green with an adapter URL present. tables_checked=27 rows=0 is the only assertion that catches it.

2. Roles — who is needed, and for what

Six roles, taken from the runbook’s Audience line plus its approvals table. One person may hold several hats; what matters is that each responsibility has a named owner before the window opens.

Scripted vs human. Steps 2-9 are entirely scripted (copy-paste from the runbook, exit-code gated) — a human reads verdicts and stops on the first failure, but types nothing original. Human judgment is required at exactly four points: the GO decision after the preflight list; the parity verdict at 9b/9e (the ordinal diff is the one place a non-empty diff is the pass condition, and its membership must be read); the three forcing functions at step 12, which are judged not asserted; and the rollback/unfreeze call.

3. Timed sequence

Times are cumulative from T+0 = the GO after the preflight list. Basis for each duration is in §7.

Total: ~5 h 20 m from GO, ~6 h 05 m including preflight.

4. Preflight NO-GO list

Every row must hold before step 5 (the freeze). Each names the command and the exit code that means STOP. A row that cannot be proven is a NO-GO, not a judgement call.

5. Point of no return, and rollback per phase

The point of no return is step 8, the pg_restore into the target (T+01:18).

It is not the freeze. Everything up to and including step 7 is reversible by starting three containers you never removed.

Why step 8 and not step 10. Before it, the target is empty and provably so (tables_checked=27 rows=0), so “abandon the window” is docker start ×3 and nothing else — the target can be dropped and re-converged at any later date with no evidence lost. After it, the target holds a copy of production data whose provenance is a specific frozen instant of the source. A second attempt cannot reuse it: --single-transaction --exit-on-error makes one retry against a still-empty target safe, but a successful restore followed by a later rollback means the next attempt must re-converge a fresh empty target and re-freeze the source, because the first copy’s freeze window is gone. That is the step where cost becomes measured in re-running the whole window rather than in restarting containers.

Step 8 is also the last step whose failure is cheap: it either commits or leaves nothing.

The freeze is not lifted by entering rollback. Only the five UNFREEZE conditions in the runbook’s final section lift it, all five at once.

6. The three forcing functions

Each is a live behavioural proof against the migrated data, not a synthetic fixture and not a health check. All three must pass before step 13.

Supporting checks that are not forcing functions (they can be green while the move is broken): health/ready 200, an unauthenticated 401/403, container healthy, and the compose render. The runbook’s own precedent is explicit that Up (healthy) is not evidence of a working service.

7. Duration estimates and their basis

Flagged as unmeasured and load-bearing: the converge boot (P2), the restore (P5), the FE dist rebuild (P9), and every forcing function (P10). The two soaks are policy choices. If the window must be time-boxed, measure the converge boot and the FE build in a rehearsal first — both are safe to rehearse without touching the source.

8. Gate 7 and the shared Console file

src/Web/StellaOps.Web/src/app/core/api/api-path-prefixes.ts is clean in the primary tree right now. Two windows edit it.

Anchor gate 7 by symbol, never by line. Line numbers in this file have been found stale three separate times today. EVD-6 (28546a50fc, merged 13f504b236) grew the Evidence-block comment and moved the whole AdvisoryAI block down: the block header sits at :305 and the six values at :346-351 as of 16ba2dfbb9, and they will move again. Locate them with grep -n 'advisoryAiSearch:\|advisoryAiApi:\|advisoryAi:\|advisoryAiRuns:\|evidencePacks:\|opsMemory:' > src/Web/StellaOps.Web/src/app/core/api/api-path-prefixes.ts. A window step that says “edit line N” is a trap.

AAI-9 owns exactly six values, all in the AdvisoryAI block (locate by symbol):

advisoryAiSearch: '/v1/search'          ->  '/api/advisoryai/v1/search'
advisoryAiApi:    '/v1/advisory-ai'     ->  '/api/advisoryai/v1/advisory-ai'
advisoryAi:       '/v1/advisory-ai'     ->  '/api/advisoryai/v1/advisory-ai'
advisoryAiRuns:   '/v1/advisory-ai/runs'->  '/api/advisoryai/v1/advisory-ai/runs'
evidencePacks:    '/v1/evidence-packs'  ->  '/api/advisoryai/v1/evidence-packs'
opsMemory:        '/api/v1/opsmemory'   ->  '/api/advisoryai/v1/opsmemory'

evidencePacks is AAI-9’s, and the reason is the verb set, not the name

The file’s own guard comment (“evidencePacks used to live here. It is NOT an Evidence-family…”) says where it belongs. EVD-6 re-opened the question — should a Console call through evidencePacks move to the consolidated Evidence host? — and the answer is no. Recorded here so nobody re-opens it mid-window. Verified at 16ba2dfbb9:

They are not one capability under two spellings. Repointing would silently drop /sign — a plausible-reading path that 404s.

No collision. git show 28546a50fc -- api-path-prefixes.ts has zero added or removed lines matching evidencePacks|advisoryAi|opsMemory — verified. EVD-6’s edits are in the Evidence block around :200-215, a disjoint region well above the six AAI-9 values.

Coordination rule: neither lane may edit the other’s region. Because the regions are disjoint, a rebase resolves cleanly — but only if AAI-9’s flip is a six-line edit and not a reformat of the surrounding comment block.

The wildcard-retirement premise test — opsMemory is already dead at HEAD

The research lane rewrote gate 7 once for a void premise. Re-tested here against the other static-route assumption, and it does not fully hold either.

^/api/v1/([^/]+)(.*) — the generic microservice wildcard — is in the retired-route registry (src/Router/__Tests/.../Configuration/retired-routes.json, 168 paths; ^/api/v2/([^/]+)(.*), ^/v1/advisory-ai(.*), ^/v1/evidence-packs(.*), ^/api/v1/search(.*) and ^/api/v1/advisory-ai(.*) are retired alongside it). The file’s own comment states that opsMemory: '/api/v1/opsmemory' “has NO explicit gateway route and never has: /api/v1/opsmemory/** reaches opsmemory-web only through ^/api/v1/([^/]+)(.*)”.

⇒ That path is dead at HEAD, today — not “goes dark when AAI-8 retires the container”. This is the direct explanation for P-13’s measurement: /api/v1/opsmemory returned 404 with TargetService=(none) through the live gateway today, while opsmemory was demonstrably replaying endpoint metadata to that same gateway.

Three consequences for gate 7:

1. For opsMemory the flip is a repair, not a flip between two working paths. The successor image maps ("/api/v1/opsmemory", "/api/advisoryai/v1/opsmemory") (AdvisoryAiConsolidatedRouteAliases.cs:28), so the window restores a surface that is currently unreachable. That strengthens the gate; it does not void it.
2. The P9 rollback row is wrong for this one value. “Revert the six values” restores a dead path for opsMemory. Reverting is still correct for the other five; for opsMemory the honest rollback statement is back to broken, as it is now — record it, do not present it as a restoration.
3. All six Console targets are covered by the successor’s alias map — verified at AdvisoryAiConsolidatedRouteAliases.cs:22-29: /v1/advisory-ai, /v1/evidence-packs, /v1/runs/, /v1/search, /api/v1/chat, /api/v1/opsmemory, all under ConsolidatedRoot = "/api/advisoryai/v1". advisoryAiRuns resolves through the /v1/advisory-ai prefix.

Do not fix the comment block from this window. It is false in three places (F-AAI9-16: the “AAI-5 gateway group”, the seven-route swap manifest, and “the gateway swap is atomic across all seven routes”). The values are correct; the prose is not. Fixing it here would collide with EVD-6b for no window benefit.

Sequencing is strict: gate 7 runs after step 10c, never before. The native /api/advisoryai/v1/** aliases exist only in the successor image (AdvisoryAiConsolidatedRouteAliases.cs, discovered by AspNetCoreEndpointDiscoveryProvider.DiscoverAspNetEndpoints). An early flip is a 404 for as long as the predecessor is serving — because the image is not running, not because any swap is atomic.

9. What could go wrong

10. Defects found in the runbook during this prep

All five were verified today. Fold them in before the window opens.

Two corrections to the briefing that commissioned this plan, both minor: evidencePacks is at api-path-prefixes.ts:350, not ~:299 (that line is a cross-reference comment in the Notify block); and rollback artifacts do exist for all three predecessors — the :dev tags are absent for web and OpsMemory, but both carry bl2-9796c887-20260724 tags and every chain pins by image ID.

11. Open NO-GO inputs, as of 2026-09-04

Seven preflight rows are unsatisfied today. Five are ordinary window paperwork; two are substantive and need work before the window can be scheduled:

• P-09 — the unified-search adapter-URL disposition. An owner decision, informed by §1.6.
• P-12 / P-13 — no working authenticated token, and the family’s published gateway paths do not resolve today. Every forcing function and gate 6’s proof depend on both. This is engineering work, not paperwork, and it must be settled before the window, not inside it.

Ordinary paperwork: P-01 (window approval and the six named roles), P-10 (the isolated restore test), P-11 (grants-forward.sql / grants-rollback.sql), P-14 (chain-file copies).