Notify routing — current native repoint and historical NTF-5 proposal

Current status (2026-08-31): RAR-6 deployed Console32907 and verified ordinary browser reads through the published native channels/override roots. RAR-7 owns the bounded atomic three-alias retirement and its route-test/live-POST gates; this is not a Notify host cutover. NTF-9’s database move is recorded DONE in SPRINT_20260722_015; it did not install /api/notify/v1. Do not execute the historical swap procedure below against the current table. Historical manifest: tools/scripts/ntf5-notify-route-swap.json; its executable route data remains unchanged for existing conformance consumers, with current-state notes alongside it. Historical script: tools/scripts/ntf5-swap-notify-routes.ps1; not the RAR-6 procedure. Guard: src/Router/__Tests/StellaOps.Gateway.WebService.Tests/Configuration/NotifyConsolidatedRouteConfigTests.cs

Update 2026-09-04 — the historical swap is retired by execution, not only by scope. RAR-5/7 (81e276c2a3, 83bfc9359f, f4b1774cf6) removed all eleven rows the manifest replaces; both route tables now hold 22 routes and Notify is served by the auto-published layer. Since ced8a2d286 pwsh tools/scripts/ntf5-swap-notify-routes.ps1 refuses on both invocations — -Check and a bare run each report the swap as RETIRED and exit 2, measured 2026-09-04 at HEAD. A static entry out-ranks publication, so reinstating the eleven rows by some other route would be a regression, not a no-op. Do instead: confirm the auto-published native Notify surfaces answer through the gateway.

Current native-consumer verification boundary

The RAR-6 deployment receipt records the served artifact and browser proof. RAR-7’s subsequent strict-TLS PRE verifies native and alias reads plus anonymous401 and insufficient-scope403. Live retirement acceptance still requires its own POST and the new Console must remain mounted in route recovery.

The native override API supports suppression-bypass overrides, not the Console’s admin model. It requires an actor, reason and positive duration for creation; revocation is POST /{id}/revoke. The existing client PUT/DELETE methods and bodies remain unchanged, so their contract gap remains. /api/v2/notify/overrides is not an alternative: it explicitly returns 501 notify_overrides_not_supported. Template-update and delivery-pagination mismatches also remain separate work, not implied fixes from the root change.

Backend route baseline: c62a2aacfa453345d80802f0c35a4df79e37405e, in NotifyApiEndpoints.cs, NotifyAdminCompatEndpoints.cs and OperatorOverrideEndpoints.cs. Re-verify before any deployment or retirement:

1. Read GET /api/openapi/aggregate; confirm Notify owns the exact native paths and methods. Do not infer an endpoint from its prefix or from the historical proposed namespace.
2. Run the paired Console registry/client specs and bash tools/scripts/validate/check-console-notify-prefix-literals.sh --self-test, then the guard without --self-test. The guard is a source check, not a browser-deployment receipt.
3. After an authorized Console deployment, verify the served bundle uses both native roots and authenticate through the gateway for a main collection read and an operator-override read. Confirm owner attribution and response shape; an anonymous 401 alone does not prove the data path.
4. Retire the specific delivery/override and broad notifier aliases atomically under RAR-7 after the consumer proof and route gates. Retiring a specific row alone leaves it captured by the broad alias. Singular delivery has no Console/CLI caller; notifications was already retired. Require registry/conformance and live native/denial/retired-path POST evidence.

The native-consumer/alias retirement does not run the historical swap script, add a consolidated gateway group, change Notify runtime flags or activate the successor worker.

Historical NTF-5 proposal (2026-08-05; not execution guidance)

All route indexes, counts, present-tense observations and commands below describe the original proposal. Published-native routing and RAR-5 retirement supersede its wildcard assumptions. The table is retained as design history, not as a claim about the currently deployed table.

What the proposed swap would do

Eleven gateway routes across seven unrelated external namespaces collapse into one group:

^/api/notify/v1(.*)  ->  http://notify.stella-ops.local/api/notify/v1$1   (Microservice, regex)

Plus a wildcard dependency with no route of its own: the entire /api/v1/notify/** API group (rules, channels, templates, deliveries, delivery/stats, digests, audit, locks, anomaly-subscriptions) reaches notify-web only through ^/api/v1/([^/]+)(.*), which derives the hostname from the first path segment. It works purely because the deployable is named notify.

Historical pre-window analysis

1. The group must be inserted ABOVE index 199, never appended

/api at index 199 is a non-regex ReverseProxy prefix to platform-web that captures every unmatched /api/* path — including the whole consolidated namespace. A grep for api/notify/v1 finds nothing and reads as “the namespace is free”; that is true and irrelevant. Appending the group hands the entire consolidated Notify plane to platform-web, which answers for paths it does not serve rather than failing visibly.

The script inserts at the first replaced route and re-parses its own output to prove the ordering, refusing to write otherwise. ConsolidatedGroup_MustOutrankThePlatformApiCatchAll pins it independently.

2. Route 8 has no production successor — and that is not a loss

^/api/v1/notify/(digest-schedules|quiet-hours|throttle-configs|simulate|escalation-policies|localizations|incidents) targets platform-web, whose handler (NotifyCompatibilityEndpoints) is registered only inside if (app.Environment.IsDevelopment() || testingEnvironment) — and PlatformProductionCompatibilityGuardTests asserts production exposes no /api/v1/notify route. In production these seven Console-facing paths already answer nothing. In Development they return fabricated constants (scheduleId: "digest-daily").

Per-name disposition (full evidence in the manifest):

Expect operators in a Development stack to report “the digest schedule list went empty” after the swap. That is the stub disappearing, not a regression.

3. Removing the routes does not fail cleanly

Three failure shapes, none a clean 404:

• Wrong-service 503 — eight routes fall to ^/api/v1/([^/]+)(.*) or ^/api/v2/([^/]+)(.*), which substitute the first path segment as a hostname (events, assurance, regulatory, notifications, observability, qa). None is a registered service.
• Right-service wrong-path 404 — the three /api/v1/notifier/* routes fall through to notifier.stella-ops.local, and that alias is on the notify-web container itself (docker-compose.stella-services.yml, notify-web: block, aliases list, commented # merged from notifier-web) — not on notifier-worker. So the request reaches the correct service at a path it has never mapped. This is the most misleading shape in the set: it reads as “the endpoint was removed” rather than “the route was removed”, and diagnosing by which container answered gives the right answer and teaches nothing. It does not improve when NTF-8 retires notifier-worker— that container does not own the alias. Deleting the alias line is separate NTF-8/NTF-10 cleanup; until then a retired external name keeps resolving to the live service.
• Silent service swap — removing route 8 moves seven paths from platform-web to notify-web. Both answer; the answering service changes.

Operator rule: discriminate on the RESPONDING SERVICE, not on status. Every failure here is a 4xx/5xx from a plausible-looking host.

4. TWO existing test classes go red BY DESIGN — and only 5 of the 11 routes were watched

Both are green today (measured: 30/30 across the two classes). They fail once the swap is applied, which is the guard working. Update them in the same window — do not revert the swap.

Delete this family’s rows rather than repointing them — NotifyConsolidatedRouteConfigTests already carries the equivalents for the consolidated group.

GatewayRouteSearchMappingsTests is shared with three other programs. Its RequiredMappings table also pins routes owned by FND-5 (^/api/v2/security), AAI-5 (^/api/v1/search, ^/api/v1/advisory-ai, ^/v1/evidence-packs, ^/v1/advisory-ai) and JOB-5 (^/api/v1/jobengine/jobs, ^/api/v1/jobengine/dag, ^/scheduler, ^/api/scheduler). Whichever window lands first turns this one file red for every program that has not yet run. Edit only this family’s rows, and expect merge contention here if two windows land close together.

Six of the eleven routes had NO watcher at all before this stage: /api/v1/assurance/reporting-timeline-profiles, /api/v1/regulatory/, the platform-fronted subset, /api/v1/notifications, /api/v1/observability and /api/v1/qa/fixtures. For those, NotifyConsolidatedRouteConfigTests is the first watcher they have ever had — which is also why three of this manifest’s four S0 corrections were available to find. Do not read a small red set as “the swap is safe”; read it as “most of this was unwatched until now”.

Not watchers, despite mentioning notify paths: RouteTableWebApplicationFactory / RouteTableIntegrationTests and RegulatoryFeedRouteAuditHostedServiceTests all build synthetic in-memory route fixtures and never read either live config.

5. The event fan-in is NOT affected by this swap (but IS by NTF-8)

/api/v1/events/{producer} looks like the riskiest transport change in the estate — five in-estate producers post to it (Scanner, Concelier, Integrations, ReleaseOrchestrator, Platform). They bypass the gateway entirely: compose pins ReleaseOrchestrator__DeviationNotify__BaseAddress=http://notify.stella-ops.local and STELLAOPS_NOTIFY_URL/STELLAOPS_NOTIFIER_URL to the alias directly. The gateway route serves external callers only.

The corollary matters more than the reassurance: those base addresses are pinned to the alias, so the NTF-8 compose rename is what can break event fan-in — with nothing in the route table to show for it.

6. The compose config is GENERATED, and one overlay key is this window’s to delete

devops/compose/router-gateway-local.json is not authored — its first key says "DO NOT EDIT BY HAND. Regenerate via devops/docker/router-gateway-config-generate.sh from src/Router/StellaOps.Gateway.WebService/appsettings.json + devops/compose/router-gateway-overlay.local.json." Every swap script in this program family hand-edits it anyway, deliberately: the generator is currently red (below), so regeneration is not an available path, and the swap must leave both configs symmetric.

This family owns exactly one overlay key, verified against the whole RouteTranslateToOverrides map:

"^/api/v1/assurance/reporting-timeline-profiles(.*)":
    "http://notify.stella-ops.local:8080/api/v1/assurance/reporting-timeline-profiles$1"

That key is where the compose copy’s :8080 comes from. It is a declared override, not drift between two configs — an earlier version of this runbook and of the manifest called it skew, which was wrong and hid this obligation.

Delete it in the same change as the swap. The generator matches overrides to source routes by exact Path and, on any override matching nothing, prints ERROR: overlay TranslatesTo overrides did not match any source route and exits 2 (router-gateway-config-generate.sh:88-90) — so a leftover key blocks regeneration of every gateway config, estate-wide, not just Notify’s. The consolidated group needs no replacement override: it targets the alias on port 80 like every other group route.

NotifyConsolidatedRouteConfigTests.Overlay_MustNotOverrideARouteTheSwapRemoved asserts the key is present pre-swap and absent post-swap. Red control verified by dropping the route from the manifest: the guard fails with the offending path named.

Hand-editing is not a shortcut — it pre-applies what the generator would emit. The GRA-5 peer measured this rather than arguing it: running the generator’s own body against post-swap sources produces 202 routes, 201 of 202 identical to the hand-edited file, and the consolidated group reproduced byte-identically. That holds because the swap edits the source (appsettings.json) too, not only the generated copy. The overlay deletion above is what keeps that true — it is the one piece a later regeneration would otherwise disagree with.

One route is genuinely not reproducible from its sources — and regenerating it is a TLS DOWNGRADE. Refuse the flip; do not accept it as “the generator agreeing with source”.

^/catalog-changes/(tenants|tenant_lifecycle)(.*) is http:// in appsettings.json and https://in the committed compose config, with zero catalog-changes keys in either override map. It is the only unexplained difference between the two configs — this family’s assurance route is the one other difference and the overlay explains it.

The https is the working value, so regeneration does not correct a stale entry, it downgrades a live Authority route to plain HTTP — and the route-test suites stay green, because they assert routing, not schemes. A window following “regenerate and diff, they should agree” would commit that downgrade during a maintenance window. (An earlier version of this runbook said “pre-existing, do not chase it”. That was wrong and would have waved the downgrade through. Direction traced by AAI-5, confirmed by GRA-5 in e89dbc344e, re-verified here.)

The repair is unambiguous and the estate already does it elsewhere. Verified across every authority-targeted route: ^/api/v1/trust(.*) and ^/api/v1/tenants(.*) are also http:// in source and carry a RouteTranslateToOverrides entry upgrading them to https://. ^/catalog-changes/… is simply missing that override. Add the matching entry — never edit the generated file. Not this sprint’s to land, but do not let a regeneration in this window ship the downgrade.

(Related but deliberately not escalated: ^/api/v1/claims(.*) is http:// in both files, so it is not a second divergence and nothing here breaks it. It is the one Authority route the deployed config still serves over plain HTTP. Recorded as a question, not asserted as a defect.)

The generator is already red on main— seven unmatched overrides on a clean tree (five ^/api/v1/federation/* plus ^/api/v1/vex(.*) and ^/api/vex(.*), all absent from appsettings.json). The vex pair is this exact defect one window earlier: B7 removed the routes without cleaning the overlay. Not this sprint’s to fix, but it means the window cannot fall back on “just regenerate”. (Re-deriving the count: a naive scan says eight, because _comment is a key in that map; the generator strips _-prefixed keys first.)

The overlay also contains duplicate keys — 68 raw entries, 52 unique, 16 duplicated (a release-orchestrator/jobengine block appears twice), counting real route keys only. (The GRA-5 runbook says 69/53/16 for the same file; the difference is _comment, counted there and excluded here — the same gotcha as seven-vs-eight, one layer down. Both are right for what they count.) Every duplicated pair currently holds an identical value, and none of them is a route any consolidation program removes, so no window has to delete two copies today. It stops being harmless the moment someone edits one copy: Python’s json.load — what the generator uses — keeps the last silently.

Because of that, the guard counts occurrences rather than testing presence: exactly one pre-swap, exactly zero post-swap. A presence test cannot distinguish “one key” from “two keys with the same value”, so it stays green through a half-deletion while the generator is still red and the diff reads as complete. Red-controlled on exactly that case — the override duplicated with an identical value — and it fails naming the count and the last-wins behaviour. Improvement adopted from GRA-5.

7. Four routes change transport

Routes 4–7 are ReverseProxy today and become Microservice under the group: a direct HTTP proxy to the DNS alias becomes a Valkey router-transport dispatch. Three of them also name an explicit :8080 that the group drops (the alias resolves on 80).

The unproven case is a large buffered response — a non-streaming handler’s body returns as a single FrameType.Response where a ReverseProxy route would have streamed it. The forcing function below therefore includes a realistically sized delivery page.

Historical window procedure — do not execute

Re-derive a future host cutover from current source, published endpoints and explicit deployment scope. The following commands record the old proposal only; they are not the RAR-6 workflow.

# 0. Confirm the manifest still matches the live route table. Exit 1 = do not proceed.
pwsh tools/scripts/ntf5-swap-notify-routes.ps1 -Check

# 1. Confirm the consolidated host actually serves /api/notify/v1 (NTF-7's deliverable).
#    Half of it is config: notify:api:basePath = /api/notify/v1 re-homes the main API
#    group (rules/channels/templates/deliveries/digests/audit/locks) with no code change.
#    The other groups hardcode their prefixes and need the NTF-7 edits.

# 2. Apply. RETIRED 2026-09-04: a bare run refuses and exits 2, writing nothing.
# pwsh tools/scripts/ntf5-swap-notify-routes.ps1

# 2a. MANDATORY, same change: delete this family's ONE overlay override key from
#     devops/compose/router-gateway-overlay.local.json —
#       "^/api/v1/assurance/reporting-timeline-profiles(.*)"
#     Leaving it wedges the config generator FOR THE WHOLE ESTATE (see section 6).
#     Do NOT add a replacement key for the consolidated group; it does not need one.

# 3. Recreate the gateway so the deployed config reloads (a same-tag image will NOT
#    swap without --force-recreate). The compose key is `router-gateway`, and it is
#    the container that bind-mounts devops/compose/router-gateway-local.json as
#    /app/appsettings.json — which is why that file, not just the service default,
#    is in the manifest's configPaths.
docker compose … up -d --force-recreate --no-deps router-gateway

# 4. Update NotifyEventsRouteConfigTests (item 4 above) and re-run the gateway suite.

Historical proposed forcing function

Not “the container is up”. All four must pass through the gateway:

1. Authorized read — GET /api/notify/v1/channels with a notify:viewer token returns the tenant’s channels. Proves the wildcard-served admin group survived the move onto an explicit route.
2. Unauthorized read — the same call without the scope returns 403, not 200 and not a platform-web response body. Proves the group did not land below /api.
3. Large buffered response — GET /api/notify/v1/deliveries on a tenant with a realistically sized delivery history. This is the transport change’s only unproven case; the answer is not readable off the config.
4. End-to-end delivery — an event produces a delivered notification through the merged worker (the sprint’s own NTF-9 forcing function). Post it direct to the alias, the way the real producers do, not through the gateway. The exact row must carry consumer.role=notify-worker; a count delta without that durable attribution can have been produced by the rollback predecessor. Follow notify-ntf9-delivery-forcing-function.md.

Related

• Manifest: tools/scripts/ntf5-notify-route-swap.json
• Console prefix guard: tools/scripts/validate/check-console-notify-prefix-literals.sh
• Console registry: src/Web/StellaOps.Web/src/app/core/api/api-path-prefixes.ts (Notify family block)
• Design: docs/modules/notify/consolidation-design.md
• Sibling runbooks: findings-cutover-fnd5-route-swap.md, jobengine-cutover-job5-route-swap.md, advisoryai-cutover-aai5-route-swap.md