Runbook: Findings - Graph asset registry source unavailable

Use this when a Findings host refuses startup, or stops after startup, with GraphAssetRegistryEventSourceUnavailableException.

Two topologies raise this exception, and they have different sources but identical honesty semantics (D-FND-GRA-1: source absence is never an empty batch):

1. Predecessor (findings-ledger-web, live until FND-10): the historical same-database SELECT FROM graph.asset_registry_events on the shared platform database. Diagnose with the same-database procedure below.
2. Consolidated (findings-web, the 010x023 seam): stellaops_findings contains no graph schema BY DESIGN. When the seam is armed (Findings:GraphAssetRegistry:Enabled=true), the host reads the Graph-owned log through graph-api’s asset-registry event feed (GET /graph/asset-registry/events, admitted only by the machine scope graph:asset-registry:read-all) and refuses typed on any contract-grade failure. Diagnose with the feed-seam procedure below. When the seam is DISABLED (the shipped default), the whole asset-registry projection plane is dormant by a recorded deployment gate: the host boots healthy and this exception cannot occur.

Source admission failure

Findings performs a zero-row SELECT against the Graph-owned relation before its projection worker starts. Any PostgreSQL failure during that admission means startup cannot prove the source and is reported through this runbook. After startup, missing schema/table or lost SELECT privilege is a contract failure, not proof that Graph has no new events; it fails the worker instead of returning an empty batch. Ordinary runtime connection/timeout faults keep bounded retry/backoff and are not mislabelled as relation loss.

The predecessor topology deliberately keeps the historical same-database read working until FND-10. This guard is not the cross-database successor and does not authorize a foreign-data wrapper, database link, cross-database grant, all-tenant bypass token, or blank replacement table — the sanctioned successor is the feed seam below.

Safety rules

1. Leave the affected Findings role stopped/failing while the source contract is unproven.
2. Do not advance or delete findings.asset_registry_projection_offsets.
3. Do not create an empty graph.asset_registry_events relation to make admission green.
4. Do not repoint Graph or Findings, change grants, or run a migration outside the approved owner window. The checks below are read-only; restoration follows the normal Graph/database change process.

Diagnose with the exact Findings service role

Run these read-only queries through the approved PostgreSQL client using the Findings Ledger DSN and role. Do not print the connection string into the incident record.

SELECT current_database() AS database_name,
       current_user AS role_name,
       to_regclass('graph.asset_registry_events') AS source_relation;

SELECT has_schema_privilege(current_user, 'graph', 'USAGE') AS graph_schema_usage,
       has_table_privilege(current_user, 'graph.asset_registry_events', 'SELECT') AS source_select;

SELECT column_name, data_type
FROM information_schema.columns
WHERE table_schema = 'graph'
  AND table_name = 'asset_registry_events'
ORDER BY ordinal_position;

Expected source columns include event_id, tenant_id, asset_id, asset_type, event_type, payload_json, the four hash fields, occurred_at, and recorded_at. If to_regclass is null, or the privilege checks are not true, admission must remain red.

Capture the last durable consumer position without changing it:

SELECT worker_id, last_graph_recorded_at, last_graph_event_id, updated_at
FROM findings.asset_registry_projection_offsets
WHERE worker_id = 'default';

default is the production value of AssetRegistryLedgerConstants.DefaultProjectionWorkerId.

Interpret the failure

Recover the current same-database path

1. Have the Graph/database owner restore the real producer relation, approved same-database grant, or connection reachability. Use forward-only migration recovery; do not edit an applied migration or seed an empty event log.
2. Re-run all read-only admission queries above with the exact Findings role. Continue only when the relation exists, both privileges are true, and the expected columns are visible.
3. Restart the affected Findings role using the site’s normal deployment procedure.
4. Verify the startup log has no source-unavailable exception and record the checkpoint.
5. Cause or identify one approved Graph asset event, then verify the Findings checkpoint advances in (recorded_at, event_id) order and the corresponding findings.asset_registry_events row preserves the Graph hashes. A successful restart alone is not catch-up proof.

Feed-seam failure (consolidated findings-web)

The armed seam proves BOTH halves of the contract at admission, without moving a checkpoint:

• Local seam state: findings.asset_registry_feed_state (migration 008) must exist. Its absence means the consolidated schema has not converged and is a typed refusal, never “no state yet”. The durable consumer position stays in findings.asset_registry_projection_offsets (worker id default), exactly as before.
• Remote feed contract: graph-api must answer GET /graph/asset-registry/events authoritatively for the seam’s signed identity envelope (subject stellaops-findings-graph-projection, sole scope graph:asset-registry:read-all, HMAC over the shared Router:IdentityEnvelopeSigningKey / STELLAOPS_IDENTITY_ENVELOPE_SIGNING_KEY).

Arming keys (deployment-supplied; the repository ships no values, and enabled-with-missing refuses startup naming the exact key): Findings:GraphAssetRegistry:Enabled, Findings:GraphAssetRegistry:FeedBaseUrl, plus the shared identity-envelope signing key on the findings-web host.

Interpret a seam refusal

The exception message names the reason. Each row is a contract failure that fails the host closed; ordinary transport transients at runtime keep the worker’s bounded retry instead.

The safety rules above apply unchanged: do not hand-create relations, do not advance or delete the offsets row outside the epoch-recovery procedure, and no cross-database grants.

GRA-9 split decision

The Graph-owned successor from path 1 below now EXISTS IN SOURCE: the tenant-carrying, (recorded_at, event_id)-ordered, byte-bounded catch-up feed with origin-derived stream-epoch handling, the dedicated service-only scope, and the Findings consumer with durable checkpoint/epoch state — proven end-to-end on disposable PostgreSQL, including a production- composition findings-web boot against a database with no graph schema (FindingsWebGraphlessProductionBootTests). What the window must still prove LIVE before revoking the same-database relation/grant: the deployed graph-api serves the feed route (Router HELLO publication included), the armed findings-web admission passes against it, and one Graph event advances the checkpoint end to end.

1. The Graph-owned, tenant-scoped event/API successor is deployed and proves bounded replay, stable ordering, stream epoch/reset handling, retention horizon, authenticated service authorization, consumer checkpoint/catch-up, and Router route publication; or
2. Product and Security explicitly accept a fail-closed Findings freeze for the window, including customer impact, rollback trigger, and the checkpoint evidence to resume safely.

An all-tenant bypass scope is not an implicit fallback: per the Q-23 precedent the feed’s dedicated machine scope is granted to NO interactive client or persona (descriptor tests pin the universal forbid), and consumers partition admitted events by the exact tenant inside each event. The current /graph/assets state API remains no replacement because it does not preserve event order, history, reset, or retention semantics.

Rollback and evidence

If recovery does not make one event advance end to end, stop Findings again and restore the last approved same-database topology/grants or the pre-window database snapshot. Preserve:

• the exact exception and SQLSTATE;
• database/role/relation/privilege query output;
• checkpoint before and after recovery;
• the Graph event id plus source and projected hashes;
• restart/build identity and the approving window or incident record.

Related references:

• docs/modules/findings-ledger/schema.md
• docs/modules/findings/consolidation-design.md
• docs/modules/graph/consolidation-design.md
• docs/runbooks/database/migration-recovery.md
• docs/runbooks/graph/graph-cutover-gra5-route-swap.md