Runbook: Feed Connector - GitHub Security Advisories (GHSA) Failures

Purpose: recover the Concelier GHSA feed connector when GitHub advisory ingestion is failing or stale — typically GitHub API rate limiting, an expired token, malformed documents, or a stuck window cursor. Audience: Platform on-call engineers responding to a stale GitHub-ecosystem advisory feed.

Metadata

Reconciliation note (2026-05-31): This runbook was reconciled against src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/ and the CLI command tree in src/Cli/StellaOps.Cli/. Several commands in the original draft (stella doctor --check ..., stella connector ..., stella offline load, stella upgrade, stella vuln query) do not exist; they have been replaced with the real stella db fetch, stella sources, and stella admin feeds commands or flagged as NOT IMPLEMENTED below. The metric names and alert names in the original draft were not found in source; the connector emits the ghsa.* OpenTelemetry instruments documented under Telemetry.

A previous reconciliation pass referenced stella db connectors status/test/configure. Those subcommands exist only as unwired dead code (DbCommandGroup.BuildDbCommand is never added to the CLI root in CommandFactory.cs); the live stella db command (BuildDatabaseCommand) exposes only fetch, merge, and export. This pass replaces those references with the registered stella sources and stella admin feeds commands, and points connector configuration at the Console / the /api/v1/advisory-sources/{id}/configuration API / concelier:sources:ghsa config keys.

Symptoms

• [ ] GHSA feed sync failing or stale (source:ghsa:fetch job erroring or not advancing its cursor)
• [ ] GitHub returns 403 Forbidden / 429 Too Many Requests (“rate limit exceeded”)
• [ ] GitHub Advisory Database vulnerabilities missing
• [ ] ghsa.ratelimit.exhausted counter increasing, or ghsa.fetch.failures increasing
• [ ] ghsa.ratelimit.headroom_pct_current gauge near zero

Note: The GHSA connector fetches from the GitHub REST advisories API (GET advisories?..., media type application/vnd.github+json). It does not use the GitHub GraphQL API for ingestion — GraphQL is only used by the per-connector credential dry-run probe (the viewer query). Symptoms framed as “GraphQL query failed” in older runbooks do not apply to the fetch path.

Impact

Built-in resilience: When a GitHub REST list fetch is rate limited, the connector automatically attempts a public OSV GHSA fallback (downloading modified_id.csv and the matching GHSA-*.json documents from https://osv-vulnerabilities.storage.googleapis.com/) so the window can still be ingested. This is controlled by the useOsvFallback option (default true). The fallback is best-effort and caps documents per fetch via maxOsvFallbackDocumentsPerFetch (default 25).

Diagnosis

Quick checks

1. Show advisory source configuration/status (all sources):

   stella sources status
   # Optional: --json
   

2. Show GHSA feed sync status:

   stella admin feeds status --source ghsa
   

3. Test GitHub connectivity for the GHSA connector:

   stella sources check ghsa
   # Optional: --timeout 30  --json  --auto-disable
   

   sources check <source> runs a real connectivity probe via the source registry (ISourceRegistry.CheckConnectivityAsync) and reports reachability. (The unwired stella db connectors test is dead code and, even if reached, only simulates a probe — do not rely on it.)

Deep diagnosis

1. Review recent sync history for the source:

   stella admin feeds history --source ghsa --limit 20
   

   Look for repeated error rows; a partial status means a window was only partly ingested (often a mid-window rate-limit deferral).

2. Check rate-limit telemetry. The connector records GitHub X-RateLimit-* headers on every fetch as the ghsa.ratelimit.* instruments (see Telemetry). In your metrics backend:

   • ghsa.ratelimit.remaining near zero, or
   • ghsa.ratelimit.headroom_pct_current below ~10%, or
   • ghsa.ratelimit.exhausted incrementing indicates throttling. A warning is logged once when remaining drops at or below rateLimitWarningThreshold (default 500).

3. Check service logs for the connector. Log lines are emitted by GhsaConnector (StellaOps.Concelier.Connector.Ghsa). Look for:

   • "GHSA list fetch was rate limited; attempting public OSV GHSA fallback"
   • "GHSA GitHub API rate limit deferred the window"
   • "GHSA rate limit exhausted for {Phase} {Resource}; delaying {Delay}"
   • "Malformed GHSA JSON for {DocumentId}" (parse quarantine)

   Filter the Concelier service logs by the StellaOps.Concelier.Connector.Ghsa category through your normal log aggregation; there is no stella connector logs CLI command.

4. Check for a GitHub outage at https://www.githubstatus.com/. (There is no stella connector ghsa api-status command.)

Resolution

Immediate mitigation

1. If rate limited, let the next scheduled run resume, or trigger a refresh after the reset window:

   stella admin feeds refresh --source ghsa
   

   The connector preserves already-fetched document ids and the current window in its cursor, so a re-run resumes rather than restarting. Detail-fetch throttling after a list page was read leaves pending docs for the next source:ghsa:parse / source:ghsa:map jobs.

2. Trigger a specific stage manually (fetch, then parse, then map):

   stella db fetch --source ghsa --stage fetch
   stella db fetch --source ghsa --stage parse
   stella db fetch --source ghsa --stage map
   

3. Rely on the OSV fallback. If useOsvFallback is enabled (default), a rate-limited list fetch already pulls the affected window from the public OSV GHSA mirror automatically — confirm via the log line in step 3 above. There is no separate stella offline load command for this path.

Root cause fix

If rate limit is consistently exceeded:

Configuration is persisted per-source (key path concelier:sources:ghsa) and edited via the Console (Configure Sources) or the /api/v1/advisory-sources/{id}/configuration API (PUT with { "values": { ... } }). There is no wired stella db connectors configure CLI command — that subcommand exists only as unreferenced code. Reduce request pressure by editing the persisted config or the compatibility concelier.yaml:

# concelier.yaml (compatibility fallback for the concelier:sources:ghsa keys)
concelier:
  sources:
    ghsa:
      apiToken: "github_pat_xxx"   # optional; raises GitHub's per-hour quota
      pageSize: 10                 # 1..100, default 10
      maxPagesPerFetch: 1          # default 1
      requestDelay: "00:00:00.500" # default 200ms (TimeSpan)
      rateLimitWarningThreshold: 500
      secondaryRateLimitBackoff: "00:02:00"
      useOsvFallback: true             # default true
      maxOsvFallbackDocumentsPerFetch: 25  # default 25

The connector does not expose a sync_interval or incremental_sync config key; scheduling is fixed by the job cron (see Default schedule). Pacing is controlled by pageSize, maxPagesPerFetch, requestDelay, and the backoff options above. Authenticated requests use the persisted apiToken (sent as Authorization: Bearer <token> with X-GitHub-Api-Version: 2022-11-28).

If a token is expired or invalid:

1. Generate a new GitHub PAT (fine-grained or classic) at https://github.com/settings/personal-access-tokens.

2. Update the persisted token via the Console (Configure Sources) or the configuration API:

   PUT /api/v1/advisory-sources/ghsa/configuration
   { "values": { "apiToken": "<new-token>" } }
   

   (Or set concelier:sources:ghsa:apiToken in config.)

3. Verify the credential is accepted. If the token is registered as a connector credential via Platform, the read-only dry-run endpoint exercises the auth path (a GitHub GraphQL viewer probe; requires read:user):

   POST /api/v1/connectors/ghsa/dry-run-credential
   { "credentialId": "<uuid>" }
   

   (Requires scope concelier:credentials:dry-run, policy Concelier.Credentials.DryRun. Returns 501 for connectors without a dry-run handler — GHSA has one.) Otherwise re-run stella sources check ghsa.

The token is optional: GitHub serves public global advisories anonymously, so the connector validates without a token. Configure one only to raise rate limits, route through enterprise GitHub, or satisfy an operator policy. There is no enforced GitHub scope list for the REST advisories path; the GraphQL dry-run probe needs read:user for viewer.

If documents are quarantined (malformed JSON):

Quarantines surface as ghsa.parse.quarantine increments and "Malformed GHSA JSON for {DocumentId}" log lines; the affected document is marked Failed and skipped. Re-fetch the window after confirming upstream content is valid:

stella admin feeds refresh --source ghsa

If the window cursor appears stuck:

There is no stella connector ghsa reset-cursor command. The cursor is stored in Concelier source state and rolls forward automatically once a window completes without hasMore/rate-limit deferral. To force re-ingestion of a window, re-run the fetch stage; for a stuck source escalate to the Platform team to inspect/reset the persisted ghsa source state.

Verification

# Trigger a refresh
stella admin feeds refresh --source ghsa

# Re-check feed status and source health
stella admin feeds status --source ghsa
stella sources status

# Confirm recent sync history shows success
stella admin feeds history --source ghsa --limit 10

Looking up an individual advisory by GHSA id uses the vulnerability explorer, not a stella vuln query command:

stella vuln observations --tenant <tenant> --alias GHSA-xxxx-xxxx-xxxx

Telemetry

The connector emits metrics under the meter StellaOps.Concelier.Connector.Ghsa (defined in Internal/GhsaDiagnostics.cs):

Suggested alerts (define in your metrics backend — these are not shipped as named platform alerts):

• ghsa.ratelimit.remaining below rateLimitWarningThreshold for > 5 min.
• ghsa.ratelimit.headroom_pct_current below 10% for longer than a reset window.
• increase(ghsa.ratelimit.exhausted[15m]) > 0.

Default job schedule

Defaults from GhsaDependencyInjectionRoutine:

Authorization

The policy → scope mappings are registered in src/Concelier/StellaOps.Concelier.WebService/Program.cs (AddAuthorization block); scope constants live in src/Authority/StellaOps.Authority/StellaOps.Auth.Abstractions/StellaOpsScopes.cs. There is no concelier.sources.manage scope — Concelier.Sources.Manage is an any-scope policy over integration:write/integration:operate.

Prevention

• [ ] Authentication: Configure a GitHub token to raise the per-hour rate limit (optional but recommended for busy deployments).
• [ ] Monitoring: Alert on ghsa.ratelimit.exhausted and on stale sync history.
• [ ] Redundancy: The built-in OSV GHSA fallback covers rate-limit windows automatically; keep useOsvFallback enabled. NVD/OSV connectors provide additional ecosystem coverage.
• [ ] Token rotation: Rotate tokens before expiration; re-validate with stella sources check ghsa.

Related Resources

• Connector architecture: docs/modules/concelier/connectors.md
• Connector operations (config keys, telemetry, throttling): docs/modules/concelier/operations/connectors/ghsa.md
• Related runbooks: connector-nvd.md, connector-osv.md, connector-vendor-specific.md
• GitHub REST advisories API: https://docs.github.com/en/rest/security-advisories