Concelier CERT-Bund Connector Operations

Last updated: 2025-10-17

Audience: Stella Ops operators running the Concelier CERT-Bund connector.

Runtime/catalog source ID: cert-de. Legacy cert-bund remains a compatibility-only configuration alias.

Germany’s Federal Office for Information Security (BSI) operates the Warn- und Informationsdienst (WID) portal. The Concelier CERT-Bund connector uses the runtime/catalog source ID cert-de, so the live fetch pipeline runs as source:cert-de:* while preserving the original German content.

1. Configuration Checklist

Example concelier.yaml fragment:

concelier:
  sources:
    cert-de:
      feedUri: "https://wid.cert-bund.de/content/public/securityAdvisory/rss"
      portalBootstrapUri: "https://wid.cert-bund.de/portal/"
      detailApiUri: "https://wid.cert-bund.de/portal/api/securityadvisory"
      maxAdvisoriesPerFetch: 50
      maxKnownAdvisories: 512
      requestTimeout: "00:00:30"
      requestDelay: "00:00:00.250"
      failureBackoff: "00:05:00"

Leave maxAdvisoriesPerFetch at 50 during normal operation. Raise it only for controlled backfills, then restore the default to avoid overwhelming the portal.

The RSS-metadata fallback intentionally does not invent affected products or version ranges. Product/range fidelity still depends on the portal JSON/search/export APIs or an offline WID export snapshot.

2. Telemetry & Logging

Structured logs (all emitted at information level when work occurs):

Alerting ideas:

  1. increase(certbund.detail.fetch.failures_total[10m]) > 0
  2. rate(certbund.map.success_total[30m]) == 0
  3. histogram_quantile(0.95, rate(concelier_source_http_duration_bucket{concelier_source="cert-de"}[15m])) > 5s

The WebService now registers the meter so metrics surface automatically once OpenTelemetry metrics are enabled.

3. Historical Backfill & Export Strategy

3.1 Retention snapshot

3.2 JSON search pagination

# 1. Bootstrap cookies (client_config + XSRF-TOKEN)
curl -s -c cookies.txt "https://wid.cert-bund.de/portal/" > /dev/null
curl -s -b cookies.txt -c cookies.txt \
     -H "X-Requested-With: XMLHttpRequest" \
     "https://wid.cert-bund.de/portal/api/security/csrf" > /dev/null

XSRF=$(awk '/XSRF-TOKEN/ {print $7}' cookies.txt)

# 2. Page search results
curl -s -b cookies.txt \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -H "X-XSRF-TOKEN: ${XSRF}" \
     -X POST \
     --data '{"page":4,"size":100,"sort":["published,desc"]}' \
     "https://wid.cert-bund.de/portal/api/securityadvisory/search" \
     > certbund-page4.json

Iterate page until the response content array is empty. Pages 0–9 currently cover 2014-present. Persist JSON responses (plus SHA256) for Offline Kit parity.

Shortcut – run python src/Tools/certbund_offline_snapshot.py --output src/__Tests/__Datasets/seed-data/cert-bund to bootstrap the session, capture the paginated search responses, and regenerate the manifest/checksum files automatically. Supply --cookie-file and --xsrf-token if the portal requires a browser-derived session (see options via --help).

3.3 Export bundles

python src/Tools/certbund_offline_snapshot.py \
  --output src/__Tests/__Datasets/seed-data/cert-bund \
  --start-year 2014 \
  --end-year "$(date -u +%Y)"

The helper stores yearly exports under src/__Tests/__Datasets/seed-data/cert-bund/export/, captures paginated search snapshots in src/__Tests/__Datasets/seed-data/cert-bund/search/, and generates the manifest + SHA files in src/__Tests/__Datasets/seed-data/cert-bund/manifest/. Split ranges according to your compliance window (default: one file per calendar year). Concelier can ingest these JSON payloads directly when operating offline.

When automatic bootstrap fails (e.g. portal introduces CAPTCHA), run the manual curl flow above, then rerun the helper with --skip-fetch to rebuild the manifest from the existing files.

3.4 Connector-driven catch-up

  1. Temporarily raise maxAdvisoriesPerFetch (e.g. 150) and reduce requestDelay.
  2. Run stella db fetch --source cert-de --stage fetch, then --stage parse, then --stage map until the fetch log reports enqueued=0.
  3. Restore defaults and capture the cursor snapshot for audit.

4. Locale & Translation Guidance

5. Verification Checklist

  1. Observe certbund.feed.fetch.success increments after runs; certbund.feed.coverage.days should hover near the observed RSS window. certbund.detail.fetch.success may store portal HTML when the WID detail API is serving the Angular shell.
  2. Ensure summary logs report truncated=false in steady state — true indicates the fetch cap was hit.
  3. During backfills, watch certbund.feed.enqueued.count trend to zero.
  4. Spot-check stored advisories in PostgreSQL (schema vuln) to confirm language="de" and reference URLs match the portal detail endpoint.
  5. For entries mapped from RSS fallback metadata, confirm aliases include any CVEs present in the RSS title or summary and affected-package counts are zero until JSON/export data is available.
  6. For Offline Kit exports, validate SHA256 hashes before distribution.