eBPF Reachability Architecture

Audience: engineers working on (or integrating with) the Stella Ops eBPF reachability collector.

This document details the internal architecture of the runtime reachability evidence system — its kernel-space probes, user-space pipeline, evidence schema, data flow, and signing chain. For an orientation and the CLI surface, see the eBPF Reachability Overview; for installation and operation, see the Deployment Guide.

Implementation status (verify against src/). This document describes the StellaOps.Signals.Ebpf library (src/Signals/__Libraries/StellaOps.Signals.Ebpf). The hand-written BPF C programs (Probes/Bpf/*.bpf.c), the user-space parsing/enrichment/output/signing pipeline, the deterministic BTF source selector, and the signals verify-chain CLI are implemented. Live kernel attachment is not yet wired: the default CoreProbeLoader simulates probe loading and drains an in-memory buffer (see CoreProbeLoader below). Treat the kernel-attach/throughput claims as design targets until a libbpf binding lands.

System Overview

The eBPF reachability system is designed to capture kernel-level events to provide cryptographic proof of runtime behavior. It targets Linux eBPF (extended Berkeley Packet Filter) with CO-RE (Compile Once, Run Everywhere) for portable deployment across kernel versions.

Design Principles

1. Minimal Kernel Footprint: eBPF programs perform only essential filtering and data capture
2. User-Space Enrichment: Complex lookups (symbols, containers, SBOMs) happen in user space
3. Deterministic Output: Same inputs produce byte-identical NDJSON output
4. Chain of Custody: Every evidence chunk is cryptographically signed and linked

Component Architecture

Kernel-Space Components

Ring Buffer (BPF_MAP_TYPE_RINGBUF)

• Single shared events buffer for all event types (default 256KB, see stella_common.h)
• Multi-producer; every probe emits via the reserve/commit pattern (bpf_ringbuf_reserve() → fill → bpf_ringbuf_submit()/bpf_ringbuf_discard()). A submit_event() helper wrapping bpf_ringbuf_output() is defined in stella_common.h but is currently unused (no probe calls it).
• Automatic backpressure: a failed bpf_ringbuf_reserve() increments the dropped-event counter in the per-CPU probe_stats map (events_dropped) and the event is skipped. Statistics live only in that per-CPU map; they are not emitted through the ring buffer.

Tracepoint Probes

Defined across syscall_openat.bpf.c, syscall_exec.bpf.c, and syscall_network.bpf.c.

A kprobe/tcp_set_state fallback is declared for kernels lacking the inet_sock_set_state tracepoint, but its body is currently a stub (returns 0).

Uprobe / Uretprobe Probes

Network/SSL uprobes attach by symbol within libc/libssl (see uprobe_libc.bpf.c, uprobe_openssl.bpf.c); the generic function tracer attaches to arbitrary target functions (function_tracer.bpf.c).

BPF Maps for Filtering

The shared maps live in Probes/Bpf/stella_common.h. Container targeting is the only in-kernel filter that gates every probe (via the should_trace_cgroup() helper):

// Cgroup filter for container targeting (Probes/Bpf/stella_common.h)
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, 1024);
    __type(key, u64);    // cgroup_id
    __type(value, u8);   // 1 = trace, 0 = ignore
} target_cgroups SEC(".maps");

// Symbol filter for the function tracer (uprobe targets only)
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, 10000);
    __type(key, u64);    // function address
    __type(value, u8);   // 1 = trace
} target_symbols SEC(".maps");

Individual probes add their own filter maps in user-configurable arrays — e.g. openat_config/path_filters (file paths), tcp_filter_config/dest_cidr_filters (TCP state and CIDR), tracer_config (sampling/stack capture), and ssl_op_config.

Namespace filtering is user-space, not a BPF map. There is no namespace_filter map in the kernel programs. Linux-namespace-based multi-tenant isolation is implemented entirely in user space by CgroupContainerResolver/NamespaceFilter, which read /proc/{pid}/ns/* inodes (pid/mnt/net/user/cgroup) and match against a configured NamespaceFilter.

User-Space Components

CoreProbeLoader

Manages the eBPF program lifecycle (Probes/CoreProbeLoader.cs, implements IEbpfProbeLoader):

• Resolves the compiled probe object directory (/usr/share/stellaops/probes, /opt/stellaops/probes, or <base>/probes) and the container’s PIDs via cgroup paths / /proc scanning
• Throws PlatformNotSupportedException off Linux
• Exposes detach, ring-buffer drain, symbol resolution, and buffer/CPU/memory stats

NOT YET A LIVE libbpf LOADER. The current CoreProbeLoader is a development/test harness: probe attachment is simulated and events are drained from a SimulatedRingBuffer (a bounded in-memory channel). The source explicitly notes the production path (“In real implementation, this would: 1. Load BPF object using libbpf; 2. Create ring buffer map; 3. Attach uprobes…”). GetCpuOverhead/GetBufferUtilization return placeholder constants. The compiled .bpf.o artifacts and the libbpf binding are not wired in yet — treat live kernel attachment as roadmap. The hand-written BPF C programs in Probes/Bpf/ are real but are not compiled or loaded by this loader today.

AirGapProbeLoader

Offline probe distribution (Probes/AirGapProbeLoader.cs, IAirGapProbeLoader):

• Loads pre-compiled probes from embedded resources or a sealed ebpf-probes.bundle (a ZIP with a manifest.json), with no build tools or network at runtime
• Verifies each probe’s SHA-256 against the manifest; can author bundles via CreateBundleAsync
• Enabled by AddEbpfRuntimeEvidenceAirGap(...) / EbpfEvidenceOptions.UseAirGapMode

EventParser

Parses binary events from the ring buffer (Parsers/EventParser.cs):

• 48-byte common header (timestamp_ns, pid, tid, cgroup_id, event-type discriminator, comm) followed by type-specific payloads parsed little-endian
• Dispatches on the event_type discriminator to FileOpen / ProcessExec / TcpState / NetConnect / SslOp / FunctionCall records; resolves symbols for function-call events
• Timestamps are carried through as the raw kernel boot-time timestamp_ns (bpf_ktime_get_boot_ns); the parser does not convert them to wall-clock time

RuntimeEvidenceCollector (unified) and RuntimeSignalCollector (legacy adapter)

Two collector facades coordinate the pipeline:

• RuntimeEvidenceCollector(Services/) is the unified collector: it loads probes, drains the ring buffer, parses + enriches (cgroup resolver) records, writes NDJSON, and raises ChunkCompleted on rotation.
• RuntimeSignalCollectoris the legacy IRuntimeSignalCollector adapter focused on function-call signals: it aggregates ObservedCallPaths, computes NodeHash/PathHash/ combined-path-hash and BTF-selection metadata, and feeds the Scanner merger. Gates on IsSupported() (which is false when no BTF source is selected).

CgroupContainerResolver

Maps PIDs / kernel cgroup IDs to container identities (Cgroup/CgroupContainerResolver.cs):

• Parses /proc/{pid}/cgroup for container runtime paths
• Supports containerd, Docker, CRI-O, and Podman path formats (64-hex-char IDs)
• Resolves primarily by PID; cgroup-ID lookups rely on a cache populated via RegisterCgroupMapping (a cold cgroup-ID returns null rather than scanning /proc)
• Also reads Linux namespace inodes from /proc/{pid}/ns/* and supports a NamespaceFilter for user-space multi-tenant isolation
• Caches per-PID/per-cgroup/per-namespace identities in plain ConcurrentDictionary maps with no time-based eviction (the CacheTtl = 5 min constant is declared but not enforced here — entries are dropped only via InvalidatePid/Dispose). Note: the 5-minute TTL is live in EnhancedSymbolResolver (its IMemoryCache sliding expiration), not in this resolver.

EnhancedSymbolResolver

Resolves addresses to human-readable symbols (Symbols/EnhancedSymbolResolver.cs):

• Parses /proc/{pid}/maps for ASLR offsets, then maps the address back to a file offset within the containing mapping
• Reads ELF symbol tables (.symtab = SHT_SYMTAB, .dynsym = SHT_DYNSYM) for 64-bit ELF only; 32-bit ELF parsing is a stub that returns null and falls back to address-based identifiers. There is no DWARF parsing — the source has no debug-line support; unresolved addresses degrade to addr:0x{hex} / {symbol}+0x{offset} forms.
• Caches resolved symbols in a size-limited IMemoryCache (sliding 5-min TTL, size limit EbpfEvidenceOptions.SymbolCacheSizeLimit) plus per-process /proc/{pid}/maps and per-file ELF symbol-table dictionaries

RuntimeEventEnricher

Decorates events with container and SBOM metadata:

• Container ID and image digest correlation
• SBOM component (PURL) lookup
• Graceful degradation on missing metadata

RuntimeEvidenceNdjsonWriter

Produces deterministic NDJSON output (Output/RuntimeEvidenceNdjsonWriter.cs):

• Deterministic JSON: snake_case property names, nulls omitted, no indentation, relaxed escaping (the writer relies on the record’s property ordering rather than an explicit key-sort pass)
• Rolling SHA-256 content hash (IncrementalHash over each serialized line + newline); the resulting chunk hash is reported as sha256:<hex>. (The source comment calls this “BLAKE3-like using SHA256 incremental” — the algorithm is SHA-256, not BLAKE3.)
• Size-based (default 100 MB) and time-based (default 1 hour) rotation, optional gzip, and a ChunkRotated callback that carries chunk statistics and the previous chunk hash

EvidenceChunkFinalizer

Signs and links evidence chunks (Signing/EvidenceChunkFinalizer.cs):

• Builds a RuntimeEvidencePredicate (predicateType stella.ops/runtime-evidence@v1) and an in-toto v0.1 statement with chunk metadata
• Delegates signing to an IEvidenceChunkSigner; submits to Rekor when requested
• Maintains per-chain state (previous_chunk_id linkage, totals) and can persist/restore it to disk for recovery across restarts
• Provides VerifyChainAsync, which checks signatures, chain linkage, sequence continuity, and time monotonicity

IEvidenceChunkSigner implementations

• AttestorEvidenceChunkSigner(production): signs via the Attestor IAttestationSigningService and submits via IRekorClient. (Per the module map the historical Signer module is consolidated into Attestor — these abstractions live in StellaOps.Attestor.Core.) Payload type application/vnd.in-toto+json; the statement is canonicalized with CanonJson before signing. VerifyAsync checks Rekor inclusion when a UUID is present.
• LocalEvidenceChunkSigner(dev/test): produces a deterministic DSSE envelope by computing the DSSE PAE and an HMAC through ICryptoHmac/HmacPurpose.Signing (algorithm follows the active compliance profile per CoC §7.4 — HMAC-SHA256 on world/fips/kcmvp/eidas, HMAC-GOST3411 on gost, HMAC-SM3 on sm). Verifies with a fixed-time HMAC comparison; does not submit to Rekor.
• NullEvidenceChunkSigner: emits an unsigned predicate (no DSSE envelope) for paths where signing is disabled.

Evidence Record Schema (runtime-evidence/v1)

Each NDJSON line is a RuntimeEvidenceRecord (Schema/RuntimeEvidence.cs). Common envelope fields, then a polymorphic event object discriminated by type:

{
  "ts_ns": 12345678901234,        // boot-time nanoseconds (bpf_ktime_get_boot_ns)
  "src": "sys_enter_openat",      // probe/source name
  "pid": 4711,
  "tid": 4711,
  "cgroup_id": 1234567890,
  "container_id": "containerd://abc…",  // enriched; omitted when null
  "image_digest": "sha256:…",           // enriched; omitted when null
  "comm": "nginx",
  "event": { "type": "file_open", /* … */ }
}

Event payload variants (type discriminator → fields):

The record carries no random identifiers, supporting the byte-identical-output goal.

Data Flow

1. Kernel Event
   │
   ├─► Tracepoint/Uprobe fires
   │   └─► BPF program captures event data
   │       └─► Filter by cgroup/namespace (optional)
   │           └─► Submit to ring buffer
   │
2. Ring Buffer Drain
   │
   ├─► EventParser reads binary data
   │   └─► Deserialize to typed event struct
   │       └─► Validate event integrity
   │
3. Resolution & Enrichment
   │
   ├─► SymbolResolver: address → symbol name (function-call events, in EventParser)
   ├─► CgroupResolver: cgroup_id / pid → container_id (RuntimeEvidenceCollector.EnrichRecord)
   │
   │   Optional deeper enrichment via RuntimeEventEnricher (Enrichment/):
   │   ├─► IContainerStateProvider: container_id → image_ref / image_digest
   │   ├─► IImageDigestResolver:    image_ref → image_digest
   │   └─► ISbomComponentProvider:  image_digest → purls[]
   │   (These providers are interfaces; defaults degrade gracefully — e.g.
   │    NullSbomComponentProvider, LocalContainerIdentityResolver with no image digests.
   │    The unified RuntimeEvidenceCollector wires only the cgroup resolver today.)
   │
4. Serialization
   │
   ├─► RuntimeEvidenceNdjsonWriter
   │   ├─► Canonical JSON serialization
   │   ├─► Append to current chunk file
   │   └─► Update rolling hash
   │
5. Rotation & Signing
   │
   ├─► Size/time threshold reached
   │   └─► Close current chunk
   │       └─► ChunkFinalizer
   │           ├─► Create in-toto statement
   │           ├─► Sign with DSSE
   │           ├─► Submit to Rekor
   │           └─► Link to previous chunk
   │
6. Verification
   │
   └─► CLI `signals verify-chain` (StellaOps.Cli SignalsCommandGroup)
       │   → EvidenceChunkFinalizer.VerifyChainAsync
       ├─► Verify signatures (Rekor inclusion when a UUID is present)
       ├─► Check chain linkage (previous_chunk_id)
       ├─► Check sequence continuity (chunk_sequence)
       └─► Validate time monotonicity (no overlapping time ranges)

The signals command group also exposes inspect, list, and summary subcommands.

Performance Characteristics

These are design targets, not measured benchmarks. The kernel-space numbers in particular assume a live libbpf attachment, which is still roadmap (see CoreProbeLoader above); the current simulated loader reports placeholder CPU/buffer figures. Rate limiting is real and enforced in RuntimeSignalCollector via MaxEventsPerSecond.

Kernel-Space

• Ring buffer prevents event loss under load (backpressure)
• In-kernel filtering reduces user-space processing
• BTF enables zero-copy field access

User-Space

Throughput

• Target: 100,000 events/second sustained
• Rate limiting available for resource-constrained environments

Memory Budget

Note: the symbol resolver declares an internal MaxCachedSymbolsPerProcess = 10000 constant, but it is currently unreferenced (dead code) — the only live symbol-cache bound is the global IMemoryCache size limit above. Do not rely on a per-process cap.

Failure Modes

Ring Buffer Overflow

• Symptom: Events dropped, warning logged
• Mitigation: Increase buffer size or enable rate limiting

Symbol Resolution Failure

• Symptom: Address shown as addr:0x{hex}
• Mitigation: Ensure debug symbols available or accept address-only evidence

Container Resolution Failure

• Symptom: container_id = "unknown:{cgroup_id}" — but only on the RuntimeEventEnricher path (FormatUnknownContainer). The unified RuntimeEvidenceCollector.EnrichRecord leaves container_id null/omitted when the cgroup/PID cannot be resolved; it does not emit an unknown: sentinel.
• Mitigation: Verify Zastava integration, check cgroup path format support (containerd/ docker/cri-o/podman 64-hex .scope paths)

Signing Failure

• Symptom: Rekor submission fails — a warning is logged and the chunk’s DSSE signature is still returned (the in-toto statement is signed before Rekor submission; only transparency-log inclusion is skipped). With NullEvidenceChunkSigner, no DSSE envelope is produced at all.
• Mitigation: Check Attestor signing service (IAttestationSigningService) and Rekor (IRekorClient) availability. For development, LocalEvidenceChunkSigner produces a self-contained HMAC-signed envelope with no transparency-log dependency.

CO-RE (Compile Once, Run Everywhere)

The probes include bpf/bpf_core_read.h and use BTF (BPF Type Format) for kernel-version-independent field access. The illustrative CO-RE pattern is:

// Illustrative: CO-RE field access without hardcoded offsets
struct task_struct *task = (void *)bpf_get_current_task();
pid_t pid  = BPF_CORE_READ(task, pid);
pid_t tgid = BPF_CORE_READ(task, tgid);

In the current probes the equivalent reads are written explicitly with bpf_probe_read_kernel(...) against generated __bindgen_anon_1 accessors (see syscall_exec.bpf.c, which reads the parent task and its tgid this way) rather than the BPF_CORE_READ convenience macro.

BTF source selection is deterministic (Services/RuntimeBtfSourceSelector.cs), in precedence order:

1. Kernel built-in BTF at /sys/kernel/btf/vmlinux
2. Configured external vmlinux BTF paths
3. Split-BTF fallback under /var/lib, /usr/share, or /usr/lib stellaops/btf/split/<release>/<arch>/…

The selected source’s path and SHA-256 digest are recorded on the collection summary. If no source is found (or the host is non-Linux), collection reports IsSupported() == false.

Requirements:

• Kernel 5.2+ with built-in BTF (recommended)
• Older kernels (e.g. 4.14+) require external/split BTF supplied via the paths above (offline-friendly; no btfhub network fetch in the offline-first posture)

Integration Points

Container state (Zastava or local)

• IContainerIdentityResolver interface (resolve by container-id / pid / cgroup-id, plus ContainerStarted/ContainerStopped lifecycle events)
• LocalContainerIdentityResolver is the default fallback — wraps the local CgroupContainerResolver, uses /proc introspection only, and does not emit lifecycle events or image digests. Container lifecycle and image-digest mapping require an external resolver (e.g. a Zastava-backed implementation).
• IContainerStateProvider supplies container metadata (image ref/digest) for enrichment.

Scanner (Reachability Merger)

• EbpfSignalMerger (Scanner.Reachability/Runtime/) merges eBPF call paths with the static call graph by delegating to RuntimeStaticMerger.
• Correlation is by caller→callee symbol-name edges (it converts each observed call path into per-edge RuntimeCallEvents keyed on symbol names). The collector separately computes a per-node hash (NodeHash = hash of PURL + symbol) and combined PathHash on the runtime summary, but the merger itself correlates on symbol names, not on those hashes. (No RuntimeNodeHash type exists in the merger; the name appears only as a test class.)

Evidence signing (Attestor)

• IAttestationSigningService for DSSE signatures and IRekorClient for transparency-log submission, both from StellaOps.Attestor.Core. The historical Signer module is consolidated into Attestor.

SBOM correlation

• ISbomComponentProvider for PURL lookup by image digest
• CachingSbomComponentProvider adds a TTL cache; NullSbomComponentProvider is the no-op default when no SBOM integration is configured

Authorization

StellaOps.Signals.Ebpf is an internal collection library with no HTTP surface of its own. Where runtime/reachability signals are exposed (the Signals service and the signals CLI group), access is governed by the canonical reachability-signal scopes from StellaOps.Auth.Abstractions/StellaOpsScopes.cs:

(Verify the policy registration vs. the scope constant before relying on a specific endpoint’s requirement — ASP.NET policy names and scope claim values can differ.)

See Also

• eBPF Reachability Overview
• Deployment Guide
• Evidence Schema Reference
• Probe Reference
• Security Model
• Operator Runbook