Feedser Architecture

Evidence-extraction libraries for backport detection: patch signatures (HunkSig), function-signature extraction, binary fingerprinting, and signal attachment.

Scope. Library architecture for the Feedser assemblies (StellaOps.Feedser.Core, StellaOps.Feedser.BinaryAnalysis). They provide patch-signature extraction, function-signature extraction, binary fingerprinting, and feed-signal attachment supporting the four-tier backport proof system. The projects live under src/Concelier/ and are consumed by Concelier’s ProofService, the Attestor ProofChain, and the Scanner PatchVerification library.

Naming note. “Feedser” was the historical name of the advisory-aggregation service that is now Concelier (see §10). The name survives only on these two helper libraries; the aggregation service itself is documented under ../concelier/architecture.md.

0) Mission & boundaries

Mission. Provide deterministic evidence extraction for backport detection. Extract patch signatures from unified diffs, function signatures from patch context, and binary fingerprints from compiled code, plus attach external feed signals (EPSS/KEV) — enabling high-confidence vulnerability-status determination for packages where upstream fixes have been backported by distro maintainers.

Boundaries.

• The Feedser assemblies are libraries, not a standalone service. They expose no REST APIs and ship no WebService/Worker host of their own.
• They do not make vulnerability decisions. They produce evidence records that feed into the Attestor ProofBlob / VEX statements and Policy Engine evaluation.
• StellaOps.Feedser.Core and StellaOps.Feedser.BinaryAnalysis do not persist data; storage of patch signatures, patch headers, and binary fingerprints is handled by consuming components (StellaOps.Concelier.ProofService.Postgres).
• Extraction is deterministic in its hashing (sorted hunk hashes, ordinal sorts). Note: BinaryFingerprint.ExtractedAt and PatchSignature.ExtractedAt carry a wall-clock UTC stamp via the supplied TimeProvider; that timestamp field is intentionally non-deterministic (annotated Determinism:Allowed in SimplifiedTlshFingerprinter).

1) Solution & project layout

The Feedser assemblies live under src/Concelier/ (not a separate src/Feedser/ tree). Actual on-disk layout:

src/Concelier/
 ├─ StellaOps.Feedser.Core/                       # Patch + function signature extraction, feed signals
 │   ├─ HunkSigExtractor.cs                        # static class — unified-diff parser/normalizer (HunkSig)
 │   ├─ FunctionSignatureExtractor.cs              # static class — function-signature extraction (C/C++/Go/
 │   │                                             #   Python/Rust/Java/JS/TS) + fuzzy matching extensions
 │   ├─ Models/
 │   │   └─ PatchSignature.cs                      # PatchSignature + PatchHunk records
 │   └─ Signals/                                   # Determinization signal attachers (SPRINT_20260106_001)
 │       ├─ ISignalAttacher.cs                     # ISignalAttacher<TInput,TSignal>, SignalState<T>,
 │       │                                         #   SignalStatus, SignalUpdatedEvent
 │       ├─ EpssSignalAttacher.cs                  # EPSS attacher + IEpssDataSource, ISignalEventEmitter
 │       ├─ KevSignalAttacher.cs                   # KEV attacher + IKevDataSource
 │       └─ SignalAttacherServiceExtensions.cs     # DI registration + timeline/in-memory emitters
 │
 ├─ StellaOps.Feedser.BinaryAnalysis/             # Binary fingerprinting engine
 │   ├─ BinaryFingerprintFactory.cs                # concrete factory over registered fingerprinters
 │   ├─ IBinaryFingerprinter.cs                    # Fingerprinter interface (async extract/match)
 │   ├─ Models/
 │   │   └─ BinaryFingerprint.cs                   # BinaryFingerprint, FingerprintMethod enum,
 │   │                                             #   FingerprintMetadata, FingerprintMatchResult
 │   └─ Fingerprinters/
 │       ├─ SimplifiedTlshFingerprinter.cs         # Simplified TLSH-style fuzzy hashing (PoC)
 │       └─ InstructionHashFingerprinter.cs        # Instruction-sequence hashing
 │
 └─ __Tests/
     ├─ StellaOps.Feedser.Core.Tests/             # HunkSig + FunctionSignature + Signals tests
     └─ StellaOps.Feedser.BinaryAnalysis.Tests/   # BinaryFingerprint model tests

Files NOT present (claimed by earlier drafts, do not exist): HunkSignature.cs, DiffParseResult.cs, Normalization/WhitespaceNormalizer.cs, a standalone FingerprintMatchResult.cs. There is no src/Concelier/StellaOps.Feedser.Core/plugins/concelier/ integration plugin — the src/Concelier/plugins/concelier/ directory holds Concelier advisory connectors, unrelated to these libraries.

2) External dependencies

• StellaOps.Concelier.ProofService(BackportProofService) — primary consumer; orchestrates the four-tier evidence query.
• StellaOps.Attestor.ProofChain— references both Feedser assemblies; BackportProofGenerator aggregates evidence into ProofBlob records.
• StellaOps.Scanner.PatchVerification— references the Feedser assemblies for patch/binary verification.
• StellaOps.Eventing— StellaOps.Feedser.Core project reference; used by the signal attachers (TimelineSignalEventEmitter emits feedser.signal.updated timeline events).
• Microsoft.Extensions.DependencyInjection.Abstractions / …Logging.Abstractions— package references in StellaOps.Feedser.Core. StellaOps.Feedser.BinaryAnalysis has no package references.
• .NET 10 (net10.0) — runtime target; Nullable + ImplicitUsings enabled, TreatWarningsAsErrors=true.
• No database dependencies in these libraries (persistence lives in StellaOps.Concelier.ProofService.Postgres).
• No outbound network calls inside the extractors; the EPSS/KEV attachers delegate lookups to injected IEpssDataSource/IKevDataSource implementations.

3) Contracts & data model

3.1 Patch Signature (Tier 3 Evidence)

Source: StellaOps.Feedser.Core/Models/PatchSignature.cs.

public sealed record PatchSignature
{
    public required string PatchSigId { get; init; }                 // "sha256:{hunkHash}"
    public required string? CveId { get; init; }                     // associated CVE
    public required string UpstreamRepo { get; init; }
    public required string CommitSha { get; init; }
    public required IReadOnlyList<PatchHunk> Hunks { get; init; }
    public required string HunkHash { get; init; }                   // SHA-256 of sorted hunk hashes (hex)
    public required IReadOnlyList<string> AffectedFiles { get; init; }
    public required IReadOnlyList<string>? AffectedFunctions { get; init; }
    public required DateTimeOffset ExtractedAt { get; init; }
    public required string ExtractorVersion { get; init; }           // "1.0.0"
}

public sealed record PatchHunk
{
    public required string FilePath { get; init; }
    public required int StartLine { get; init; }                     // old-side start line ("@@ -N,..")
    public required string Context { get; init; }                    // joined context lines
    public required IReadOnlyList<string> AddedLines { get; init; }
    public required IReadOnlyList<string> RemovedLines { get; init; }
    public required string HunkHash { get; init; }                   // SHA-256 of normalized hunk (hex)
}

Hashing is SHA-256 (hex, lower-cased) throughout — see HunkSigExtractor.ComputeSha256. Earlier drafts claiming BLAKE3-256 were incorrect.

3.2 Extracted Function Signature (Tier 3/4 helper)

Source: StellaOps.Feedser.Core/FunctionSignatureExtractor.cs. Used to populate PatchSignature.AffectedFunctions and for fuzzy function matching.

public sealed record ExtractedFunction
{
    public required string Name { get; init; }
    public required string Signature { get; init; }
    public required ProgrammingLanguage Language { get; init; }
    public required string FilePath { get; init; }
    public required double Confidence { get; init; }
    public string? Receiver { get; init; }                           // e.g. Go receiver
    public bool IsAsync { get; init; }
}

public enum ProgrammingLanguage
{
    Unknown, C, Cpp, Go, Python, Rust, Java, JavaScript, TypeScript, CSharp, Ruby, Php, Swift, Kotlin
}

Fuzzy matching (FunctionMatchingExtensions.ComputeMatch) produces a FunctionMatchResult with a weighted score: name similarity (Levenshtein) 0.50, signature similarity 0.35, language-compat bonus 0.15; IsStrongMatch at >= 0.70, IsWeakMatch at >= 0.40.

3.3 Binary Fingerprint (Tier 4 Evidence)

Source: StellaOps.Feedser.BinaryAnalysis/Models/BinaryFingerprint.cs.

public sealed record BinaryFingerprint
{
    public required string FingerprintId { get; init; }              // "fingerprint:{method}:{hash}"
    public required string? CveId { get; init; }
    public required FingerprintMethod Method { get; init; }          // enum (see below)
    public required string FingerprintValue { get; init; }
    public required string TargetBinary { get; init; }               // binary file / symbol name
    public string? TargetFunction { get; init; }
    public required FingerprintMetadata Metadata { get; init; }
    public required DateTimeOffset ExtractedAt { get; init; }
    public required string ExtractorVersion { get; init; }
}

public enum FingerprintMethod
{
    TLSH,             // Trend Micro Locality Sensitive Hash (fuzzy)
    CFGHash,          // Function-level control-flow-graph hash
    InstructionHash,  // Normalized instruction-sequence hash
    SymbolHash,       // Symbol-table hash
    SectionHash       // Section hash (e.g. .text)
}

public sealed record FingerprintMetadata
{
    public required string Architecture { get; init; }               // x86_64, aarch64, armv7, …
    public required string Format { get; init; }                     // ELF, PE, Mach-O
    public string? Compiler { get; init; }
    public string? OptimizationLevel { get; init; }
    public required bool HasDebugSymbols { get; init; }
    public long? FileOffset { get; init; }
    public long? RegionSize { get; init; }
}

public sealed record FingerprintMatchResult
{
    public required bool IsMatch { get; init; }
    public required double Similarity { get; init; }                 // 0.0-1.0 (double, not decimal)
    public required double Confidence { get; init; }                 // 0.0-1.0 (double, not decimal)
    public string? MatchedFingerprintId { get; init; }
    public required FingerprintMethod Method { get; init; }
    public Dictionary<string, object>? MatchDetails { get; init; }   // e.g. hamming_distance, hashes
}

Only TLSH and InstructionHash are registered in BinaryFingerprintFactory today (CFGHash/SymbolHash/SectionHash are declared in the enum but have no fingerprinter implementation — requesting one throws NotSupportedException).

3.4 Feed Signal Attachment (determinization pipeline)

Source: StellaOps.Feedser.Core/Signals/. The signal subsystem attaches external feed scores (EPSS, KEV) to the determinization pipeline.

public interface ISignalAttacher<TInput, TSignal>
{
    Task<SignalState<TSignal>> AttachAsync(TInput input, CancellationToken ct = default);
    Task<IReadOnlyList<SignalState<TSignal>>> AttachBatchAsync(IReadOnlyList<TInput> inputs, CancellationToken ct = default);
    string SignalType { get; }   // "epss" | "kev"
}

public enum SignalStatus { Available, NotFound, Failed, Pending, Expired, NotConfigured }

EpssSignalAttacher (SignalType = "epss") wraps IEpssDataSource and yields an EpssSignal { CveId, Score, Percentile, ScoreDate, ModelVersion }; KevSignalAttacher (SignalType = "kev") wraps IKevDataSource. Both emit a SignalUpdatedEvent through ISignalEventEmitter. The default DI registration installs a fail-closed UnsupportedSignalEventEmitter; production must opt into AddTimelineSignalEventEmitter (durable, emits feedser.signal.updated via ITimelineEventEmitter), and tests use AddSignalAttachersForTesting (InMemorySignalEventEmitter).

3.5 Evidence Tier Confidence Levels

The Feedser libraries themselves do not assign per-tier confidence ranges; confidence is assigned by the Attestor BackportProofGeneratorwhen it builds a ProofBlob. The canonical values are:

Per-evidence base strength (BackportProofGenerator.Confidence.cs, ProofStrength enum, /100):

Aggregate confidence = strongest evidence strength + a diminishing corroboration bonus (+0.05 for 2 evidences, +0.08 for 3, +0.10 for 4+), capped at 0.98.

Per-generator emitted confidences differ from the aggregate base above (BackportProofGenerator.Tier*.cs):

4) Core Components

4.1 HunkSigExtractor

HunkSigExtractor is a static partial class (not an interface). Its single public entry point is:

public static PatchSignature ExtractFromDiff(
    string cveId,
    string upstreamRepo,
    string commitSha,
    string unifiedDiff,
    TimeProvider? timeProvider = null);

There is no IHunkSigExtractor interface and no Extract / ExtractMultiple methods. Multi-file diffs are handled inside ExtractFromDiff: ParseUnifiedDiff walks ---/+++/@@ headers and emits one PatchHunk per @@ block across all files.

Normalization rules (NormalizeLine / NormalizeHunk):

• Normalize line endings to LF (\r\n and \r → \n) before parsing.
• Trim leading/trailing whitespace per line.
• Remove C-style comments (/* … */ and // …) via regex.
• Collapse runs of whitespace to a single space.
• Drop blank/whitespace-only lines.
• Per-hunk hash = SHA-256 of (normalized added lines)\n(normalized removed lines); the patch-level HunkHash = SHA-256 over the sorted set of per-hunk hashes, making the signature order-independent.
• AffectedFiles are de-duplicated and ordinal-sorted; AffectedFunctions are derived via FunctionSignatureExtractor.

The source comment in NormalizeHunk mentions lowercasing, but the implementation does not lower-case content; only trimming, comment removal, and whitespace collapse are applied.

4.2 BinaryFingerprintFactory

BinaryFingerprintFactory is a concrete sealed class (no IBinaryFingerprintFactory interface). It maintains a Dictionary<FingerprintMethod, IBinaryFingerprinter> populated with SimplifiedTlshFingerprinter and InstructionHashFingerprinter.

public sealed class BinaryFingerprintFactory
{
    public IBinaryFingerprinter GetFingerprinter(FingerprintMethod method);              // throws NotSupportedException if unregistered
    public Task<IReadOnlyList<BinaryFingerprint>> ExtractAllAsync(string binaryPath, string? cveId, string? targetFunction = null, CancellationToken ct = default);
    public Task<IReadOnlyList<BinaryFingerprint>> ExtractAllAsync(ReadOnlyMemory<byte> binaryData, string binaryName, string? cveId, string? targetFunction = null, CancellationToken ct = default);
    public Task<FingerprintMatchResult?> MatchBestAsync(string candidatePath, IEnumerable<BinaryFingerprint> known, CancellationToken ct = default);
    public Task<FingerprintMatchResult?> MatchBestAsync(ReadOnlyMemory<byte> candidateData, IEnumerable<BinaryFingerprint> known, CancellationToken ct = default);
    public IReadOnlyList<FingerprintMethod> GetAvailableMethods();
}

IBinaryFingerprinter is async and works on file paths or byte buffers (not ReadOnlySpan<byte>):

public interface IBinaryFingerprinter
{
    FingerprintMethod Method { get; }                                                    // typed enum, not string
    Task<BinaryFingerprint> ExtractAsync(string binaryPath, string? cveId, string? targetFunction = null, CancellationToken ct = default);
    Task<BinaryFingerprint> ExtractAsync(ReadOnlyMemory<byte> binaryData, string binaryName, string? cveId, string? targetFunction = null, CancellationToken ct = default);
    Task<FingerprintMatchResult> MatchAsync(string candidatePath, BinaryFingerprint knownFingerprint, CancellationToken ct = default);
    Task<FingerprintMatchResult> MatchAsync(ReadOnlyMemory<byte> candidateData, BinaryFingerprint knownFingerprint, CancellationToken ct = default);
}

MatchBestAsync only invokes fingerprinters whose Method matches a known fingerprint, then returns the highest-Confidence (tie-broken by Similarity) result among those with IsMatch == true.

Registered fingerprinting methods:

SimplifiedTlshFingerprinter is explicitly a proof-of-concept; its header notes production use should integrate a real TLSH library (e.g. libtlsh via P/Invoke).

5) Integration with Concelier

The Feedser libraries are consumed via StellaOps.Concelier.ProofService.BackportProofService (file: src/Concelier/__Libraries/StellaOps.Concelier.ProofService/BackportProofService.cs). Its public surface is GenerateProofAsync(cveId, packagePurl, ct) and GenerateProofBatchAsync(...), both returning Attestor ProofBlob records.

BackportProofService (Concelier.ProofService)
  ├─ Tier 1: IDistroAdvisoryRepository.FindByCveAndPackageAsync  → EvidenceType.DistroAdvisory
  ├─ Tier 2: ISourceArtifactRepository.FindChangelogsByCveAsync  → EvidenceType.ChangelogMention
  ├─ Tier 3: IPatchRepository.FindPatchHeadersByCveAsync         → EvidenceType.PatchHeader
  │          IPatchRepository.FindPatchSignaturesByCveAsync (HunkSig; also tagged PatchHeader)
  ├─ Tier 4: ResolveBinaryPathAsync → IPatchRepository.FindBinaryFingerprintsByCveAsync
  │          + BinaryFingerprintFactory.MatchBestAsync          → EvidenceType.BinaryFingerprint
  └─ BackportProofGenerator.CombineEvidence → ProofBlob (Attestor.ProofChain)

Notes grounded in the current source:

• Each evidence Data payload is canonicalised and hashed with StellaOps.Canonical.Json.CanonJson.Sha256Prefixed (so DataHash is sha256:-prefixed).
• The repository interfaces (IDistroAdvisoryRepository, ISourceArtifactRepository, IPatchRepository) and their DTOs (DistroAdvisoryDto, ChangelogDto, PatchHeaderDto, PatchSigDto) are declared in BackportProofService.cs; the Postgres implementation lives in StellaOps.Concelier.ProofService.Postgres (PostgresPatchRepository, ProofServiceDbContext, migration 20251223000001_AddProofEvidenceTables.sql).
• ResolveBinaryPathAsync currently returns null (PURL→binary resolution is a stub), so Tier 4 is only exercised when a binary path is supplied through that resolver.
• When no evidence is found, the service returns BackportProofGenerator.Unknown(...) rather than null.

The Attestor ProofChain consumes the Feedser assemblies directly via BackportProofGenerator (Tier1–Tier4 partial classes) and BinaryFingerprintEvidenceGenerator; StellaOps.Scanner.PatchVerification also references them.

6) Security & compliance

• Determinism: Hunk/patch hashing is order-independent (sorted hunk hashes); file and function lists are ordinal-sorted. Canonicalisation/sorted-key JSON is applied by the consumer (CanonJson in BackportProofService), not inside the Feedser libraries. The ExtractedAt timestamp fields are deliberately non-deterministic wall-clock stamps.
• Tamper evidence: SHA-256 content hashes for all signatures and hunks (hex-encoded, lower-cased). Proof evidence hashes are sha256:-prefixed canonical JSON. (Earlier “BLAKE3-256” claims were incorrect — no BLAKE3 is used.)
• No secrets: The libraries handle only public patch/binary data; the signal attachers do not hold credentials (data sources are injected).
• Offline capable: No outbound network calls inside the extractors or fingerprinters.

7) Performance targets (aspirational)

These targets are design goals, not asserted by any test or benchmark in the repo. Note in particular that the current implementation does not stream large binaries — SimplifiedTlshFingerprinter.ExtractAsync(string) calls File.ReadAllBytesAsync and operates on the full buffer.

• Patch extraction: < 10ms for a typical unified diff (< 1000 lines).
• Binary fingerprinting: < 100ms for a 10 MB ELF binary.
• Memory: full-file buffering today; streaming is a future goal.
• Parallelism: extractors are stateless/static; BinaryFingerprintFactory.ExtractAllAsync and MatchBestAsync fan out with Task.WhenAll, and AttachBatchAsync runs lookups concurrently.

8) Observability

Not implemented. No feedser.* metrics are emitted anywhere in the codebase, and the Feedser libraries declare no Meter / Counter / Histogram / ActivitySource. The metric names listed in earlier drafts (feedser.hunk_extraction_duration_seconds, feedser.binary_fingerprint_duration_seconds, feedser.fingerprint_match_score, feedser.evidence_tier_confidence) do not exist — treat them as a proposed/aspirational design, not current behaviour.

What is actually present:

• EpssSignalAttacher / KevSignalAttacher log debug/error lines via ILogger and emit a SignalUpdatedEvent per lookup.
• TimelineSignalEventEmitter emits a feedser.signal.updated timeline event (event kind, not a metric) through ITimelineEventEmitter with correlation id feedser.signal:{signalType}:{cve|purl}.
• BackportProofService emits structured ILogger information/warning entries per tier and per generated proof.

9) Testing matrix

Actual test projects (src/Concelier/__Tests/):

• StellaOps.Feedser.Core.Tests
  • HunkSigExtractorTests — diff parsing, multi-hunk/multi-file extraction, PatchSigId/HunkHash/AffectedFiles assertions.
  • FunctionSignatureExtractorTests — per-language signature extraction.
  • Signals/EpssSignalAttacherTests, Signals/KevSignalAttacherTests, Signals/SignalAttacherServiceExtensionsTests — attach success/not-found/failure states and DI wiring (incl. fail-closed emitter behaviour).
• StellaOps.Feedser.BinaryAnalysis.Tests
  • BinaryFingerprintTests, FingerprintMetadataTests, FingerprintMatchResultTests — record/enum shape, required-property, and value-range assertions. (These exercise the model contracts; they do not yet assert end-to-end similarity scores for known binary pairs.)

Gaps relative to earlier drafts: there are no dedicated cross-run determinism tests or performance/benchmark tests for these libraries in the current tree.

10) Historical note

The advisory-aggregation service was formerly named “Feedser” and is now Concelier (see ../concelier/architecture.md). After the rename:

• Concelier is the advisory-aggregation service (under src/Concelier/, with WebService/connectors/persistence).
• The StellaOps.Feedser.*name survives only on the two evidence-extraction libraries documented here, which physically live inside src/Concelier/ and have no host of their own.

Related Documentation

• Concelier architecture: ../concelier/architecture.md
• Attestor ProofChain: ../attestor/architecture.md
• Backport proof system: ../../features/checked/attestor/four-tier-backport-detection-system.md, ../../features/checked/concelier/4-tier-backport-evidence-resolver.md