Surface.FS Workflow Guide

Version: 1.0 (2025-11-28) · Audience: Scanner Worker/WebService integrators, Zastava operators, Offline Kit builders

Overview

Surface.FS is the content-addressable storage layer that holds Scanner-derived artefacts (layer fragments, EntryTrace graphs, SBOM fragments) in Stella Ops. This guide walks the end-to-end workflow from artefact generation through storage to consumption, and covers how the same artefacts move through an air-gapped Offline Kit.

Workflow Stages

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Scanner Worker │───▶│  Surface.FS     │───▶│  Consumers      │
│  - Scan image   │    │  - Store manifest│   │  - WebService   │
│  - Generate     │    │  - Store payload │   │  - Zastava      │
│    artefacts    │    │  - Local cache   │   │  - CLI          │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                      │                      │
         ▼                      ▼                      ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Generate:      │    │  Store:         │    │  Consume:       │
│  - Layer frags  │    │  - RustFS/S3    │    │  - Report API   │
│  - EntryTrace   │    │  - Local disk   │    │  - Drift detect │
│  - SBOM frags   │    │  - Offline kit  │    │  - Rescan plan  │
└─────────────────┘    └─────────────────┘    └─────────────────┘

Stage 1: Artefact Generation (Scanner Worker)

1.1 Configure Surface.FS

// In Scanner Worker startup
builder.Services.AddSurfaceFileCache();
builder.Services.AddSurfaceManifestStore();

Environment variables (see Surface.Env guide):

SCANNER_SURFACE_FS_ENDPOINT=http://rustfs:8080
SCANNER_SURFACE_FS_BUCKET=surface-cache
SCANNER_SURFACE_CACHE_ROOT=/var/lib/stellaops/surface
SCANNER_SURFACE_TENANT=default

1.2 Generate and Publish Artefacts

public async Task<ScanResult> ExecuteScanAsync(ScanJob job, CancellationToken ct)
{
    // 1. Run analyzers to generate artefacts
    var layerFragments = await AnalyzeLayersAsync(job.Image, ct);
    var entryTrace = await AnalyzeEntryPointsAsync(job.Image, ct);
    var sbomFragments = await GenerateSbomAsync(job.Image, ct);

    // 2. Create manifest document
    var manifest = new SurfaceManifestDocument
    {
        Schema = "stellaops.surface.manifest@1",
        Tenant = _environment.Settings.Tenant,
        ImageDigest = job.Image.Digest,
        ScanId = job.Id,
        GeneratedAt = DateTimeOffset.UtcNow,
        Source = new SurfaceManifestSource
        {
            Component = "scanner.worker",
            Version = _version,
            WorkerInstance = Environment.MachineName,
            Attempt = job.Attempt
        },
        Artifacts = new List<SurfaceManifestArtifact>()
    };

    // 3. Add artefacts to manifest
    foreach (var fragment in layerFragments)
    {
        var payloadUri = await _manifestWriter.StorePayloadAsync(
            fragment.Content,
            "layer.fragments",
            ct);

        manifest.Artifacts.Add(new SurfaceManifestArtifact
        {
            Kind = "layer.fragments",
            Uri = payloadUri,
            Digest = fragment.Digest,
            MediaType = "application/vnd.stellaops.layer-fragments+json",
            Format = "json",
            SizeBytes = fragment.Content.Length
        });
    }

    // 4. Publish manifest
    var result = await _manifestWriter.PublishAsync(manifest, ct);

    _logger.LogInformation(
        "Published manifest {Digest} with {Count} artefacts",
        result.Digest,
        manifest.Artifacts.Count);

    return new ScanResult
    {
        ManifestUri = result.Uri,
        ManifestDigest = result.Digest
    };
}

1.3 Cache EntryTrace Results

public async Task<EntryTraceGraph?> GetOrComputeEntryTraceAsync(
    ImageReference image,
    EntryTraceOptions options,
    CancellationToken ct)
{
    // Create deterministic cache key
    var cacheKey = new SurfaceCacheKey(
        @namespace: "entrytrace.graph",
        tenant: _environment.Settings.Tenant,
        digest: ComputeOptionsHash(options, image.Digest));

    // Try cache first
    var cached = await _cache.TryGetAsync<EntryTraceGraph>(cacheKey, ct);
    if (cached is not null)
    {
        _logger.LogDebug("EntryTrace cache hit for {Key}", cacheKey);
        return cached;
    }

    // Compute and cache
    var graph = await ComputeEntryTraceAsync(image, options, ct);
    await _cache.SetAsync(cacheKey, graph, ct);

    return graph;
}

Stage 2: Storage (Surface.FS)

2.1 Manifest Storage Layout

<bucket>/
├── manifests/
│   └── <tenant>/
│       └── <digest[0..1]>/
│           └── <digest[2..3]>/
│               └── <digest>.json
└── payloads/
    └── <tenant>/
        └── <kind>/
            └── sha256/
                └── <digest[0..1]>/
                    └── <digest[2..3]>/
                        └── <digest>.json.zst

2.2 Local Cache Layout

<cache_root>/
├── manifests/          # Manifest JSON files
│   └── <tenant>/...
├── cache/              # Hot artefacts
│   └── <namespace>/
│       └── <tenant>/
│           └── <digest>
└── temp/               # In-progress writes

2.3 Manifest URI Format

cas://<bucket>/<prefix>/<tenant>/<digest[0..1]>/<digest[2..3]>/<digest>.json

Example:

cas://surface-cache/manifests/acme/ab/cd/abcdef0123456789...json

Stage 3: Consumption

3.1 WebService API

GET /api/v1/scans/{id}

Response includes Surface manifest pointer:

{
  "id": "scan-1234",
  "status": "completed",
  "surface": {
    "manifestUri": "cas://surface-cache/manifests/acme/ab/cd/...",
    "manifestDigest": "sha256:abcdef...",
    "artifacts": [
      {
        "kind": "layer.fragments",
        "uri": "cas://surface-cache/payloads/acme/layer.fragments/...",
        "digest": "sha256:123456...",
        "mediaType": "application/vnd.stellaops.layer-fragments+json"
      }
    ]
  }
}

3.2 Zastava Drift Detection

public async Task<DriftResult> DetectDriftAsync(
    string imageDigest,
    CancellationToken ct)
{
    // 1. Fetch baseline manifest
    var manifestUri = await _surfacePointerService.GetManifestUriAsync(imageDigest, ct);
    var manifest = await _manifestReader.TryGetByUriAsync(manifestUri, ct);

    if (manifest is null)
    {
        return DriftResult.NoBaseline();
    }

    // 2. Get EntryTrace artefact
    var entryTraceArtifact = manifest.Artifacts
        .FirstOrDefault(a => a.Kind == "entrytrace.graph");

    if (entryTraceArtifact is null)
    {
        return DriftResult.NoEntryTrace();
    }

    // 3. Compare with runtime
    var baseline = await _payloadStore.GetAsync<EntryTraceGraph>(
        entryTraceArtifact.Uri, ct);

    var runtime = await _runtimeCollector.CollectAsync(ct);

    return CompareGraphs(baseline, runtime);
}

3.3 Scheduler Rescan Planning

public async Task<RescanPlan> CreateRescanPlanAsync(
    string imageDigest,
    CancellationToken ct)
{
    // 1. Read manifest to understand what was scanned
    var manifest = await _manifestReader.TryGetByDigestAsync(imageDigest, ct);

    if (manifest is null || IsExpired(manifest))
    {
        return RescanPlan.FullRescan();
    }

    // 2. Check for layer changes
    var layerArtifact = manifest.Artifacts
        .FirstOrDefault(a => a.Kind == "layer.fragments");

    if (layerArtifact is not null)
    {
        var layers = await _payloadStore.GetAsync<LayerFragments>(
            layerArtifact.Uri, ct);

        var changedLayers = await DetectChangedLayersAsync(layers, ct);

        if (changedLayers.Any())
        {
            return RescanPlan.IncrementalRescan(changedLayers);
        }
    }

    return RescanPlan.NoRescanNeeded();
}

Offline Kit Workflow

Export (Online Environment)

# 1. Build offline kit with Surface manifests
python ops/offline-kit/build_offline_kit.py \
  --version 2025.10.0 \
  --include-surface-manifests \
  --output-dir out/offline-kit

# 2. Kit structure includes:
# offline/
#   surface/
#     manifests/
#       <tenant>/<digest[0..1]>/<digest[2..3]>/<digest>.json
#     payloads/
#       <tenant>/<kind>/sha256/<digest[0..1]>/<digest[2..3]>/<digest>.json.zst
#     manifest-index.json

Import (Air-Gapped Environment)

public async Task ImportOfflineKitAsync(
    string kitPath,
    CancellationToken ct)
{
    var surfacePath = Path.Combine(kitPath, "surface");
    var indexPath = Path.Combine(surfacePath, "manifest-index.json");

    var index = await LoadIndexAsync(indexPath, ct);

    foreach (var entry in index.Manifests)
    {
        // 1. Load and verify manifest
        var manifestPath = Path.Combine(surfacePath, entry.RelativePath);
        var manifest = await LoadManifestAsync(manifestPath, ct);

        // 2. Verify digest
        var computedDigest = ComputeDigest(manifest);
        if (computedDigest != entry.Digest)
        {
            throw new InvalidOperationException(
                $"Manifest digest mismatch: expected {entry.Digest}, got {computedDigest}");
        }

        // 3. Import via Surface.FS API
        await _manifestWriter.PublishAsync(manifest, ct);

        _logger.LogInformation(
            "Imported manifest {Digest} for image {Image}",
            entry.Digest,
            manifest.ImageDigest);
    }

    // 4. Import payloads
    foreach (var payload in index.Payloads)
    {
        var payloadPath = Path.Combine(surfacePath, payload.RelativePath);
        await _payloadStore.ImportAsync(payloadPath, payload.Uri, ct);
    }
}

Offline Operation

Once imported, Surface.FS consumers operate normally:

// Same code works online and offline
var manifest = await _manifestReader.TryGetByUriAsync(manifestUri, ct);
var payload = await _payloadStore.GetAsync(artifact.Uri, ct);

Configuration Reference

SurfaceManifestStoreOptions

SurfaceCacheOptions

Metrics

Troubleshooting

Manifest Not Found

1. Check tenant matches between writer and reader
2. Verify Surface.FS endpoint is reachable
3. Check bucket permissions
4. Review surface_manifest_published_total metric

Cache Miss Despite Expected Hit

1. Verify cache key components match (namespace, tenant, digest)
2. Check cache quota - eviction may have occurred
3. Review surface_manifest_cache_hit_total metric

Offline Import Failures

1. Verify manifest digest matches index
2. Check file permissions on import path
3. Ensure Surface.FS endpoint is writable
4. Review import logs for specific errors

References

• Surface.FS Design
• Surface.Env Design
• Surface.Validation Extensibility Guide
• Offline Kit Documentation