Doctor — Architecture
Source ownership, 2026-10-03. PCD-253 withdrew the uncompiled AdvisoryAI drafts under
src/Doctor/__Libraries/StellaOps.Doctor/and the oldIDoctorPlugin.cscontract undersrc/Doctor/__Plugins/. The maintained AdvisoryAI implementation remains insrc/__Libraries/StellaOps.Doctor/AdvisoryAI/; current plugin abstractions and checks keep their existing projects. See the per-file source disposition.
Verified against commit
942a18727c(2026-09-26, DOC-2 of SPRINT_20260926_001). Re-verify by reading the evidence paths listed in the Evidence section.
Audience: operators and SREs diagnosing a Stella Ops deployment, and engineers writing checks.
Doctor is the diagnostic framework: check contracts, the check engine and its plugins, the per-service doctor-check/v1 SDK, and the stella doctor CLI. There is no Doctor deployable. The central doctor-web host was retired on 2026-08-17 (SPRINT_20260722_009 DOC-5 stage 4) together with its compose keys, gateway routes, image publish key, host alias and the scheduler doctor job kind. The description of that host (HTTP API, scopes, report storage, mounted plugin loader, configuration) is kept as a frozen record in docs-archive/modules/doctor/architecture.md.
1) Where Doctor runs today
| Surface | What runs | Source |
|---|---|---|
| Per-service checks | Each adopting service hosts its own doctor-check/v1 checks at GET /doctor/<service>/checks, guarded by ops.health. | src/__Libraries/StellaOps.Doctor.Plugin.Abstractions/ (AddServiceDoctorChecks, MapServiceDoctorEndpoints) |
| Registry and aggregate | Platform records which checks each service claims and serves the combined view the Console reads (GET /api/v1/platform/doctor/aggregate). | src/Platform/StellaOps.Platform.WebService/Endpoints/DoctorCapabilityEndpoints.cs; see Platform architecture |
| Estate packs | Estate-level diagnostics run as signed packs scheduled by JobEngine; the estate worker compiles only StellaOps.Doctor.Contracts, never the engine. | src/JobEngine/StellaOps.JobEngine.EstateWorker/; design in estate-pack-design.md |
| CLI | stella doctor runs the engine in-process with the Core, Database and BinaryAnalysis plugins. | src/Cli/StellaOps.Cli/Program.cs, src/Cli/StellaOps.Cli/Commands/DoctorCommandGroup.cs |
Per-service SDK (doctor-check/v1)
- Contract.
ServiceDoctorCheckResult.ContractVersionValueisdoctor-check/v1(Contract.cs). The response carriesservice,contractVersionandchecks[]withcheckId,category,severity,healthy,messageandevidence. - Route.
ServiceDoctorRoute.ChecksPathFor(service)is/doctor/<service>/checks; the service name must be a lowercase DNS-style label. The flat/doctor/checkspath was retired without an alias (SPRINT_20260722_023 GRA-11). An unprovisioned host answers 503 instead of failing composition. - Standard checks.
AddServiceDoctorChecksregistersdoctor.standard.db.connection,doctor.standard.db.migration-statusanddoctor.standard.db.size-budgetagainst the host’s own data source (StandardChecks.cs), plus optional owned-state budgetsdoctor.standard.state.size-budget.<state>(OwnedStateChecks.cs; Valkey measurement insrc/__Libraries/StellaOps.Doctor.Valkey). Services add their own checks beside these. - Registration.
AddServiceDoctorRegistrationbindsDoctor:Registration(Enabled,PlatformBaseAddress,SelfEndpoint,HeartbeatIntervaldefault 1 minute,RetryIntervaldefault 30 seconds) and registers with Platform through the named clientdoctor-registrationunderplatform:doctor:register(ServiceDoctorRegistrar.cs). Registration is off unless enabled; the release bundle’s.env.exampleenables it per service with<SERVICE>_DOCTOR_REGISTRATION_ENABLED=true. - Adoption. The recipe for a service is in the adoption guide; the contract design is in plugin-hosting-design.md.
Release estate owner adoption (CCP-071)
check.release.environment.matches-approved is served by the existing Release Orchestrator IServiceDoctorCheck host, using its own repositories and the Watch classification. Its operational output contains owner-wide counts only. Unbaselined state is unhealthy/Critical; absent, failed, unknown, unevaluated or empty data is never healthy. The old Release plugin check and tests are frozen under src/__Obsoleted/Doctor/; the other four Release plugin checks remain. The neutral SDK is unchanged and no release family is added to the five-family estate-doctor worker profile. See the check article for the owner pull, the Platform registration settings, the identity grant (Authority Startup migration 040) and the activation checks. Source adoption and host tests do not establish live activation.
Withdrawn WebService source
The WebService source was withdrawn to src/__Obsoleted/Doctor/StellaOps.Doctor.WebService with its tests on 2026-09-27 (PCD-73, CoC 15.4). It had no compose file, release-bundle service, gateway route or image publish key since DOC-5 stage 4, and must not be restored to repair a leftover container (src/Doctor/README.md). Its persistence library is also withdrawn and unsupported (PCD-252, 2026-10-03): no live project referenced its marker type or embedded migration resources. The files are frozen unchanged at src/__Obsoleted/Doctor/__Libraries/StellaOps.Doctor.Persistence/ under CoC 15.4, and the unadopted persistence exception is removed. DOC-5 stage 5 recorded the drop of the legacy doctor schema and scheduler.doctor_trends on 2026-08-24. Current per-service checks, the Platform aggregate, JobEngine estate packs and the CLI use their existing owners; none adopts this retired report/remediation store. Recheck the withdrawal with the architecture pack’s PersistenceAdoptionConformanceTests and ObsoletedIsolationConformanceTests. The standalone Doctor Scheduler is frozen under src/__Obsoleted/Doctor/StellaOps.Doctor.Scheduler/, and the Timestamping plugin under src/__Obsoleted/Doctor/__Plugins/ (timestamp health is the Evidence family’s doctor.evidence.timestamp.* checks).
Gated self-healing
Remediation execution follows ADR-026 Decision 5 and is implemented DB-free in src/__Libraries/StellaOps.Doctor.Remediation (RemediationGate, RemediationOptions). Nothing runs unless RemediationOptions.Enabled is set and the action id is listed in EnabledActionIds. Destructive actions additionally require an approval token, a dry-run preview, an active maintenance window, durable audit, the rate limit and the doctor:remediate scope; otherwise they fall back to manual guidance. The live entry point is stella doctor remediate (--action, --approval-token, --dry-run).
Air-gap awareness
DoctorPluginContextExtensions.IsAirgapped (src/__Libraries/StellaOps.Doctor/Plugins/DoctorPluginContextExtensions.cs) lets a check short-circuit external probes in sealed mode with the canonical skip reason air-gap mode: external probe disabled; IsLoopbackOrPrivateAddress keeps internal collectors probeable.
2) Plugin Architecture
The interfaces below are the canonical contracts in namespaces
StellaOps.Doctor.PluginsandStellaOps.Doctor.Models, compiled insrc/__Libraries/StellaOps.Doctor.Contracts(Plugins/IDoctorPlugin.cs,Plugins/IDoctorCheck.cs,Plugins/DoctorPluginContext.cs,Models/DoctorCheckResult.cs);StellaOps.Doctorforwards the types. A separate, simplerIDoctorPlugin(Name/Categories/RunChecksAsync) exists insrc/Doctor/__Plugins/StellaOps.Doctor.Plugins.Corefor legacy aggregation; the engine and all built-in plugins use the contracts shown here.
Core Interfaces
public interface IDoctorPlugin
{
string PluginId { get; } // e.g. "stellaops.doctor.integration"
string DisplayName { get; }
DoctorCategory Category { get; } // enum, not string
Version Version { get; }
Version MinEngineVersion { get; }
bool IsAvailable(IServiceProvider services);
IReadOnlyList<IDoctorCheck> GetChecks(DoctorPluginContext context);
Task InitializeAsync(DoctorPluginContext context, CancellationToken ct);
}
public interface IDoctorCheck
{
string CheckId { get; } // e.g. "check.integration.oci.referrers"
string Name { get; }
string Description { get; }
DoctorSeverity DefaultSeverity { get; }
IReadOnlyList<string> Tags { get; }
TimeSpan EstimatedDuration { get; }
bool CanRun(DoctorPluginContext context);
Task<DoctorCheckResult> RunAsync(DoctorPluginContext context, CancellationToken ct);
}
DoctorCategory is an enum (Core, Database, Security, Cryptography, Attestation, Docker, Integration, Observability, Notify, ServiceGraph, Sources, Verification, Authority, AI, Environment, Release, Scanner, Infrastructure).
Plugin Context
public sealed class DoctorPluginContext
{
public required IServiceProvider Services { get; init; }
public required IConfiguration Configuration { get; init; }
public required TimeProvider TimeProvider { get; init; }
public required ILogger Logger { get; init; }
public required string EnvironmentName { get; init; }
public string? TenantId { get; init; }
public IConfiguration PluginConfig { get; init; } // bound to Doctor:Plugins
// Factory for building results, and a static credential redactor.
public DoctorCheckResultBuilder CreateResult(string checkId, string pluginId, string category);
public static string Redact(string? value);
}
Check Results
The result type returned by a check is DoctorCheckResult (StellaOps.Doctor.Models). The simpler CheckResult record (CheckId/Severity/IsHealthy/Message/Metadata) is a separate dashboard projection produced by DoctorEngine.RunChecksAsync.
public sealed record DoctorCheckResult
{
public required string CheckId { get; init; }
public required string PluginId { get; init; }
public required string Category { get; init; }
public required DoctorSeverity Severity { get; init; }
public required string Diagnosis { get; init; }
public required Evidence Evidence { get; init; }
public IReadOnlyList<string>? LikelyCauses { get; init; }
public Remediation? Remediation { get; init; }
public string? VerificationCommand { get; init; }
public required TimeSpan Duration { get; init; }
public required DateTimeOffset ExecutedAt { get; init; }
}
public enum DoctorSeverity
{
Pass = 0, // Check passed successfully
Info = 1, // Informational (not a problem)
Warn = 2, // Warning condition that should be addressed
Fail = 3, // Failure (needs attention)
Skip = 4 // Skipped (not applicable in current context)
}
Evidence carries a Description, an IReadOnlyDictionary<string, string> Data, and an optional SensitiveKeys list. Remediation carries ordered Steps (each with a CommandType of Shell/Sql/Api/FileEdit/Manual), a SafetyNote, RequiresBackup, and an optional RunbookUrl.
Engine resilience (ADR-026 D8)
The executor (Engine/CheckExecutor.cs) and registry (Engine/CheckRegistry.cs) are hardened so a single misbehaving check or plugin can never crash, stall, or silently shrink a run. See ADR-026 decisions D8 (engine resilience) and D6 (rejected/lost checks must not read as all-green).
CanRunisolation.check.CanRun(context)is invoked inside the per-check guarded region. A throwingCanRun(e.g. it dereferences an unregistered service or reads a missing config value) becomes a structuredFailresult whose diagnosis/evidence namesCanRunas the failing stage (Evidence.Data["Stage"] = "CanRun"), instead of escapingExecuteCheckAsyncand aborting the wholeParallel.ForEachAsyncbody. A normalfalsereturn still yields the existingSkip“not applicable” result.Per-check timeout sizing. The effective per-check timeout is derived from
IDoctorCheck.EstimatedDurationasclamp(max(globalTimeout, EstimatedDuration × 3), globalTimeout, 5 min). A check with an unset/zero estimate gets exactly the globalDoctorRunOptions.Timeout(unchanged behaviour); a legitimately slow check (large estimate) is granted proportional headroom so it is not falsely timed out, bounded by a 5-minute ceiling so a garbage estimate cannot pin a slot indefinitely.Cooperative-timeout / blocking-probe watchdog.
CancelAfteronly signals the token; a check that ignores itsCancellationTokenand blocks (busy loop, blocking sync I/O) would otherwise pin one of theParallelismslots for the full hang. The executor dispatchesRunAsyncto the thread pool and races it againstTask.Delay(effectiveTimeout + grace). A check that honours its token unwinds within the grace window and produces the existing clean-timeoutFail. A check that ignores its token loses the race and is reported as a distinct blocking-probeFail(diagnosis “blocking probe (ignored cancellation)”,Evidence.Data["BlockingProbe"] = "true") without awaiting the stuck task — it is left to run orphaned, and a structured warning plus thedoctor.check.blocking_probemetric (meterStellaOps.Doctor.Engine, tagged bycheck_id/plugin_id) is emitted. Known limitation: modern.NET cannot safely abort a thread that ignores its token, so the contract is detect-and-report — the orphaned probe still occupies its parallel slot until it returns; the run as a whole completes instead of hanging.Discovery-time timeout. Per-plugin
IsAvailableandGetChecksdiscovery calls each run under a boundedDoctorRunOptions.DiscoveryTimeout(default 10s, distinct from the per-checkTimeout). The synchronous call is dispatched to the thread pool and awaited up to the discovery timeout; a plugin that blocks in either is skipped for that run rather than stalling discovery before any check executes, and a hang in one plugin does not delay the others. A timed-out plugin is not silently dropped: the registry injects a synthetic failing check (check.core.plugins.discovery-timeout.<pluginId>, severityFail) so the run goes red — closing the D6 “rejected plugin → checks vanish → all green” hole. The existing per-plugintry/catchfor throwing plugins is preserved.Opt-in probe retry (
ProbeRetry). A check that performs a single network/IPC attempt and fails it on a transient blip (a dropped connection, a momentary DNS hiccup) would otherwise report a falseFail. The sharedStellaOps.Doctor.Resilience.ProbeRetry.ExecuteAsync<T>(probe, options, timeProvider, jitter, logger, ct)helper wraps an outbound probe in a bounded retry-with-full-jitter, distinct from and composed within the per-check timeout (the timeout bounds the whole check; retry repeats the probe call inside it). It is opt-in — a non-idempotent / side-effecting probe must not opt in — and is a standalone primitive that does not depend onIDoctorCheckor theIntegrationCheckbase; the D2IntegrationCheckbase (sprintSPRINT_20260608_005_*) advertises a retry-policy descriptor and wraps its probe body in this helper. Semantics:Retries only thrown transient failures, classified via
CheckExceptionLogging.IsTransientProbeException(HTTP / socket / IO / probe-timeout). A non-transient exception (JsonException/contract/programming error) is rethrown on the first attempt and never retried.A successful return is returned immediately — even a “Fail-shaped” value. The helper retries thrown transient exceptions of the probe call, not result severities.
Bounded attempts (
ProbeRetryOptions.MaxAttempts, default 3) with exponential backoff (BaseDelay, default 200 ms; doubling, capped byMaxDelay, default 5 s) and full jitter — the delay is sampled uniformly in[0, ceiling]. Jitter randomness comes from an injectedIJitterSource(CryptoJitterSourcein production, a fixed/fake source in tests — neverRandom.Shared, per the Doctor coding standard), and backoff delays use the injectedTimeProvider(Task.Delay(delay, timeProvider, ct)) so tests run on virtual time and stay fast/offline.Cancellation from the supplied token propagates as
OperationCanceledExceptionand is never treated as a transient retry (even thoughTaskCanceledExceptionis otherwise in the transient set); retrying stops immediately on cancel.On attempt exhaustion the last transient exception is rethrown (stack preserved) so the caller’s check turns it into a
Failvia the existing engine/timeout path. The returnedProbeRetryResult<T>carries the attempt count so the check can record it in result evidence; the total retry budget is bounded by the caller’sct(the engine ties it to the DOC-ENG-003 effective per-check timeout, so it never exceeds the per-check ceiling).Current DOC-OBS-005 audit. The
IntegrationCheckbase supplies the default bounded retry policy tocheck.db.connection,check.postgres.schemas.present,check.servicegraph.valkey,check.router.messaging.transport,check.servicegraph.nats,check.servicegraph.gateway-router,check.servicegraph.sibling-reachability,check.integration.oci.registry,check.integration.oci.credentials,check.integration.oci.pull,check.integration.oci.referrers,check.integration.git,check.integration.oidc,check.security.authority-signing-key, andcheck.smremote.remote-hsm.connectivity. The Observability OTLP and Prometheus reachability checks override a shorter two-attempt policy for local collector/scrape probes. The log-shipping check intentionally stays single-attempt because it reads one local forwarder-state snapshot and reports the returned drain/backlog state; retrying a returnedstalled/unavailablesnapshot would hide operator-visible state.check.integration.oci.pushis also intentionally single-attempt because it initiates a registry upload session and cleanup; retry/fan-out would need a separate idempotency design before it can safely repeat that write-shaped probe.Tenant fan-out for tenant-scoped
IntegrationChecks. Tenant-scoped integration checks enumerateIDoctorTenantCatalogbefore probing. The library default catalog is read-only and local-configuration backed (Doctor:TenantCatalog:Tenants, withDoctor:RequiredTenants/Authority:RequiredTenantsfallbacks) for offline tests and embedded hosts. (The withdrawn Doctor WebService source (now frozen undersrc/__Obsoleted/Doctor/) replaced that seam with a read-onlyshared.tenantscatalog; no deployed host does so today.) If no catalog is registered, the catalog is empty, or exactly one tenant is cataloged, the check preserves historical single-run behavior and evidence shape. When two or more tenants are present, the base runs the probe once per cataloged tenant withDoctorPluginContext.TenantIdset, resolves tenant overrides before global config (Doctor:TenantOverrides:<tenant>:<key>thenTenants:<tenant>:<key>), and emits aggregate evidence:TenantCount,TenantResults,TenantProbeAttempts,TenantProbeFailures, and per-tenant severity/retry-attempt fields. Aggregate severity is worst-result: any tenantfailfails the aggregate, so a green default tenant no longer masks a failing second tenant.Current DOC-OBS-001 audit. Tenant fan-out is enabled for the tenant-scoped integration checks that already derive from
IntegrationCheck:check.db.connection,check.postgres.schemas.present,check.servicegraph.valkey,check.router.messaging.transport,check.integration.oci.registry,check.integration.oci.credentials,check.integration.oci.pull,check.integration.oci.referrers,check.integration.git,check.integration.oidc, andcheck.platform.identity-envelope. Platform/shared-state checks such as gateway/router, NATS, sibling reachability, circuit-breaker state, connection-pool state, Authority JWKS, and telemetry backends remain single-probe.check.integration.oci.pushalso remains outside this fan-out because its probe opens a registry upload session.
Bare-catch analyzer scope (DOCTOR0001, ADR-026 D2/D7)
The BareCatchAnalyzer (src/__Analyzers/StellaOps.Doctor.Analyzers/BareCatchAnalyzer.cs) flags unfiltered catch { } / catch (Exception) so probe failures cannot be silently swallowed (the canonical fix is the catch (Exception ex) when (CheckExceptionLogging.Is…Exception(ex)) pattern; see §4 Check Patterns and src/Doctor/AGENTS.md).
- Scope (DOC-ENG-005): the analyzer fires inside types implementing
StellaOps.Doctor.Plugins.IDoctorCheckand inside the non-check delegate-helper classes a check directly invokes (a method call, member access, or construction of a source-defined non-check type from within a check). A swallow in a probe-helper the analyzer previously could not see is now caught. Detection is sound — only helpers provably reached from a check fire, so unrelated handlers/utilities in the same assembly stay silent — and compilations that do not reference theIDoctorChecksymbol early-exit at zero added cost, keeping non-Doctor csprojs unaffected. - Severity: Error. DOC-DEBT-001 completed the Phase-2 flip on 2026-06-11:
DOCTOR0001now ships asDiagnosticSeverity.Errorand is absent from theWarningsNotAsErrorsshim insrc/Directory.Build.props. The final blocker was the remaining in-process AI plugin raw catches (LocalInferenceCheckandOllamaProviderCheck), now narrowed to typed parse catches withCheckExceptionLogging. A full Doctor-surface build (Plugins.*, mountedPlugin.*, engine, WebService, and the since-frozen Scheduler) passed with DOCTOR0001 at Error before the shim was removed. No project-level suppression should be added to bypass this gate. - Deferred — swallow-to-
Pass: a rule to flag returning aPass-shapedDoctorCheckResultfrom inside acatch(the ADR-026 D7 fake-green) was scoped but deferred — distinguishing a fabricatedPassfrom a legitimately-justified one is a semantic judgment a syntactic analyzer cannot make without false positives. It stays enforced by code review and the no-hardcoded-Passtest discipline. - Timestamping exception closed (2026-09-07, TA-9): the plugin that declared its own local
IDoctorCheckfor the mounted legacy facade is frozen undersrc/__Obsoleted/, so the canonical-interface gate no longer carries an uncovered check family; the Evidence timestamp checks implementStellaOps.Doctor.Plugins.IDoctorCheck.
3) Plugin libraries
Check families are grouped in plugin libraries. In-process families live under src/__Libraries/StellaOps.Doctor.Plugins.* (Attestation, Authority, Core, Core.Worker, Cryptography, Database, Docker, Integration, Observability, Security, ServiceGraph, Verification); further families live under src/Doctor/__Plugins/StellaOps.Doctor.Plugin.* (Agent, BinaryAnalysis, Compliance, Environment, EvidenceLocker, Notify, Operations, Release, Scanner, Storage). The integration catalog and its check bindings are in src/__Libraries/StellaOps.Doctor.IntegrationChecks.
No deployed host composes the full catalog: the CLI registers Core, Database and BinaryAnalysis, and only the WebService, withdrawn to src/__Obsoleted/Doctor/ by PCD-73, registered the remaining families. Check articles are required for every IDoctorCheck (docs/doctor/articles/<plugin>/<check-slug>.md, see src/Doctor/AGENTS.md). The OCI registry checks are described in registry-checks.md.
4) Check Patterns
Non-Destructive Probing
Registry checks use non-destructive operations:
// Pull check: HEAD request only (no data transfer)
var response = await client.SendAsync(new HttpRequestMessage(HttpMethod.Head, manifestUrl), ct);
// Push check: Start upload then immediately cancel
var uploadResponse = await client.PostAsync(uploadsUrl, null, ct);
if (uploadResponse.StatusCode == HttpStatusCode.Accepted)
{
var location = uploadResponse.Headers.Location;
await client.DeleteAsync(location, ct); // Cancel upload
}
Capability Detection
Registry capability probing sequence:
1. GET /v2/ → Extract OCI-Distribution-API-Version header
2. GET /v2/{repo}/referrers/{digest} → Check referrers API support
3. POST /v2/{repo}/blobs/uploads/ → Check chunked upload support
└─ DELETE {location} → Cancel upload session
4. POST /v2/{repo}/blobs/uploads/?mount=...&from=... → Check cross-repo mount
5. OPTIONS /v2/{repo}/manifests/{ref} → Check delete support (Allow header)
6. OPTIONS /v2/{repo}/blobs/{digest} → Check blob delete support
Evidence Collection
All checks collect structured evidence via the fluent DoctorCheckResultBuilder created from the context. WithEvidence(description, configure) takes the evidence description plus a callback over EvidenceBuilder:
var result = context.CreateResult(CheckId, PluginId, Category)
.Pass("Registry authentication successful")
.WithEvidence("OCI registry authentication probe", eb => eb
.Add("registry_url", registryUrl)
.Add("auth_method", "bearer")
.Add("response_time_ms", (int)elapsed.TotalMilliseconds)
.AddSensitive("token_preview", DoctorPluginContext.Redact(token)))
.Build();
EvidenceBuilder.Add has overloads for string, bool, int, long, and double. AddConnectionString redacts the value automatically and marks the key sensitive; AddSensitive marks an already-redacted value sensitive.
Credential Redaction
Sensitive values are redacted by the static DoctorPluginContext.Redact helper (Plugins/DoctorPluginContext.cs). It shows the first 3 and last 3 characters for values longer than 8 characters; shorter or empty values collapse to a fixed token:
public static string Redact(string? value)
{
if (string.IsNullOrWhiteSpace(value))
return "[EMPTY]";
if (value.Length <= 8)
return "[REDACTED]";
return $"{value[..3]}***{value[^3..]}";
}
// "mysecretpassword" → "mys***ord"
// "short" → "[REDACTED]"
5) CLI Integration
The stella doctor command group (src/Cli/StellaOps.Cli/Commands/DoctorCommandGroup.cs) runs the engine in-process. The CLI registers the Core, Database and BinaryAnalysis plugins (src/Cli/StellaOps.Cli/Program.cs). Subcommands: run (also the default action), list, export, fix and remediate.
stella doctor # run all checks (default action == `doctor run`)
stella doctor --mode quick # quick | normal (default) | full
stella doctor --category Database --tag quick # filter by category or tag (repeatable)
stella doctor --check <check-id> # run one check
stella doctor --format json --output report.json
stella doctor --parallel 8 --timeout 60 --fail-on-warn
stella doctor --watch --interval 60 # continuous monitoring
stella doctor list --category Database
stella doctor export --output bundle.zip --include-logs --log-duration 4h
stella doctor fix --from report.json --apply # dry-run unless --apply
stella doctor remediate --check <check-id> --action <action-id> --dry-run
Exit codes follow the run’s overall severity: failures map to CliExitCodes.DoctorFailed; warnings map to CliExitCodes.DoctorWarning only with --fail-on-warn.
6) Extensibility
Creating a Custom Check
public sealed class MyCustomCheck : IDoctorCheck
{
public string CheckId => "check.custom.mycheck";
public string Name => "My Custom Check";
public string Description => "Validates custom integration";
public DoctorSeverity DefaultSeverity => DoctorSeverity.Fail;
public IReadOnlyList<string> Tags => ["custom", "integration"];
public TimeSpan EstimatedDuration => TimeSpan.FromSeconds(5);
public bool CanRun(DoctorPluginContext context)
{
// Return false if preconditions not met
return context.Configuration["Custom:Enabled"] == "true";
}
public async Task<DoctorCheckResult> RunAsync(DoctorPluginContext context, CancellationToken ct)
{
var builder = context.CreateResult(CheckId, "custom", "Integration");
try
{
// Perform check logic
var result = await ValidateAsync(context, ct);
if (result.Success)
{
return builder
.Pass("Custom validation successful")
.WithEvidence("Custom validation", eb => eb.Add("detail", result.Detail))
.Build();
}
return builder
.Fail("Custom validation failed")
.WithCauses("Configuration is invalid")
.WithRemediation(rb => rb
.AddManualStep(1, "Check configuration", "Verify Custom:Setting is correct")
.WithRunbookUrl("https://docs.stella-ops.org/runbooks/custom-check"))
.Build();
}
catch (Exception ex)
{
return builder
.Fail($"Check failed with error: {ex.Message}")
.WithEvidence("Check exception", eb => eb.Add("exception_type", ex.GetType().Name))
.Build();
}
}
}
Creating a Custom Plugin
public sealed class MyCustomPlugin : IDoctorPlugin
{
public string PluginId => "custom";
public string DisplayName => "Custom Checks";
public DoctorCategory Category => DoctorCategory.Integration; // enum
public Version Version => new(1, 0, 0);
public Version MinEngineVersion => new(1, 0, 0);
public bool IsAvailable(IServiceProvider services) => true;
public IReadOnlyList<IDoctorCheck> GetChecks(DoctorPluginContext context) =>
[
new MyCustomCheck(),
new AnotherCustomCheck()
];
public Task InitializeAsync(DoctorPluginContext context, CancellationToken ct)
{
// Optional initialization
return Task.CompletedTask;
}
}
7) Telemetry
The engine emits the StellaOps.Doctor.Engine meter with the doctor.check.blocking_probe counter (tags check_id / plugin_id) from CheckExecutor when a check ignores its cancellation token past the effective timeout plus grace window. Discovery timeouts and CanRun failures surface as visible Fail results and Warning logs. Per-service checks are exposed over HTTP only; they emit no Doctor-specific metrics.
8) Related Documentation
- Adoption guide, plugin hosting design, estate pack design
- Owned-state operator surfaces, storage budget sizing, Valkey owned-state measurement
- Registry diagnostic checks
- Check articles:
docs/doctor/articles/ - ADR-026
Evidence
| Claim | Source |
|---|---|
| Deployable retired; retained source | devops/compose/docker-compose.stella-services.yml (RETIRED note), src/Doctor/README.md, src/Doctor/AGENTS.md |
| Per-service SDK: contract, route, standard and owned-state checks, registration | src/__Libraries/StellaOps.Doctor.Plugin.Abstractions/Contract.cs, ServiceDoctorHosting.cs, StandardChecks.cs, OwnedStateChecks.cs, ServiceDoctorRegistrar.cs |
| Adopting hosts | AddServiceDoctorChecks(...) call sites in each service’s Program.cs (for example src/Platform/StellaOps.Platform.WebService/Program.cs, src/JobEngine/StellaOps.JobEngine.WebService/Program.cs) |
| Registration defaults in the bundle | devops/release/bundle/.env.example (*_DOCTOR_REGISTRATION_ENABLED) |
| Registry and aggregate | src/Platform/StellaOps.Platform.WebService/Endpoints/DoctorCapabilityEndpoints.cs, Services/DoctorAggregateService.cs |
| Contracts | src/__Libraries/StellaOps.Doctor.Contracts/ |
| Engine, resilience, telemetry meter | src/__Libraries/StellaOps.Doctor/Engine/CheckExecutor.cs, Engine/CheckRegistry.cs |
| Analyzer | src/__Analyzers/StellaOps.Doctor.Analyzers/BareCatchAnalyzer.cs |
| Remediation gate | src/__Libraries/StellaOps.Doctor.Remediation/RemediationGate.cs, RemediationOptions.cs |
| Air-gap helpers | src/__Libraries/StellaOps.Doctor/Plugins/DoctorPluginContextExtensions.cs |
| Estate worker closure | src/JobEngine/StellaOps.JobEngine.EstateWorker/StellaOps.JobEngine.EstateWorker.csproj |
| CLI | src/Cli/StellaOps.Cli/Commands/DoctorCommandGroup.cs, src/Cli/StellaOps.Cli/Program.cs |
| Plugin libraries | src/__Libraries/StellaOps.Doctor.Plugins.*, src/Doctor/__Plugins/StellaOps.Doctor.Plugin.* |
