Scenario: several instances and regions

Run a service as several processes to share load and survive the loss of one. Put instances in more than one region to keep serving when a whole site stops. No load balancer is needed: each instance connects to the gateway, and the gateway chooses an instance per request.

The worked example runs one service three times: inventory-01 and inventory-02 in region demo, the gateway’s region, and inventory-03 in region eu-west. Use a service of your own, built like the service of Getting started: only its registration below matters for this page, and the example calls its GET /items endpoint; put a path of your service instead. Before each release, an automated run stops the two demo instances and checks that eu-west serves.

Configure the instances

The service reads its region and instance identifier from configuration:

builder.Services.AddStellaMicroservice(options =>
{
    options.ServiceName = "inventory";
    options.Version = "1.0.0";
    // The region of this instance (Router:Region); the gateway prefers instances in its own region.
    options.Region = builder.Configuration["Router:Region"] ?? "demo";
    options.InstanceId = builder.Configuration["Router:InstanceId"] ?? $"inventory-{Environment.MachineName}";
    // Below the gateway's degraded threshold (Gateway:Health:DegradedThreshold; 20 s in effect by default).
    options.HeartbeatInterval = TimeSpan.FromSeconds(10);
    options.Routers =
    [
        new RouterEndpointConfig { Host = gatewayHost, Port = gatewayPort, TransportType = TransportType.Tcp }
    ];
});

The gateway’s region is Gateway:Node:Region in its appsettings.json (demo in the sample). Start three instances of the service, each in its own terminal, in the service project’s directory:

dotnet run -- --Router:InstanceId=inventory-01
dotnet run -- --Router:InstanceId=inventory-02
dotnet run -- --Router:InstanceId=inventory-03 --Router:Region=eu-west

List them with a GET request to the gateway’s instance list, /api/v1/router/instances, with a token that carries router:routing:read.

Expected result: three inventory instances, two with "region":"demo" and one with "region":"eu-west", each with its endpoint count, transport and status.

How the gateway chooses an instance

For each request, among the instances that serve the endpoint:

  1. Version. A request that names a version goes only to instances of exactly that version. It names one with the header X-Api-Version: 1.0.0, or with Accept: application/vnd.stellaops.v1.0.0+json. A request that names none can reach any version. A pinned request also takes that version’s timeout. Each version keeps its own claims: a request is authorized against the claims of the version it is routed to, and claim overrides from Authority apply to every version. A version that no connected instance serves is answered 503 No instances available, naming the requested version.
  2. Health. The healthy instances; when none is healthy and Gateway:Routing:AllowDegradedInstances is true (default), the degraded ones. Never a draining or unhealthy instance.
  3. Region. With Gateway:Routing:PreferLocalRegion (default true), the instances in the gateway’s region first, then the instances in the gateway’s neighbour regions (Gateway:Node:NeighborRegions, a list), then the others.
  4. Within the chosen group, the instance with the lowest measured ping, then the most recent heartbeat. Only instances tied on both are chosen by weight, then at random.

When no instance is left, the gateway answers 503 with No instances available.

Failover

Stop inventory-01 and inventory-02. The gateway removes them from the instance list when their connections close, and routes to inventory-03:

curl -i http://localhost:5080/items -H "Authorization: Bearer $TOKEN"

Expected result: 200, served by the eu-west instance. Start a demo instance again and the gateway prefers it as soon as it connects.

An instance that stops without closing its connection (a crash, a network cut) stays listed until its heartbeats are late: degraded after Gateway:Health:DegradedThreshold, unhealthy after Gateway:Health:StaleThreshold (20 and 30 seconds in effect with the defaults). Keep the service’s HeartbeatInterval below the degraded threshold.

Weights

Each instance advertises Weight (default 100). Weights decide only between instances that tie on measured ping and heartbeat time, so they are a tie-breaker, not a traffic split. An administrator can override weights at run time through the gateway’s routing-weight endpoints (router:routing:read to read them, router:routing:write to change them).

Rolling updates

  1. Start the new version’s instances next to the old ones.
  2. Stop one old instance at a time. Each one reports Draining, finishes its requests in flight within DrainDeadline, and disconnects.
  3. Callers that send no version reach old and new instances during the change. Callers that send X-Api-Version (or the Accept form) reach only instances of that version, with that version’s timeout and claims. A caller that sends no version can reach either version, so until the old version is gone it needs the claims that both versions require.

Pitfalls