Scenario: several instances and regions
Run a service as several processes to share load and survive the loss of one. Put instances in more than one region to keep serving when a whole site stops. No load balancer is needed: each instance connects to the gateway, and the gateway chooses an instance per request.
The worked example runs one service three times: inventory-01 and inventory-02 in region demo, the gateway’s region, and inventory-03 in region eu-west. Use a service of your own, built like the service of Getting started: only its registration below matters for this page, and the example calls its GET /items endpoint; put a path of your service instead. Before each release, an automated run stops the two demo instances and checks that eu-west serves.
Configure the instances
The service reads its region and instance identifier from configuration:
builder.Services.AddStellaMicroservice(options =>
{
options.ServiceName = "inventory";
options.Version = "1.0.0";
// The region of this instance (Router:Region); the gateway prefers instances in its own region.
options.Region = builder.Configuration["Router:Region"] ?? "demo";
options.InstanceId = builder.Configuration["Router:InstanceId"] ?? $"inventory-{Environment.MachineName}";
// Below the gateway's degraded threshold (Gateway:Health:DegradedThreshold; 20 s in effect by default).
options.HeartbeatInterval = TimeSpan.FromSeconds(10);
options.Routers =
[
new RouterEndpointConfig { Host = gatewayHost, Port = gatewayPort, TransportType = TransportType.Tcp }
];
});
The gateway’s region is Gateway:Node:Region in its appsettings.json (demo in the sample). Start three instances of the service, each in its own terminal, in the service project’s directory:
dotnet run -- --Router:InstanceId=inventory-01
dotnet run -- --Router:InstanceId=inventory-02
dotnet run -- --Router:InstanceId=inventory-03 --Router:Region=eu-west
List them with a GET request to the gateway’s instance list, /api/v1/router/instances, with a token that carries router:routing:read.
Expected result: three inventory instances, two with "region":"demo" and one with "region":"eu-west", each with its endpoint count, transport and status.
How the gateway chooses an instance
For each request, among the instances that serve the endpoint:
- Version. A request that names a version goes only to instances of exactly that version. It names one with the header
X-Api-Version: 1.0.0, or withAccept: application/vnd.stellaops.v1.0.0+json. A request that names none can reach any version. A pinned request also takes that version’s timeout. Each version keeps its own claims: a request is authorized against the claims of the version it is routed to, and claim overrides from Authority apply to every version. A version that no connected instance serves is answered503No instances available, naming the requested version. - Health. The healthy instances; when none is healthy and
Gateway:Routing:AllowDegradedInstancesistrue(default), the degraded ones. Never a draining or unhealthy instance. - Region. With
Gateway:Routing:PreferLocalRegion(defaulttrue), the instances in the gateway’s region first, then the instances in the gateway’s neighbour regions (Gateway:Node:NeighborRegions, a list), then the others. - Within the chosen group, the instance with the lowest measured ping, then the most recent heartbeat. Only instances tied on both are chosen by weight, then at random.
When no instance is left, the gateway answers 503 with No instances available.
Failover
Stop inventory-01 and inventory-02. The gateway removes them from the instance list when their connections close, and routes to inventory-03:
curl -i http://localhost:5080/items -H "Authorization: Bearer $TOKEN"
Expected result: 200, served by the eu-west instance. Start a demo instance again and the gateway prefers it as soon as it connects.
An instance that stops without closing its connection (a crash, a network cut) stays listed until its heartbeats are late: degraded after Gateway:Health:DegradedThreshold, unhealthy after Gateway:Health:StaleThreshold (20 and 30 seconds in effect with the defaults). Keep the service’s HeartbeatInterval below the degraded threshold.
Weights
Each instance advertises Weight (default 100). Weights decide only between instances that tie on measured ping and heartbeat time, so they are a tie-breaker, not a traffic split. An administrator can override weights at run time through the gateway’s routing-weight endpoints (router:routing:read to read them, router:routing:write to change them).
Rolling updates
- Start the new version’s instances next to the old ones.
- Stop one old instance at a time. Each one reports
Draining, finishes its requests in flight withinDrainDeadline, and disconnects. - Callers that send no version reach old and new instances during the change. Callers that send
X-Api-Version(or theAcceptform) reach only instances of that version, with that version’s timeout and claims. A caller that sends no version can reach either version, so until the old version is gone it needs the claims that both versions require.
Pitfalls
- Two instances with the same
InstanceId. When a second process connects with the same service name, version, region, transport andInstanceId, the gateway replaces the first one’s registration and routes only to the second. Give each process its own identifier. - A remote region receives traffic only when the local one is empty or unhealthy. That is failover, not load sharing.
- A heartbeat interval above 30 seconds makes instances flap with the gateway’s default thresholds (Troubleshooting).
- A pinned version no instance serves gets
503. Keep at least one instance of every version callers still pin until they move.
