Rate limiting

This page shows how to limit requests at the gateway and how an endpoint limits its own work. Limits apply in two places:

WhereWhat it limitsConfigured in
The gatewayRequests per target service, in each gateway process and per environment across gateways, with overrides per service and per routeGateway configuration, rate_limiting
An endpointIts own work per key you choose (a report, a customer, an item)C# in the service, with StellaOps.Messaging.RateLimiting

Use the gateway’s limits to protect the system as a whole, and an endpoint’s limits for rules that depend on the request’s content.

Gateway limits

The gateway reads the root section rate_limiting (snake_case keys). It counts requests per target service: every request to the billing service counts against the same window, whatever its path. When the caller’s token carries a stellaops:tenant claim, the gateway counts per tenant and service instead, so one tenant’s burst does not use up another tenant’s budget. A denied request gets 429:

Per gateway process

An in-memory sliding window per target service (and tenant), counted by each gateway process on its own:

{
  "rate_limiting": {
    "process_back_pressure_when_more_than_per_5min": 5000,
    "for_instance": {
      "rules": [ { "per_seconds": 60, "max_requests": 600 } ]
    }
  }
}

Per environment, across gateways

A fixed window per target service (and tenant) counted in Valkey, shared by every gateway of the environment. The .NET tool registers the Valkey store; a custom host adds StellaOps.Router.Gateway.RateLimiting.Valkey and calls AddRouterValkeyRateLimitStore(). A host that configures for_environment without a store refuses to start.

{
  "rate_limiting": {
    "process_back_pressure_when_more_than_per_5min": 0,
    "for_environment": {
      "valkey_connection": "valkey.contoso.example:6379",
      "valkey_bucket": "router-rate-limit",
      "circuit_breaker": { "failure_threshold": 5, "timeout_seconds": 30, "half_open_timeout": 10 },
      "rules": [
        { "per_seconds": 1, "max_requests": 10 },
        { "per_seconds": 3600, "max_requests": 3000 }
      ]
    }
  }
}

Per service and per route

Overrides replace, not merge: route rules replace the service’s rules, and service rules replace the environment rules. A level without rules uses the next wider level.

{
  "rate_limiting": {
    "for_environment": {
      "valkey_connection": "valkey.contoso.example:6379",
      "rules": [ { "per_seconds": 60, "max_requests": 600 } ],
      "microservices": {
        "billing": {
          "rules": [ { "per_seconds": 60, "max_requests": 300 } ],
          "routes": {
            "invoice_create": {
              "pattern": "/invoices",
              "match_type": "exact",
              "rules": [ { "per_seconds": 10, "max_requests": 20 } ]
            },
            "invoice_reads": {
              "pattern": "/invoices/*",
              "match_type": "prefix",
              "rules": [ { "per_seconds": 1, "max_requests": 50 } ]
            }
          }
        }
      }
    }
  }
}
match_typeMatches
exactThe path, in any letter case, ignoring a trailing /
prefixPaths that start with the pattern; patterns usually end with *
regexA regular expression, compiled at start-up; an invalid one stops the gateway

When several routes match, exact wins over prefix (the longest prefix first), and prefix over regex (the longest pattern first). Route overrides apply to the environment limits. Write service names exactly as the services announce them.

Overrides are chosen by service and route, for callers with or without a stellaops:tenant claim. With the claim, the counts of an override are kept per tenant as well.

As environment variables, the same keys use __: rate_limiting__for_instance__rules__0__per_seconds=60.

Endpoint limits

StellaOps.Messaging.RateLimiting gives an endpoint a limiter with a fixed window per key and policy. The limiter packages are under the Router licence, also when only a microservice uses them; the microservice SDK itself is Apache-2.0 (Router packages).

Register one backend in the service:

BackendPackageRegistrationCounts
In-memoryStellaOps.Messaging.RateLimiting.InMemoryAddInMemoryRateLimiter()Per service process; for one instance, development and tests
PostgreSQLStellaOps.Messaging.RateLimiting.Postgres with StellaOps.Messaging.Transport.PostgresThe PostgreSQL Messaging transport, then AddPostgresRateLimiter()Shared by every instance that uses the same database
ValkeyStellaOps.Messaging.RateLimiting.Valkey with StellaOps.Messaging.Transport.ValkeyThe Valkey Messaging transport, then AddValkeyRateLimiter()Shared by every instance that uses the same Valkey server

The in-memory backend needs nothing else:

// The C# rate limiter (in-memory backend), for the per-report limits of the attachment upload.
builder.Services.AddInMemoryRateLimiter();

The PostgreSQL and Valkey backends take their connection from the matching Messaging transport, so the service registers the transport as well. Without it, resolving the limiter fails with a dependency injection error that names the missing connection factory.

// The PostgreSQL Messaging transport provides the connection: it reads
// RateLimiting:postgres:ConnectionString and RateLimiting:postgres:Schema (default "messaging").
// The limiter creates one table per limiter name in that schema when it first runs.
services.AddMessagingTransport<PostgresTransportPlugin>(configuration, "RateLimiting");
services.AddPostgresRateLimiter();
// The Valkey Messaging transport provides the connection: it reads
// RateLimiting:valkey:ConnectionString (default "localhost:6379") and RateLimiting:valkey:Database.
// A service that already registers the Valkey Messaging transport (for example for the Messaging
// router transport) calls only AddValkeyRateLimiter: the limiter uses that connection.
services.AddMessagingTransport<ValkeyTransportPlugin>(configuration, "RateLimiting");
services.AddValkeyRateLimiter();

With the section name RateLimiting used above, the settings are, in appsettings.json form:

{
  "RateLimiting": {
    "postgres": { "ConnectionString": "Host=db.contoso.example;Database=reports;Username=reports", "Schema": "messaging" },
    "valkey": { "ConnectionString": "valkey.contoso.example:6379" }
  }
}

Pass the PostgreSQL connection string with its password from your secret store, for example as the environment variable RateLimiting__postgres__ConnectionString.

In the endpoint, create a named limiter from IRateLimiterFactory and acquire a permit per key. The reports sample allows 10 attachment uploads per report per minute and 100 per day:

foreach (var policy in Policies)
{
    var permit = await _limiter.TryAcquireAsync(reportId, policy, cancellationToken);
    if (!permit.IsAllowed)
    {
        var deniedHeaders = new HeaderCollection();
        deniedHeaders.Set("Content-Type", "text/plain; charset=utf-8");
        deniedHeaders.Set(
            "Retry-After",
            Math.Ceiling(permit.RetryAfter?.TotalSeconds ?? 1).ToString(CultureInfo.InvariantCulture));
        return new RawResponse
        {
            StatusCode = 429,
            Headers = deniedHeaders,
            Body = new MemoryStream(Encoding.UTF8.GetBytes("Too many attachments for this report."))
        };
    }
}

Choosing

GoalUse
Protect a gateway process from bursts to one servicefor_instance
One budget for a whole environment, or for a service or routefor_environment with overrides
A limit per customer, report or other business keyAn endpoint limiter
Several service instances share an endpoint limitThe PostgreSQL or Valkey backend

Pitfalls