Rate limiting
This page shows how to limit requests at the gateway and how an endpoint limits its own work. Limits apply in two places:
| Where | What it limits | Configured in |
|---|---|---|
| The gateway | Requests per target service, in each gateway process and per environment across gateways, with overrides per service and per route | Gateway configuration, rate_limiting |
| An endpoint | Its own work per key you choose (a report, a customer, an item) | C# in the service, with StellaOps.Messaging.RateLimiting |
Use the gateway’s limits to protect the system as a whole, and an endpoint’s limits for rules that depend on the request’s content.
Gateway limits
The gateway reads the root section rate_limiting (snake_case keys). It counts requests per target service: every request to the billing service counts against the same window, whatever its path. When the caller’s token carries a stellaops:tenant claim, the gateway counts per tenant and service instead, so one tenant’s burst does not use up another tenant’s budget. A denied request gets 429:
Retry-After: <seconds>;X-RateLimit-Limit,X-RateLimit-RemainingandX-RateLimit-Reset(Unix seconds);- a JSON body with
error(rate_limit_exceeded),message,retryAfter,limit,current,windowandscope.
Per gateway process
An in-memory sliding window per target service (and tenant), counted by each gateway process on its own:
{
"rate_limiting": {
"process_back_pressure_when_more_than_per_5min": 5000,
"for_instance": {
"rules": [ { "per_seconds": 60, "max_requests": 600 } ]
}
}
}
Per environment, across gateways
A fixed window per target service (and tenant) counted in Valkey, shared by every gateway of the environment. The .NET tool registers the Valkey store; a custom host adds StellaOps.Router.Gateway.RateLimiting.Valkey and calls AddRouterValkeyRateLimitStore(). A host that configures for_environment without a store refuses to start.
{
"rate_limiting": {
"process_back_pressure_when_more_than_per_5min": 0,
"for_environment": {
"valkey_connection": "valkey.contoso.example:6379",
"valkey_bucket": "router-rate-limit",
"circuit_breaker": { "failure_threshold": 5, "timeout_seconds": 30, "half_open_timeout": 10 },
"rules": [
{ "per_seconds": 1, "max_requests": 10 },
{ "per_seconds": 3600, "max_requests": 3000 }
]
}
}
}
- Activation.
process_back_pressure_when_more_than_per_5minchecks the environment limits only above that many requests per 5 minutes in this process;0checks always. - Several rules apply together: a request is denied when any rule is exceeded, and
Retry-Afteris the longest of the exceeded rules. - Valkey unavailable. Environment limiting lets requests through; the per-process limits still apply.
Per service and per route
Overrides replace, not merge: route rules replace the service’s rules, and service rules replace the environment rules. A level without rules uses the next wider level.
{
"rate_limiting": {
"for_environment": {
"valkey_connection": "valkey.contoso.example:6379",
"rules": [ { "per_seconds": 60, "max_requests": 600 } ],
"microservices": {
"billing": {
"rules": [ { "per_seconds": 60, "max_requests": 300 } ],
"routes": {
"invoice_create": {
"pattern": "/invoices",
"match_type": "exact",
"rules": [ { "per_seconds": 10, "max_requests": 20 } ]
},
"invoice_reads": {
"pattern": "/invoices/*",
"match_type": "prefix",
"rules": [ { "per_seconds": 1, "max_requests": 50 } ]
}
}
}
}
}
}
}
match_type | Matches |
|---|---|
exact | The path, in any letter case, ignoring a trailing / |
prefix | Paths that start with the pattern; patterns usually end with * |
regex | A regular expression, compiled at start-up; an invalid one stops the gateway |
When several routes match, exact wins over prefix (the longest prefix first), and prefix over regex (the longest pattern first). Route overrides apply to the environment limits. Write service names exactly as the services announce them.
Overrides are chosen by service and route, for callers with or without a stellaops:tenant claim. With the claim, the counts of an override are kept per tenant as well.
As environment variables, the same keys use __: rate_limiting__for_instance__rules__0__per_seconds=60.
Endpoint limits
StellaOps.Messaging.RateLimiting gives an endpoint a limiter with a fixed window per key and policy. The limiter packages are under the Router licence, also when only a microservice uses them; the microservice SDK itself is Apache-2.0 (Router packages).
Register one backend in the service:
| Backend | Package | Registration | Counts |
|---|---|---|---|
| In-memory | StellaOps.Messaging.RateLimiting.InMemory | AddInMemoryRateLimiter() | Per service process; for one instance, development and tests |
| PostgreSQL | StellaOps.Messaging.RateLimiting.Postgres with StellaOps.Messaging.Transport.Postgres | The PostgreSQL Messaging transport, then AddPostgresRateLimiter() | Shared by every instance that uses the same database |
| Valkey | StellaOps.Messaging.RateLimiting.Valkey with StellaOps.Messaging.Transport.Valkey | The Valkey Messaging transport, then AddValkeyRateLimiter() | Shared by every instance that uses the same Valkey server |
The in-memory backend needs nothing else:
// The C# rate limiter (in-memory backend), for the per-report limits of the attachment upload.
builder.Services.AddInMemoryRateLimiter();
The PostgreSQL and Valkey backends take their connection from the matching Messaging transport, so the service registers the transport as well. Without it, resolving the limiter fails with a dependency injection error that names the missing connection factory.
// The PostgreSQL Messaging transport provides the connection: it reads
// RateLimiting:postgres:ConnectionString and RateLimiting:postgres:Schema (default "messaging").
// The limiter creates one table per limiter name in that schema when it first runs.
services.AddMessagingTransport<PostgresTransportPlugin>(configuration, "RateLimiting");
services.AddPostgresRateLimiter();
// The Valkey Messaging transport provides the connection: it reads
// RateLimiting:valkey:ConnectionString (default "localhost:6379") and RateLimiting:valkey:Database.
// A service that already registers the Valkey Messaging transport (for example for the Messaging
// router transport) calls only AddValkeyRateLimiter: the limiter uses that connection.
services.AddMessagingTransport<ValkeyTransportPlugin>(configuration, "RateLimiting");
services.AddValkeyRateLimiter();
With the section name RateLimiting used above, the settings are, in appsettings.json form:
{
"RateLimiting": {
"postgres": { "ConnectionString": "Host=db.contoso.example;Database=reports;Username=reports", "Schema": "messaging" },
"valkey": { "ConnectionString": "valkey.contoso.example:6379" }
}
}
Pass the PostgreSQL connection string with its password from your secret store, for example as the environment variable RateLimiting__postgres__ConnectionString.
In the endpoint, create a named limiter from IRateLimiterFactory and acquire a permit per key. The reports sample allows 10 attachment uploads per report per minute and 100 per day:
foreach (var policy in Policies)
{
var permit = await _limiter.TryAcquireAsync(reportId, policy, cancellationToken);
if (!permit.IsAllowed)
{
var deniedHeaders = new HeaderCollection();
deniedHeaders.Set("Content-Type", "text/plain; charset=utf-8");
deniedHeaders.Set(
"Retry-After",
Math.Ceiling(permit.RetryAfter?.TotalSeconds ?? 1).ToString(CultureInfo.InvariantCulture));
return new RawResponse
{
StatusCode = 429,
Headers = deniedHeaders,
Body = new MemoryStream(Encoding.UTF8.GetBytes("Too many attachments for this report."))
};
}
}
RateLimitPolicy.PerSecond(n),PerMinute(n)andPerHour(n)build policies;new RateLimitPolicy(n, window)takes any window from 1 second to 366 days, in whole milliseconds, with at least one permit.- Windows are fixed and aligned to the Unix epoch: a one-minute window starts at each full minute. Up to twice the limit can therefore pass within one window length that spans a window boundary.
- A denied attempt takes no permit. A key’s window never moves backwards, so a host whose clock lags cannot reopen permits.
- A limiter name has 1 to 40 characters, starts with a lower-case letter and holds only
a-z,0-9and_; a key has 1 to 512 characters and is case-sensitive. Other values throw anArgumentException. GetStatusAsync(key, policy)reads the count without taking a permit;ResetAsync(key)clears the key under every policy and returnstruewhen it had permits counted in an open window.- A limiter keeps one window per key and policy, so the two policies above count independently.
TryAcquireAsyncreturns a permit withIsAllowedandRetryAfter. Return429withRetry-After, as the sample does.- Before each release, an automated run sends 11 uploads to one report within a minute and receives ten
200and one429.
Choosing
| Goal | Use |
|---|---|
| Protect a gateway process from bursts to one service | for_instance |
| One budget for a whole environment, or for a service or route | for_environment with overrides |
| A limit per customer, report or other business key | An endpoint limiter |
| Several service instances share an endpoint limit | The PostgreSQL or Valkey backend |
Pitfalls
- The in-memory endpoint limiter counts per instance. With three instances, a limit of 10 lets up to 30 through. Use a shared backend.
for_environmentwithout a Valkey store stops a custom gateway host at start-up.- Overrides replace. A service override with one short rule drops the environment’s long rule for that service; repeat the rules you want to keep.
- The gateway’s limits are per service, not per caller. One caller can use up a service’s whole window for every other caller without a tenant claim. Limit per caller in the endpoint, or in a proxy in front of the gateway.
