Transports
A transport carries requests and responses between the gateway and the microservices. Each transport has a server half in the gateway and a client half in each service, in separate packages. The gateway can offer several transports at once; each service uses one.
| Transport | Server package (gateway) | Client package (service) | Encrypted | Insecure switch needed |
|---|---|---|---|---|
| TCP | StellaOps.Router.Transport.Tcp.Server | StellaOps.Router.Transport.Tcp | No | Yes |
| TLS | StellaOps.Router.Transport.Tls.Server | StellaOps.Router.Transport.Tls | Yes, TLS 1.2 or 1.3 | No |
| UDP | StellaOps.Router.Transport.Udp.Server | StellaOps.Router.Transport.Udp | No | Yes |
| RabbitMQ | StellaOps.Router.Transport.RabbitMq.Server | StellaOps.Router.Transport.RabbitMq | When the broker uses TLS | No |
| Messaging (Valkey) | StellaOps.Router.Transport.Messaging.Server | StellaOps.Router.Transport.Messaging with StellaOps.Messaging.Transport.Valkey | When Valkey uses TLS | No |
| InMemory | StellaOps.Router.Transport.InMemory (both halves) | the same | Not applicable | No |
The server packages are under the Router licence; the client packages are Apache-2.0.
On every transport the client half runs each request it receives as soon as it arrives, so one connection serves several requests at the same time; only Messaging limits how many (MaxConcurrentRequests). When a caller disconnects or an endpoint timeout expires, the gateway sends a cancel frame with the request’s correlation id and the endpoint’s CancellationToken is cancelled; Messaging delivers it only with frame protection.
Choosing a transport
| Situation | Transport |
|---|---|
| Development on one machine | TCP, or InMemory in tests |
| Production across a network | TLS, with client certificates where services and gateway belong to different teams |
| Services and gateway already share a RabbitMQ broker | RabbitMQ |
| Services and gateway already share a Valkey server, or must not open connections to the gateway host | Messaging over Valkey |
| Small, frequent requests on a trusted network, loss acceptable | UDP |
The running samples use TCP and InMemory. The other registrations on this page are compiled before each release (Examples.Transports) but not run, because they need certificates or servers.
Each request and each response travels in one frame, and the gateway and the service hold it in memory. The largest body each transport carries:
| Transport | Largest body | Setting |
|---|---|---|
| TCP, TLS | About 12 MiB (a 16 MiB frame; the body travels Base64-encoded) | MaxFrameSize on both sides |
| UDP | One datagram, 8192 bytes for the whole frame | MaxDatagramSize on both sides |
| RabbitMQ | About 12 MiB (a 16 MiB message by default; the body travels Base64-encoded) | MaxMessageSize on both sides, at most the broker’s max_message_size |
| Messaging | No limit of its own |
A larger request is answered 413, a larger response 502 Response too large (File uploads and downloads).
The insecure-transport switch
TCP and UDP send frames unencrypted. Both halves refuse to register unless the process environment variable STELLAOPS_ROUTER_ALLOW_INSECURE_TRANSPORTS is true, yes or 1. It is an environment variable, not a configuration key, and it affects only TCP and UDP. Set it for development and for networks you trust completely; use TLS everywhere else.
Configuring the gateway side
The gateway host reads each transport from Gateway:Transports:<Name> and starts it only when Enabled is true. Pass the server assemblies to AddRouterGatewayHost (a custom host), or use the .NET tool, which has them all (Router gateway host).
Without Port, a listener uses its transport’s default: TCP 9100, TLS 9443, UDP 5102. A transport client without Port connects to the same default.
Time spans in the transport sections use .NET syntax: 00:00:10 is ten seconds. A bare number is a number of days.
TCP
{
"Gateway": {
"Transports": {
"Tcp": { "Enabled": true, "BindAddress": "0.0.0.0", "Port": 9100 }
}
}
}
The service side is the getting-started registration: AddTcpTransportClient with Host and Port.
TLS, mTLS and certificate pinning
Gateway side:
{
"Gateway": {
"Transports": {
"Tls": {
"Enabled": true,
"Port": 9443,
"ServerCertificatePath": "/etc/router/tls/gateway.crt",
"ServerCertificateKeyPath": "/etc/router/tls/gateway.key",
"RequireClientCertificate": true,
"EnabledProtocols": "Tls13"
}
}
}
}
ServerCertificatePathtakes a.pfxor.p12file (withServerCertificatePassword), or a.pem,.crtor.cerfile. For a PEM certificate, setServerCertificateKeyPathto the private key file; the transport does not read a key from the certificate file.RequireClientCertificateturns on mTLS: a service without a valid client certificate cannot connect.EnableCertificateHotReloadreloads the certificate files when they change.CheckCertificateRevocationchecks revocation online.
Service side, with a client certificate and the gateway’s certificate pinned:
// TLS with a client certificate (mTLS) and the gateway certificate pinned by SHA-256 thumbprint.
services.AddTlsTransportClient(options =>
{
options.Host = "gateway.contoso.example";
options.Port = 9443;
options.ExpectedServerHostname = "gateway.contoso.example";
options.ClientCertificatePath = "/etc/contoso/billing/client.pfx";
options.ClientCertificatePassword = Environment.GetEnvironmentVariable("BILLING_CLIENT_CERT_PASSWORD");
options.EnabledProtocols = SslProtocols.Tls13;
options.CertificatePinning = new CertificatePinningOptions
{
ThumbprintsSha256 = ["9F2D6C4E7A1B3D5F8E0A2C4B6D8F1A3C5E7B9D0F2A4C6E8B1D3F5A7C9E0B2D4F"]
};
});
Certificate validation, on both sides:
- A certificate is required when the other side requires it.
- The host name must match.
ExpectedServerHostnameis the name the service expects (and sends as SNI); without it,Hostis used. - A chain error rejects the certificate, unless
AllowSelfSignedistrue, the only error is an untrusted root or a partial chain, and the certificate matches a pin. - Pins:
CertificatePinning.ThumbprintsSha256(hex SHA-256 of the certificate),AllowedSubjects(exact subject) andAllowedSans(exact DNS names).AllowSelfSignedrequires at least one pin.
Rejections are counted in the instrument router_tls_validate_rejected_total of the .NET meter StellaOps.Router.Transport.Tls; collect it with an OpenTelemetry exporter, because the gateway’s /metrics does not include it.
UDP
{
"Gateway": {
"Transports": {
"Udp": { "Enabled": true, "Port": 5102, "MaxDatagramSize": 8192 }
}
}
}
// UDP: unencrypted, one datagram per frame; needs STELLAOPS_ROUTER_ALLOW_INSECURE_TRANSPORTS=true.
services.AddUdpTransportClient(options =>
{
options.Host = "127.0.0.1";
options.Port = 5102;
options.MaxDatagramSize = 8192;
});
A frame must fit in one datagram, so UDP suits small requests and responses only.
RabbitMQ
{
"Gateway": {
"Transports": {
"RabbitMq": {
"Enabled": true,
"HostName": "rabbitmq.contoso.example",
"Port": 5671,
"UseSsl": true,
"VirtualHost": "/router",
"NodeId": "gw-eu1"
}
}
}
}
// RabbitMQ: NodeId must equal the gateway's Gateway:Transports:RabbitMq:NodeId.
services.AddRabbitMqTransportClient(options =>
{
options.HostName = "rabbitmq.contoso.example";
options.Port = 5671;
options.UseSsl = true;
options.VirtualHost = "/router";
options.UserName = "billing";
options.Password = Environment.GetEnvironmentVariable("BILLING_RABBITMQ_PASSWORD") ?? string.Empty;
options.NodeId = "gw-eu1";
options.DurableQueues = true;
});
NodeIdis required on the gateway, and the services must set the sameNodeId: a service without one usesdefaultand never reaches the gateway.- The default credentials are
guest/guest. Set a user for each side. - The gateway reads its queue
<QueuePrefix>.gw.<NodeId>.in; each service instance has<QueuePrefix>.svc.<InstanceId>.ExchangePrefixdefaults tostella.router,QueuePrefixtostella.
Messaging over Valkey
{
"Gateway": {
"Transports": {
"Messaging": {
"Enabled": true,
"valkey": { "ConnectionString": "valkey.contoso.example:6379,ssl=true" }
}
}
}
}
Service side; the Valkey backend reads Router:Transport:Messaging:valkey:ConnectionString from the service’s configuration:
// Messaging over Valkey: the Valkey backend reads Router:Transport:Messaging:valkey:ConnectionString,
// and the queue names must equal the gateway's (the defaults below are the gateway's defaults).
services.AddMessagingTransport<ValkeyTransportPlugin>(configuration, "Router:Transport:Messaging");
services.AddMessagingTransportClient(options =>
{
options.RequestQueueTemplate = "router:requests:{service}";
options.ResponseQueueName = "router:responses";
});
- Queue names must be the same on both sides:
RequestQueueTemplate(router:requests:{service}),CancelControlQueueTemplate(router:cancels:{service}:{connection}) andResponseQueueName(router:responses). - The .NET tool registers the Valkey backend only;
Gateway:Transports:Messaging:transportmay be left out, and when set must bevalkey. - Durations in this section accept
30s,5m,1hor1d, or .NET syntax. An invalid value falls back to the default without an error.
InMemory
Gateway and service in one process share an InMemoryConnectionRegistry. Use it in tests (Testing with the in-memory harness).
Connection behaviour by transport
TCP and TLS clients
| Option | Default | Meaning |
|---|---|---|
ConnectTimeout | 10 seconds | How long one connection attempt may take. |
MaxReconnectAttempts | 10 | Reconnection attempts the client makes on its own after a connection drops. |
MaxReconnectBackoff | 1 minute | The longest wait between two of those attempts. The wait doubles from 200 milliseconds. |
KeepAliveInterval | 30 seconds | TCP keep-alive on the connection, on the service and on the gateway: after this long without traffic the system probes the other side, and closes the connection after three unanswered probes, so a host that vanished is noticed. Zero turns it off. |
ClientCertificateKeyPath (TLS) | none | The private key file of a PEM client certificate. |
ClientCertificate, ServerCertificate (TLS, in code) | none | Certificate objects instead of file paths. |
After a connection drops, the client reconnects on its own and sends its HELLO again. When all MaxReconnectAttempts fail (about 2.7 minutes with the defaults), it logs Max reconnection attempts (N) reached; reporting the transport as dead so the service connects again, and the service connects again with the SDK’s ReconnectBackoffInitial and ReconnectBackoffMax until the gateway answers. A gateway outage of any length therefore ends with the service registered again, without a restart. A TLS authentication error on an open connection is handled the same way, with a new handshake.
TLS, both sides:
- With
Gateway:Transports:Tls:Enabledand noServerCertificatePath, the gateway stops at start-up withTLS transport requires Gateway:Transports:Tls:ServerCertificatePath when enabled. - A certificate matches the pins when any one of them matches: a thumbprint, a subject or a DNS name. Thumbprints may be written in either letter case, with
:,-or spaces between the bytes. AllowSelfSigned=truewithout a pin stops start-up on that side withTlsTransportOptions: AllowSelfSigned=true requires at least one of CertificatePinning.ThumbprintsSha256, ....
UDP limits
- The whole HELLO (every endpoint with its schemas and OpenAPI text) must fit one datagram. A larger one fails the connection with a payload-too-large error, and the service never connects. Keep UDP for services with few, small endpoints, or raise
MaxDatagramSizeon both sides. AllowBroadcast(false),ReceiveBufferSizeandSendBufferSize(65536 bytes) are set on both sides.
RabbitMQ delivery
| Option | Default | Meaning |
|---|---|---|
SslCertPath | none | A client certificate for the broker, with UseSsl. |
InstanceId | the SDK’s instance id | The instance part of the service’s queue name. |
AutomaticRecoveryEnabled | true | The client library reconnects to the broker after a network failure. |
NetworkRecoveryInterval | 5 seconds | The wait between two recovery attempts. |
DefaultTimeout | 30 seconds | How long a request or response message may wait in its queue. |
Messages are not persistent and are delivered at most once: a message that waits in its queue longer than DefaultTimeout, or is lost with a broker restart, is not delivered, and the caller gets the endpoint’s 504 when its timeout expires.
Messaging delivery
- A request runs once. The service runs a request until the request’s own timeout, the one at which the gateway answers
504, keeps its message leased while it runs, and acknowledges the message however the run ends: a request that fails or times out is not delivered again.RequestTimeout(55 seconds by default) applies to a request only when the gateway sent no timeout, and to the other messages of the service queues. - A second run after a failure. A request can still run twice: when the service cannot reach Valkey for more than 30 seconds during the run, when its acknowledgement is lost, or when the service stops before the run ends, another instance takes the message over. Make Messaging endpoints that change data idempotent.
- Cancellation keys (both sides, under the transport section):
CancelTombstoneTtl(5 minutes; at leastRequestTimeout),MaxPendingCancellations(10000),CancelControlApproximateMaxLength(1000) andCancelControlBatchSize(128). A value of zero or less, or aCancelTombstoneTtlbelowRequestTimeout, stops start-up and names the key. - Reserved name. The gateway’s control traffic uses the queue of the service name
gateway-control; do not name a servicegateway-control. - Polling.
valkey:QueueWaitTimeoutSeconds(10) is how long a consumer waits for a notification before it polls the queue again;0turns the poll off and is not recommended.
Frame protection for queue transports
RabbitMQ and Messaging carry frames through a broker that other systems may read. Frames are signed and verified when the root configuration section Router:Frame:Envelope exists:
| Key | Meaning |
|---|---|
Router:Frame:Envelope:ActiveKeyId | The key that signs new frames (default primary). |
Router:Frame:Envelope:Keys:<id> | A secret per key id; at least one. |
Router:Frame:Envelope:ReplayWindow | How long a frame is accepted (default 5 minutes). |
A section that is present but incomplete stops start-up with Router frame-envelope protection requires ...: an empty ActiveKeyId, no key, an ActiveKeyId that names no key, an empty key, or a ReplayWindow of zero or less.
The gateway host reads the section itself. In a service, Add*TransportClient does not; register the protection as well, with the same keys:
// RabbitMQ and Messaging: protect frames with the keys under Router:Frame:Envelope, the same keys
// as the gateway's. Without that section the call registers nothing.
services.AddConfiguredFrameEnvelopeProtection(configuration);
Supply the secrets from your secret store, never from a committed file. The samples do not run a queue transport, so this pairing is compiled before each release but not run.
Pitfalls
- The service logs
Host is not configured. SetHoston the transport client. - TCP or UDP refuses to start. Set
STELLAOPS_ROUTER_ALLOW_INSECURE_TRANSPORTSon that process, or move to TLS. - TLS fails with a name mismatch. Set
ExpectedServerHostnameto a name in the gateway certificate. - A PEM gateway certificate fails with a missing private key. Set
ServerCertificateKeyPath. - RabbitMQ services never connect. The services’
NodeIddiffers from the gateway’s. - A Messaging endpoint runs twice. The service lost Valkey for more than 30 seconds during the run, its acknowledgement was lost, or it stopped before the run ended; another instance then took the request over (above).
