Sage

Appearance

Rate limiting ​

A rate limiter controls how often something may happen, such as requests per second for an API key, login attempts per account, or a fair-use quota per tenant. Rate limiting is included with every Sage client.

The limiter is distributed. Its state lives on the server, not in process memory, so the limit holds across every process pointed at the same server. Each check decides and consumes atomically on the server, in a single round trip.

rateLimiter binds a policy to the client. Each tryAcquire consumes tokens for a subject and returns a Decision.

scala
val limiter  = client.rateLimiter[String](RateLimit.perSecond(100))
val decision = limiter.tryAcquire("user:42")
val allowed  = decision.isAllowed
scala
val limiter = client.rateLimiter[String](RateLimit.perSecond(100))
for {
  decision <- limiter.tryAcquire("user:42")
} yield decision.isAllowed

The algorithm: token bucket ​

Each subject has a bucket that holds up to capacity tokens and refills continuously over time. A tryAcquire takes cost tokens (one by default) when the bucket holds them, and is denied otherwise. capacity is the burst ceiling: a subject that has been idle can spend up to a full bucket at once, then is paced by the refill rate.

Refill is smooth rather than stepped, so a caller regains its allowance gradually instead of all at once at a window boundary.

Build a policy with the constructors:

scala
RateLimit.perSecond(100)                              // 100 per second, bursting to 100
RateLimit.perMinute(5000)                             // 5000 per minute, bursting to 5000
RateLimit(permits = 100, per = 1.second, burst = 200) // 100/s sustained, bursting to 200

Reading the decision ​

Decision is a return value, not an exception. Both allowed and denied requests are normal results.

  • isAllowed: whether the request was admitted.
  • remainingTokens: tokens left in the bucket. A denial consumes nothing, so it reports the untouched balance. Convenient for an X-RateLimit-Remaining header.
  • Allowed(remaining, resetAfter): resetAfter is the time until the bucket refills to full.
  • Denied(remaining, retryAfter): retryAfter is the time until enough tokens are available. Convenient for a Retry-After header.

tryAcquire returns immediately instead of waiting for capacity. To retry, sleep for retryAfter at the call site and try again.

Peek and reset ​

  • peek(subject) checks the current state without consuming a token. Elapsed time still refills the bucket. It returns Allowed while at least one token is available. Otherwise, it returns Denied with the time until a token becomes available.
  • reset(subject) clears a subject's bucket, so its next request starts from full capacity.

Cost and subjects ​

Pass cost to charge a heavier request more than one token:

scala
limiter.tryAcquire("user:42", cost = 10)

A subject can be any type with a KeyCodec. It is encoded and prefixed with a namespace (ratelimit by default) to form the bucket's key. Give a custom namespace when more than one limiter runs against the same server:

scala
client.rateLimiter[String](RateLimit.perSecond(100), namespace = "login")

Give each policy that is active at the same time its own namespace. Changing the policy on an existing namespace is safe during a rolling deployment: a bucket remembers the policy that created it, and on a change each subject carries over the lesser of its current tokens and the new capacity, so overlapping old and new instances cannot hand out full buckets repeatedly.

Invalid policies ​

A policy needs a positive capacity and refillTokens and a refillPeriod of at least a microsecond, and cost must be between 1 and capacity. Very large values are also rejected, since the server-side arithmetic has to stay exact.

These are programming errors, not runtime outcomes: tryAcquire and peek fail with SageException.InvalidArgument before any server call.

When the store is unreachable ​

Each check contacts the server and fails through the effect F if the server is unreachable. Sage does not choose a fallback behavior. Your application must decide whether to allow or reject requests during an outage.

Capacity planning ​

Each check is one script call and one constant-time atomic operation. It writes even on a denial, so a limiter on every application call adds real write throughput. One key exists per subject whose bucket is not full, expiring once that bucket would be full again, so live keys track the distinct subjects seen within one refill window: long windows over many subjects hold the most state.

If the limiter would take a material share of an existing store, give it its own deployment or more cluster shards, and keep denied callers from retrying in a tight loop.

The composable command ​

tryAcquire runs on the client directly. To pipeline the check or run it yourself instead, command returns the underlying Command to pass to client.run:

scala
client.run(limiter.command("user:42"))

Topology ​

The limiter works with every topology. Each subject's state uses one key and therefore one cluster slot. The same code works with standalone, master-replica, and cluster clients, without requiring hash tags.