Rate Limiter Lab

Replicas, shared counters and failure modes, live.

Guided lab

Each scenario sets up replicas, Redis and policy, resets counters, then sends traffic.

1

One limiter, one truth

A single replica with in-memory counters.

Exactly 10 allowed, 20 denied.

2

The race: two private counters

Add a second replica, each still counting in its own memory.

About 20 allowed. Each replica happily allows its own 10.

3

The fix: one shared counter

Both replicas count atomically in Redis.

Back to exactly 10 allowed across both replicas.

4

Redis dies: fail open

Stop Redis while the limiters depend on it.

All 30 allowed, uncounted. The API stays up, unprotected.

5

Redis dies: fail closed

Same outage, but the limiter refuses when it can't count.

All 30 rejected with 503. Protected, but fully down.

6

Token bucket: burst, then drip

10 tokens, refilling at 1/s. Requests arrive every 250 ms for 10 s.

A burst of 10, then roughly every 4th request passes (~20 total).

Limiter policy

Applied live to every running replica

Algorithm

One INCR per request. Allows up to 2× the limit across a boundary.

Limit

10 requests per 60 s, per client.

Counter store

Every replica increments one atomic counter in Redis, so the limit holds globally.

If Redis is unreachable

Let requests through uncounted. The API stays up, unprotected.

Replicas

    Infrastructure

    Real containers, started and stopped through Docker

    Second limiter replicalimiter2, behind the same nginx load balancer
    Redis serverTurn off to simulate a counter-store outage

    This switches the Redis container. Whether the limiters count in it is the Counter store setting.

    Traffic generator

    Requests travel nginx → limiter → api, exactly like real clients

    30 requests one at a time, as a single client.

    No traffic yet. Run a scenario above, or set a pattern and send traffic.

    Container logs

    Every ALLOW / DENY decision, straight from the containers

    No output yet.