A rate limiter is the bouncer at the door of your API. Get it wrong and you either let a stampede crush your backend or wrongly turn away paying customers at twice your real limit.
Why you need one at all
Rate limiting is not only about stopping abuse, though it does that too. It protects a shared resource from any client — malicious, buggy, or just popular — consuming more than its fair share. A single customer with a runaway retry loop can saturate a database connection pool and take down the service for everyone. A limiter turns "one bad client breaks the system" into "one bad client gets throttled." It is a stability control first and a security control second.
That framing matters because it tells you where to enforce. You want the limiter as close to the edge as possible — an API gateway or reverse proxy — so rejected traffic never reaches your expensive backend at all. Rejecting a request after it has already opened a database transaction defeats half the purpose.
Pick your algorithm
The naive choice is a fixed window counter, and it has a nasty flaw: the boundary-burst problem. A client can spend its full quota in the last second of one window and again in the first second of the next, sneaking through 2x the limit in a blink. If your limit is 100 requests per minute, a client can land 200 requests in a two-second span straddling the boundary.
Here are the four algorithms worth knowing, and when each earns its place:
- Fixed window. One counter per time bucket. Trivial to build, cheap to store, but suffers the boundary burst. Fine for coarse limits where 2x overshoot is survivable.
- Sliding window log. Store a timestamp for every request and count how many fall in the trailing window. Perfectly accurate — and memory-hungry, since a client doing 10k requests stores 10k timestamps. Rarely worth it.
- Sliding window counter. A clever approximation: weight the previous window's count by how far you are into the current one. Smooths the boundary burst with one number, not a log. This is the sweet spot for most APIs.
- Token bucket. Tokens refill at a steady rate, requests spend them, and the bucket caps the burst.
Token bucket is my default. It is simple, allows controlled bursts, and maps cleanly onto one counter plus a timestamp. The bucket size sets how big a burst you tolerate; the refill rate sets the sustained throughput. Those two knobs express almost any real-world policy — "1000 per hour but allow short spikes" falls out naturally.
Make it distributed
Once you run more than one app server, in-memory counters lie. Client X hits server A five times and server B five times; each sees five and neither trips a limit of eight, so the client sails past at 10. The counter has to live somewhere all servers can see it.
The fix is a shared store, and Redis is the obvious pick: fast, single-threaded (so operations are naturally serialized), atomic, and built for exactly this. An in-memory counter with sub-millisecond latency means the limiter adds negligible overhead to each request.
The trap everyone falls into is doing INCR and EXPIRE as two separate calls. If a process dies or a request races between them, you get a counter that never expires and a permanently throttled client. Two round trips, one race condition — and it fails in the worst possible way, locking out a legitimate user until someone notices and deletes the key by hand.
Kill the race with Lua
Redis runs Lua scripts atomically, start to finish, with no interleaving. Collapse the read, increment, and expiry into one script and the race simply cannot happen.
local n = redis.call("INCR", KEYS[1])
if n == 1 then
redis.call("EXPIRE", KEYS[1], ARGV[1])
end
if n > tonumber(ARGV[2]) then return 0 end
return 1One network round trip, one indivisible operation. The if n == 1 guard sets the TTL only on the first request of a window, so the expiry rides with the key from birth. Whether you implement fixed window, sliding window, or token bucket, the pattern is the same: all the read-modify-write logic executes server-side in Redis, atomically, and your app just reads the yes/no answer.
Choosing the key
The algorithm is only half the design; the other half is what you count. The key decides who shares a budget.
- By API key or user ID for authenticated traffic — the fairest unit, since it maps to an actual account.
- By IP for anonymous traffic — but beware that corporate NATs and mobile carriers put thousands of users behind one address, so an IP limit can throttle a whole office.
- By endpoint tier — a
POST /logindeserves a far tighter limit than aGET /health, because it is the one attackers brute-force.
Real systems layer these: a generous global per-user limit, plus a strict per-endpoint limit on the sensitive routes. Compose them and each catches a different abuse pattern.
Reject with manners
When a client is over the limit, return HTTP 429 Too Many Requests — not a 500, not a silent drop. Add a Retry-After header so well-behaved clients know exactly when to come back instead of hammering you in a retry loop. It is also good manners to expose the budget on every response so clients can self-pace:
HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1735689630A good limiter does not just say no; it says not yet, try again in 30 seconds — and tells honest clients how much room they have left before they hit the wall.
The trade-offs nobody skips
Two decisions will come up in any serious design review:
- Fail open or fail closed? If Redis is down, do you let every request through (fail open, favoring availability) or reject everything (fail closed, favoring protection)? Most APIs fail open — a rate limiter outage should not become a full outage — but a limiter guarding a payment or auth endpoint may choose to fail closed. Decide deliberately; do not let a stack trace decide for you.
- Accuracy versus cost. Perfect enforcement means synchronous, strongly-consistent counting on every request. At extreme scale, some systems accept slight overcounting by batching or using local counters that sync periodically. You trade a little precision for a lot of throughput — a fine bargain when the limit is 10,000/sec and being off by a few dozen does not matter.
The takeaway
- Token bucket by default — it handles bursts gracefully and dodges the fixed-window boundary-burst trap.
- Enforce at the edge — reject before the request touches your expensive backend.
- One Lua script, zero races — never split
INCRandEXPIRE; atomicity is the whole game. - The key defines fairness — limit by user where you can, by IP where you must, and tier the sensitive endpoints.
- 429 plus
Retry-After— reject clearly and tell clients when to return, or they will retry you to death.