Rate Limiting Strategies: Token Bucket, Sliding Window, and When to Use Each
For an internal services project, we started with a fixed window counter because it's the simplest thing to implement — increment a Redis key, expire it every 60 seconds. It fell apart at window boundaries: a client could send 100 requests in the last second of one window and 100 more in the first second of the next, doubling the intended rate for a brief burst. That was fine for protecting a database from accidental hammering, but not fine for a customer-facing quota.
We moved customer-facing limits to a sliding window log implemented with Redis sorted sets, storing a timestamp per request and trimming anything older than the window on each check. It's accurate but costs more memory — for a client with tens of thousands of active API keys, that meant sizing Redis for the log volume, not just key count. For most of them, request rates were low enough that this was a non-issue.
Token bucket is what we reach for when we want to allow bursts without allowing sustained abuse — think a mobile client that syncs in batches after being offline. We size the bucket capacity to cover a reasonable burst (say, 50 requests) and the refill rate to match the sustained rate we actually want to enforce (say, 5 per second), which lets legitimate bursty clients through without raising the sustained ceiling.
Sliding window counters — an approximation that blends the current and previous fixed windows weighted by elapsed time — became our default for anything high-volume, since they're nearly as accurate as the sorted-set approach at a fraction of the memory and CPU cost. We only reach for the exact sliding log when a client's contract requires hard, auditable limits, like a billing-tied API quota.
The mistake we made early on was applying one global rate limiter in front of everything. Different endpoints have wildly different costs — a search endpoint hitting Elasticsearch needs a tighter limit than a cached lookup — so we now scope limits per route group, not just per API key, and return Retry-After headers so well-behaved clients can back off correctly instead of hammering us in a retry loop.
