Designing API Rate Limits From the Client's Point of View
The first rate limiter we built for a public API client shipped was a simple fixed-window counter: 1,000 requests per hour, reset on the hour. It was easy to implement and immediately unpopular, because a partner integration that front-loaded its sync at the top of the hour would get throttled hard for the remaining fifty-nine minutes, then burst again at the next reset — a thundering herd we'd built ourselves.
We switched to a sliding-window log using Redis sorted sets, which smooths this out — the limit is always 'requests in the trailing 60 minutes' rather than a hard reset boundary. It costs a bit more in Redis memory (each request timestamp stored per client) but eliminated the boundary-burst problem entirely, and for our request volumes the memory cost was negligible.
The bigger change was communication, not algorithm. We started returning X-RateLimit-Remaining and X-RateLimit-Reset headers on every response, not just 429s, so client integrations could self-throttle before hitting the wall instead of discovering the limit via failure. Within a month, 429 responses from our top five API partners dropped by roughly 70%, simply because their code could now see the limit coming.
We also split limits by endpoint cost rather than a single global counter. A cheap GET on a single resource and an expensive search-and-aggregate endpoint were counted identically at first, which meant partners doing legitimate heavy reporting queries were penalized the same as someone hammering a lightweight endpoint. Weighting endpoints by actual server cost (1 unit for simple reads, up to 20 for aggregation queries) let us keep the same server protection with much less friction for normal usage patterns.
For a burst-tolerant use case — a mobile client that needs to sync in short intense windows — we added a token bucket option per API key, configurable by our team for trusted partners, allowing short bursts above the steady-state rate as long as the average holds. Not every client needs this, but having it as a per-key setting meant we didn't have to choose one policy for every use case.
