What is rate limiting?
Rate limiting is a technique for controlling the number of requests a client can make to a system within a given time window, protecting that system from being overwhelmed. It’s a defensive control that most end users only notice when they hit it.
Rate limits can be applied at many levels: per API key, per user account, per IP address, or globally across an entire service, depending on what the system is trying to protect and from whom.
How does rate limiting work?
A rate limiter tracks requests per client (by API key, IP address, user, or token) against a defined threshold, for example, 100 requests per minute. Once a client exceeds that threshold, subsequent requests are delayed, queued, or rejected, commonly with an HTTP 429 “Too Many Requests” response, until the window resets.
Common algorithms include fixed window (a hard reset every N seconds), sliding window (a rolling count that avoids edge-of-window spikes), token bucket (requests consume tokens that refill at a steady rate), and leaky bucket (requests are processed at a constant output rate regardless of burst size). Each trades off burst tolerance, fairness, and implementation complexity differently.
What happens when a rate limit is exceeded?
Depending on the system’s design, an over-limit request is typically either rejected outright with a 429 status code, queued and processed once capacity frees up, or throttled to a slower response rate. Well-designed APIs return headers indicating the limit, how many requests remain, and when the window resets, so clients can back off intelligently.
Client-side retries that fire immediately and repeatedly add more load right as the system is trying to shed it. This is why most API clients implement exponential backoff when they hit a 429.
Why does rate limiting matter?
A single misbehaving client, a traffic spike, or a runaway automated process can degrade performance for every other user or exhaust shared backend resources if requests go unchecked. Rate limiting is a core defense for API stability, fair usage, and cost control.
It’s increasingly important as AI agents and automated systems generate request volumes that no human would generate manually. An agent stuck in a retry loop, or one designed to poll aggressively, can produce request patterns that look nothing like human traffic, which is exactly the scenario rate limiting exists to contain.
