A rate limit caps how many requests a client may send to an API in a period, such as a minute or a day. When the cap is reached, the server refuses further requests, usually with HTTP status 429, until the period resets. Limits protect the service and share its capacity between clients.
How limits are set
A limit is counted against an identity, most often an API key, and over a window. Two windows are common together: a short one, such as a minute, which controls bursts, and a long one, such as a day, which caps total volume. The server can count with a fixed window, a sliding window or a token bucket.
A refused request carries status 429, defined in RFC 6585, and the response may include a Retry-After header giving the wait in seconds or as a date. Headers that report the remaining quota differ between APIs.
Planning a load against two limits
Take an illustrative key allowed 60 requests a minute and 5,000 a day, with 500 rows to a page. The minute limit alone would allow 3,600 requests an hour, but the daily limit stops you after about 83 minutes of continuous calling, at 2,500,000 rows. A backfill of 40,000 pages therefore takes eight days of calls. In this case the daily limit decides how long the backfill takes, and the minute limit decides how you pace each day.
Three habits keep a client inside its limits:
- Space requests evenly and size pages as large as the API allows.
- Share one budget across all workers that use the same key, because the limit is per key.
- After a 429, wait for
Retry-Afterif it is present. Otherwise back off with a growing, randomised delay, and never retry in a tight loop.
In Fokals data
The Fokals API applies rate limits per key, by minute and by day, with one bearer key per client. The API documentation sets out the limits, so plan a long run around whichever of the two binds first. Use a bulk export for a first load, so the allowance goes to keeping the copy current with cursor pagination.
Related terms
- Cursor pagination: how a long result is fetched in pages, each page counting as a request.
- Incremental sync: the daily run that a rate limit has to accommodate.
- Web crawler: a program that reads websites and is itself held to request limits.
Frequently asked questions
What does HTTP 429 Too Many Requests mean?
It means the client has sent more requests than the server allows in the current period. The status is defined in RFC 6585, and the response may include a Retry-After header saying how long to wait. Stop sending, wait for that time, then continue at a lower rate. Retrying at once spends more of the limit and usually returns another 429.
What is the difference between a rate limit and a quota?
A rate limit controls the speed of requests, such as 60 a minute, and resets within seconds or minutes. A quota caps the total over a longer period, such as 5,000 a day or a month. Many APIs apply both, and whichever is reached first decides what you can send. The Fokals API applies limits per key by minute and by day.
How do I avoid hitting an API rate limit?
Spread requests evenly rather than in bursts, use the largest page size the API allows, and cache what you have already fetched. Keep one request budget per key and share it across workers. Honour Retry-After, add a random delay when backing off, and use a bulk export, not paging, for a first load of history.