Rate limits
Concurrency, requests per minute, retry headers and long-running requests.
Per-key limits
| Limit | Default | When you reach it |
|---|---|---|
| Concurrent requests | 5 | 429 with Retry-After: 1 until one of your requests finishes. |
| Requests per minute | Off unless set on your key | 429 with Retry-After set to the wait in seconds. |
To raise a key's limits, ask the person or team who issued your key.
Other 429 responses
- Credit held: your running requests have set aside all of the key's remaining credit.
Retry-After: 2. - Busy model: the model is receiving too many requests.
Retry-After: 2or5. - Failed keys: after 30 failed key attempts within a minute from one IP address, further attempts with a bad key
get
429withRetry-After: 60. Valid keys from the same address keep working.
Retrying
Every retryable 429 carries a Retry-After header in seconds. Wait that long, and add exponential backoff with
jitter if you get several in a row. A 429 for an out-of-credit key has no Retry-After and won't succeed until the
key is topped up. See Errors.
Long requests
- Stream anything that may take more than 30 seconds. ORYNX sends a heartbeat after 15 seconds of silence, so the connection stays open while the model thinks.
- Non-streaming requests that take longer than 30 seconds get status
200and a newline every 20 seconds until the answer is ready. JSON parsers ignore the extra whitespace, and proxies don't close the connection. - A request ends if the model sends nothing for 300 seconds, and no request can run longer than 60 minutes.
Key lookup
The Key lookup page is limited per network (a few lookups per minute and a daily cap) to protect keys from guessing. Repeated failed lookups block the network for an hour.