OpenAI API 429 Rate Limit: Causes, Retry Logic and Production Fixes

Last verified: July 21, 2026

Short path: Cross-provider resilience on production LLM API. Full multi-model policy: LLM 429 and timeouts.

HTTP 429 Too Many Requests on an OpenAI API integration (including OpenAI-compatible gateways) means the client exceeded a rate or concurrency limit — not that the model rejected the prompt logically. Production systems that retry blindly can amplify outages; systems that never retry frustrate users during brief spikes.

This guide focuses on Chat Completions through LumeAPI (https://api.lumeapi.site/v1). Principles apply to direct OpenAI as well, but limits and headers differ by account tier.

LumeAPI is an independent gateway — not OpenAI.

Quick Answer

Starting policy: max 3–5 attempts, initial delay 0.5–1s, cap delay 30s, add random jitter.

Why 429 happens on OpenAI-shaped APIs

Your app
  → burst of parallel requests
  → exceeds RPM / TPM / concurrent streams
  → 429 + optional Retry-After

Common triggers:

• Launch traffic spike without queue
• Too many parallel SSE streams
• Batch job and realtime sharing one key
• Autoscaling workers all retrying simultaneously (retry storm)

Gemini and Claude have analogous limits — see Gemini 429 guide.

Python retry with exponential backoff

import os
import random
import time
from openai import OpenAI, RateLimitError, APIStatusError

client = OpenAI(
    api_key=os.environ["LUMEAPI_KEY"],
    base_url="https://api.lumeapi.site/v1",
    timeout=60.0,
    max_retries=0,  # we control retries explicitly
)

def complete_with_retry(messages, model="gpt-5.4-mini", max_attempts=5):
    delay = 0.5
    for attempt in range(max_attempts):
        try:
            return client.chat.completions.create(
                model=model,
                messages=messages,
                max_tokens=1024,
            )
        except RateLimitError as exc:
            if attempt == max_attempts - 1:
                raise
            retry_after = None
            if exc.response is not None:
                retry_after = exc.response.headers.get("retry-after")
            sleep = float(retry_after) if retry_after else delay + random.uniform(0, 0.25)
            time.sleep(min(sleep, 60))
            delay = min(delay * 2, 30)
        except APIStatusError as exc:
            if exc.status_code >= 500 and attempt < max_attempts - 1:
                time.sleep(delay + random.uniform(0, 0.25))
                delay = min(delay * 2, 30)
                continue
            raise

Do not retry 401/404 unchanged. For side-effecting tool calls, add idempotency keys at the application layer.

Node.js equivalent

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.LUMEAPI_KEY,
  baseURL: 'https://api.lumeapi.site/v1',
  maxRetries: 0,
});

export async function completeWithRetry(messages, model = 'gpt-5.4-mini', maxAttempts = 5) {
  let delay = 500;
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    try {
      return await client.chat.completions.create({ model, messages, max_tokens: 1024 });
    } catch (err) {
      if (err instanceof OpenAI.RateLimitError && attempt < maxAttempts - 1) {
        await new Promise((r) => setTimeout(r, delay + Math.random() * 250));
        delay = Math.min(delay * 2, 30_000);
        continue;
      }
      throw err;
    }
  }
}

Concurrency limiting (prevent 429)

Retries fix symptoms; token buckets fix causes:

import asyncio

sem = asyncio.Semaphore(20)  # max 20 concurrent LLM calls

async def bounded_complete(messages):
    async with sem:
        return await asyncio.to_thread(
            complete_with_retry, messages
        )

Tune semaphore size from load tests. Streaming counts toward concurrency — each open SSE may hold a slot.

Application-level model fallback

LumeAPI does not silently swap models. Your code chooses fallbacks:

MODEL_CHAIN = ["gpt-5.4-mini", "gemini-3-flash", "claude-sonnet-4-6"]

def complete_resilient(messages):
    last_err = None
    for model in MODEL_CHAIN:
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except RateLimitError as e:
            last_err = e
            continue
    raise last_err

Log which model served each request — Usage should align with logs.

429 during streaming

If 429 occurs before stream start, retry the whole request. If mid-stream connection drops, treat as transport failure — recover stream guide.

Cap parallel streams per tenant during traffic spikes.

Monitoring

Track per model:

• 429 rate (% of requests)
• p95 latency
• retry count distribution
• cost per successful task

Alert when 429 rate > 1% for 5 minutes — scale queue or reduce concurrency before users notice.

When NOT to fix 429 with more retries

FAQ

Does a cheaper model reduce 429s?

Only if limits are token-based and the cheaper route has headroom. Often you need concurrency control.

OpenAI official vs LumeAPI limits?

Account tiers differ. Load-test your deployment; do not assume 1:1 parity with openai.com documentation.

Should I use SDK max_retries?

SDK defaults help prototypes. Production should use explicit backoff with metrics — see LLM fallback.

429 vs 503?

429 = rate limit (slow down). 503 = server/gateway stress (retry with circuit breaker).

How does this relate to agent loops?

Agents multiply call volume — agent bills guide.

Sources and verification

• OpenAI Python SDK error types and retries.
• LumeAPI production LLM API for gateway context.

Policy tables align with handle LLM 429 in production — this article is OpenAI-path specific.