Real-Time LLM API Bills Too High? Cut OpenAI, Claude and Gemini Costs by 50% With Batch APIs

Last updated: July 2026

Not every LLM request needs an immediate response.

A customer-facing chatbot must reply within seconds. But dataset classification, content tagging, nightly report generation, document extraction, evaluation runs and bulk content processing can often wait several minutes or hours.

Sending these workloads through a standard real-time API means paying for speed that the user does not need.

OpenAI, Anthropic and Google all provide asynchronous batch-processing options that can reduce supported model input and output prices by approximately 50%. Batch jobs also use separate capacity or rate-limit pools, making them useful for workloads that are too large or too slow to process efficiently through ordinary synchronous requests. ([OpenAI API][1])

Batch APIs are not the right answer for live chat, interactive agents or user-facing requests. They are designed for work that can be submitted now and collected later.

The most important question is:

Does this task need an immediate answer, or does it only need to be completed reliably by a deadline?

If the answer is "it can wait," a Batch API may cut model spending more effectively than changing models or switching API providers.

Quick Answer

Use a Batch API when your workload is:

• Large
• Repetitive
• Asynchronous
• Easy to divide into independent requests
• Not visible to a waiting user
• Able to tolerate delayed completion

Strong batch-processing use cases include:

• Dataset classification
• Information extraction
• Content moderation queues
• Product tagging
• Document summarization
• Embedding generation
• Offline evaluations
• Synthetic data generation
• Nightly AI reports
• Image or video generation jobs

Avoid Batch APIs for:

• Live chat
• Interactive AI agents
• Real-time search
• Coding copilots
• Voice assistants
• Customer-support conversations
• Requests requiring immediate tool use
• Workflows where later steps depend on earlier results

OpenAI says its Batch API provides 50% lower costs than synchronous endpoints, separate higher rate limits and completion within 24 hours. Google prices Gemini Batch requests at 50% of the equivalent interactive API rate. Anthropic also states that its Message Batches API costs 50% less than standard Claude API calls. ([OpenAI API][1])

What Is an LLM Batch API?

A normal synchronous request follows this pattern:

Send one request
→ Wait for the model
→ Receive the answer

A batch workflow is different:

Prepare many requests
→ Submit them as a batch job
→ Let the provider process them asynchronously
→ Check job status
→ Download the completed results

Instead of holding an application connection open for every request, the provider processes a queue of independent jobs using asynchronous infrastructure.

This trade creates the discount:

• The provider receives more scheduling flexibility.
• The workload does not compete directly with low-latency interactive traffic.
• Requests can be grouped and processed more efficiently.
• The developer accepts delayed rather than immediate results.

The model itself is not necessarily weaker. A supported batch request can use the same model and request parameters as the corresponding standard endpoint, although availability and feature support vary by provider.

Batch API vs Sending Requests in a Loop

A common mistake is to call a normal API repeatedly from a script and describe that as batch processing.

For example:

for item in records:
    call_realtime_api(item)

This is bulk usage, but it is still synchronous API traffic. It usually receives standard pricing and standard rate limits.

A true provider Batch API normally involves:

1. Creating a list or file of requests
2. Assigning each request a unique identifier
3. Uploading or submitting the request collection
4. Creating a batch job
5. Waiting for asynchronous processing
6. Retrieving success and error results

The discount applies only when the provider's actual batch-processing interface is used.

OpenAI Batch API Pricing

OpenAI's Batch API provides a 50% discount relative to its standard synchronous prices for supported models and endpoints. It also uses a separate pool of higher rate limits and offers a defined completion window of up to 24 hours. ([OpenAI API][1])

Current examples include:

Prices are per one million tokens and represent short-context rates. OpenAI applies different pricing to some long-context requests and also publishes separate cached-input and cache-write rates. ([OpenAI API][2])

OpenAI currently supports batch requests for several endpoints, including:

• Responses
• Chat Completions
• Embeddings
• Completions
• Moderations
• Image generation
• Image editing
• Video generation

Each input file can contain requests for only one model, and each individual request needs a unique custom_id so the returned result can be matched to the original item. ([OpenAI API][1])

OpenAI Cost Example

Assume an offline content-analysis job uses:

• 100 million input tokens
• 20 million output tokens
• GPT-5.6 Terra

Using the standard endpoint:

Input:
100 × $2.50 = $250

Output:
20 × $15 = $300

Total:
$550

Using OpenAI Batch:

Input:
100 × $1.25 = $125

Output:
20 × $7.50 = $150

Total:
$275

Estimated saving:

Monthly batch saving: $275
Reduction: 50%

This is the same standard price currently listed by LumeAPI for GPT-5.6 Terra. Therefore, for eligible asynchronous GPT-5.6 Terra workloads, OpenAI Batch and LumeAPI's published real-time rate may have similar token pricing, while serving different needs:

• OpenAI Batch: official asynchronous access
• LumeAPI: lower-cost standard real-time access through an OpenAI-compatible gateway

The best choice depends on whether the request needs an immediate response and whether the application requires official OpenAI features.

Claude Batch API Pricing

Anthropic's Message Batches API processes large collections of Claude requests asynchronously at 50% below standard API pricing. Anthropic lists batch processing as generally available and notes that it is not eligible for Zero Data Retention. ([Claude Platform][3])

Current Claude batch examples include:

Prices are per one million tokens. ([Claude Platform][4])

Claude Sonnet Cost Example

Assume a document-extraction workflow uses:

• 100 million input tokens
• 20 million output tokens
• Claude Sonnet 4.6

Using Anthropic's standard API:

Input:
100 × $3 = $300

Output:
20 × $15 = $300

Total:
$600

Using Anthropic Batch:

Input:
100 × $1.50 = $150

Output:
20 × $7.50 = $150

Total:
$300

LumeAPI currently lists Claude Sonnet 4.6 standard real-time pricing at:

Input: $1.80 per million tokens
Output: $9 per million tokens

That produces a monthly cost of:

100 × $1.80 + 20 × $9
= $180 + $180
= $360

For this workload:

Anthropic Batch is the lowest token-cost option when the work can be asynchronous.

LumeAPI may be more appropriate when the same Claude Sonnet requests must be completed in real time and standard OpenAI-compatible Chat Completions are sufficient.

Batch and Prompt Caching

Batch processing and prompt caching solve different problems.

Batch reduces the price of asynchronous requests.

Prompt caching reduces the cost of repeated prompt prefixes.

Anthropic documents both features, and a workload may benefit from combining them where supported. However, the pricing calculation must include normal input, cache creation, cache reads and batch rates rather than simply applying two discounts to the same number without checking provider rules. ([Claude Platform][4])

For prompt caching depth, see the [Prompt Caching guide](/research/prompt-caching-cut-openai-claude-gemini-api-costs-lumeapi).

Gemini Batch API Pricing

Google prices Gemini Batch API requests at 50% of the corresponding standard interactive API rate. It supports a range of Gemini models and uses separate batch rate limits. ([Google AI for Developers][5])

Representative prices include:

Output rates include thinking tokens where applicable. Gemini 3.1 Pro uses higher rates when prompts exceed 200,000 tokens. ([Google AI for Developers][6])

Gemini 3.5 Flash Cost Example

Assume a dataset-enrichment process uses:

• 100 million input tokens
• 20 million output tokens
• Gemini 3.5 Flash

Google standard pricing:

100 × $1.50 + 20 × $9
= $150 + $180
= $330

Google Batch pricing:

100 × $0.75 + 20 × $4.50
= $75 + $90
= $165

LumeAPI's published standard real-time price:

100 × $0.90 + 20 × $5.40
= $90 + $108
= $198

Comparison:

Once again, the official Batch API is less expensive for an eligible asynchronous workload, while LumeAPI offers a lower-cost option for real-time standard requests.

Google's Batch API also has separate limits. Its documentation currently lists up to 100 concurrent batch requests, a maximum input file size of 2 GB and model-specific limits on the number of enqueued tokens. ([Google AI for Developers][7])

OpenAI vs Claude vs Gemini Batch Pricing

The three providers share the same broad value proposition:

Accept delayed processing and receive approximately 50% lower token rates.

However, they differ in model selection, job format, limits, endpoint support and surrounding platform features.

OpenAI explicitly commits to a 24-hour completion window for Batch jobs. Processing details and expiration behavior should be reviewed separately for Anthropic and Gemini before building a time-sensitive pipeline. ([OpenAI API][1])

Five Workloads That Should Use Batch

1. Dataset Classification

Suppose you need to categorize one million support tickets into:

• Billing
• Technical problem
• Account access
• Product request
• Cancellation
• Other

No user is waiting for each individual result.

Create one request per ticket, submit the collection and process the completed output when the batch finishes.

Batch pricing can cut the inference portion of the job in half.

2. Information Extraction

Examples include extracting:

• Company names
• Product codes
• Dates
• Monetary values
• Contract clauses
• Sentiment
• Structured fields

Each document can be processed independently, which makes the workload naturally suitable for batch execution.

Structured-output validity still needs to be checked after completion. Invalid results should enter a controlled retry queue rather than causing the entire batch to be resubmitted.

3. Offline Evaluations

AI teams frequently test models against thousands of examples.

A model evaluation may compare:

• Prompt versions
• Model versions
• Structured-output accuracy
• Tool-call decisions
• Safety behavior
• Language performance
• Code quality

Evaluation results are rarely needed within seconds. Batch is therefore one of the most economical ways to run large test suites.

4. Bulk Content Generation

Batch can process:

• Product descriptions
• Metadata
• SEO title candidates
• Summaries
• Translation
• Email drafts
• Social-post variants

This does not mean publishing generated content without review. Batch only changes how the model requests are processed and billed.

Quality checks, duplication detection and human review may still be required.

5. Nightly AI Workflows

A system may collect work during the day and process it overnight:

• Summarize customer feedback
• Categorize support cases
• Analyze sales calls
• Generate account reports
• Enrich CRM records
• Review agent logs
• Identify failed automation steps

This produces a simple architecture:

Daytime:
Collect jobs

Night:
Submit batch

Next morning:
Validate and import results

Five Workloads That Should Stay Real Time

1. Live Chat

Users expect visible progress or a complete response within seconds.

Waiting for a batch job is unacceptable.

2. Interactive AI Agents

An agent's next action often depends on the previous model response. These calls cannot easily be grouped in advance because the request chain is sequential.

Some independent subtasks may be batched, but the main interactive loop generally remains real time.

3. Coding Assistants

Developers expect code suggestions, debugging help and explanations during an active session.

A 50% discount does not compensate for an hours-long delay.

4. User-Facing Search

A search or research interface must usually return results during the current session.

Offline indexing, document tagging and evaluation can use Batch, but the final user query should remain real time.

5. Transactional Decisions

Fraud checks, order decisions, authentication flows and automated approvals may require immediate responses and strict latency targets.

These workflows should not be moved to asynchronous processing solely to save inference costs.

Real-Time, Batch or LumeAPI?

A practical architecture does not need to choose only one route.

Use each option where it creates the most value.

Official Batch APIs

Best for:

• Offline jobs
• Large independent request collections
• Provider-native features
• Lowest supported asynchronous price
• Work that can tolerate delayed results

LumeAPI

Best for:

• Standard real-time text calls
• Lower-cost access to supported mainstream models
• OpenAI-compatible integrations
• One interface for GPT, Claude and Gemini
• Work that cannot wait for Batch completion

Official Standard APIs

Best for:

• Native provider features
• Enterprise agreements
• Provider-specific tools
• Direct support
• Strict compliance requirements
• Newly released models
• Workloads not supported through other routes

A hybrid routing policy may look like this:

Live chatbot requests
→ LumeAPI or official real-time API

Nightly classification
→ Official Batch API

Claude prompt-caching workload
→ Anthropic native API

Gemini Search Grounding
→ Google native API

OpenAI hosted tools
→ OpenAI native API

The goal is not to force all traffic through one endpoint.

The goal is to place each workload on the least expensive route that still supports its latency, quality and feature requirements.

For real-time catalog pricing across GPT, Claude and Gemini, see the [AI API pricing](/ai-api-pricing) page and the [High-volume LLM API](/high-volume-llm-api) commercial guide.

How to Calculate Whether Batch Is Worth It

Start with your current synchronous cost:

Standard cost =
Input tokens × standard input price
+ Output tokens × standard output price

Then calculate Batch cost:

Batch cost =
Input tokens × batch input price
+ Output tokens × batch output price

Add engineering and operational costs:

Total batch cost =
Batch inference cost
+ Job preparation
+ File storage
+ Result processing
+ Retry handling
+ Monitoring

Then compare:

Net batch saving =
Standard total cost
− Batch total cost

For a small job saving $5 per month, building a complex batch pipeline may not be worthwhile.

For a job saving $5,000 per month, the engineering effort is easier to justify.

Design a Reliable Batch Pipeline

A Batch API reduces inference cost, but it introduces operational responsibilities.

Give Every Request a Unique ID

The result order may not match the original input order.

Each request needs an identifier that maps it back to:

• Source record
• User
• Document
• Task type
• Prompt version
• Model version

Never depend only on line order.

Make Requests Idempotent

A retry should not accidentally apply the same business action twice.

For example, generating the same summary twice is usually harmless. Sending two customer emails or updating a financial record twice is not.

Separate model inference from side effects:

Batch inference
→ Validate result
→ Check task state
→ Apply business action once

Validate Every Result

A completed batch job may contain individual failed requests.

Check:

• HTTP or provider error
• Missing response
• Invalid JSON
• Truncated content
• Refusal
• Wrong language
• Schema mismatch
• Empty output

Do not import all results blindly.

Retry Only Failed Items

If 2% of a batch fails, create a smaller retry batch containing those items.

Do not resubmit the successful 98%, because that creates duplicated costs and potentially duplicated outputs.

Version Prompts and Models

Store:

prompt_version
model_id
batch_id
submitted_at
completed_at
custom_id

Without versioning, it becomes difficult to reproduce a result or compare old and new workflows.

Set a Deadline

Batch work should have an internal deadline earlier than the business deadline.

For example:

Provider completion target: 24 hours
Internal business deadline: tomorrow at 09:00
Submission deadline: today at 06:00

This leaves time for failures and retries.

Monitor Cost per Successful Record

The useful metric is not only the listed batch discount.

Use:

Cost per successful record =
Total batch and retry spending
÷ Number of valid imported results

A pipeline with many malformed outputs may not achieve the expected 50% saving.

Batch API and AI Agents

A live agent cannot usually send its entire execution as one batch because each next step depends on earlier results.

But parts of an agent system can still be batched.

Examples include:

• Pre-classifying incoming tasks
• Generating embeddings
• Evaluating completed agent runs
• Summarizing execution logs
• Creating next-day reports
• Extracting data from accumulated documents
• Scoring agent responses
• Producing synthetic test cases

A cost-efficient agent architecture can divide work into two paths:

Immediate path:
Planning, tools and user response
→ Real-time API

Offline path:
Evaluation, enrichment and analytics
→ Batch API

This can reduce total system cost without making the user wait.

Batch API and Prompt Caching

Batch processing and prompt caching can both reduce costs, but they target different inefficiencies.

Batch saves money because the provider can schedule work asynchronously.

Prompt caching saves money because stable input is reused.

For a workload containing a large repeated system prompt, the best route depends on:

• Whether Batch supports cached pricing for the chosen model
• Whether cache writes are billed separately
• How often the prompt is reused
• Whether the batch job preserves identical prefixes
• Whether provider caching parameters are supported in Batch mode

Do not assume:

50% batch discount
+ 90% cache discount
= 140% saving

Discounts apply to specific billing categories and cannot simply be added together.

Use the provider's actual usage response and pricing rules to calculate effective cost.

Frequently Asked Questions

It is an asynchronous API that processes a large group of model requests without returning every response immediately. Results are retrieved after the batch job is completed.

OpenAI, Anthropic and Gemini currently advertise approximately 50% lower token prices for supported Batch workloads. ([OpenAI API][1])

Batch pricing generally applies to supported versions of the same models. The main trade-off is delayed processing, not intentionally lower model capability.

You should still run evaluations because endpoint features and parameters may differ.

OpenAI states that Batch jobs complete within 24 hours and often finish sooner. ([OpenAI API][1])

Not for the live user response. You can use Batch for offline chatbot analytics, classification, evaluation and conversation summarization.

For supported asynchronous Gemini workloads, Google Batch prices are currently lower than LumeAPI's published standard real-time prices.

LumeAPI is intended for lower-cost real-time access, while Batch accepts delayed completion.

For supported Claude workloads, Anthropic's 50%-discounted batch rates are currently lower than LumeAPI's published standard real-time rates.

LumeAPI remains relevant when the request must be completed interactively.

For several supported GPT models, OpenAI's 50%-discounted batch prices are similar to the current LumeAPI real-time catalog prices.

The decision should therefore consider latency, provider-native features, compatibility and operational complexity.

OpenAI says each input file can contain requests to only one model. Provider rules differ, so check the relevant Batch documentation before preparing jobs. ([OpenAI API][1])

OpenAI and Google document separate Batch capacity or rate limits. Google currently publishes limits for concurrent batches, file size, storage and enqueued tokens. ([OpenAI API][1])

Final Recommendation

Many AI teams overpay because they send every workload through real-time APIs.

The correct first question is:

Is a user waiting for this result?

If yes, use a reliable real-time API.

If no, evaluate Batch.

OpenAI, Anthropic and Google currently offer approximately 50% lower pricing for supported asynchronous workloads. This makes Batch especially valuable for classification, extraction, evaluation, enrichment and nightly processing.

LumeAPI solves a different part of the cost problem. It offers lower standard real-time prices for supported GPT, Claude and Gemini models through one OpenAI-compatible gateway.

A cost-efficient production architecture can use both:

Official Batch APIs
→ Lowest-cost asynchronous processing

LumeAPI
→ Lower-cost supported real-time calls

Official standard APIs
→ Native tools, special features and enterprise requirements

Do not choose between Batch and real time at the company level.

Choose at the workload level.

Keep user-facing work fast. Move offline work to Batch. Validate every result, retry only failed items and measure cost per successfully processed record.

The easiest 50% API saving may not come from changing the model.

It may come from accepting that some requests never needed an immediate answer in the first place.

[1]: https://developers.openai.com/api/docs/guides/batch "Batch API | OpenAI API" [2]: https://developers.openai.com/api/docs/pricing "Pricing | OpenAI API" [3]: https://platform.claude.com/docs/en/build-with-claude/overview "Features overview | Claude Platform" [4]: https://platform.claude.com/docs/en/about-claude/pricing "Pricing | Claude Platform" [5]: https://ai.google.dev/gemini-api/docs/batch-api "Batch API | Gemini API" [6]: https://ai.google.dev/gemini-api/docs/pricing "Gemini Developer API pricing" [7]: https://ai.google.dev/gemini-api/docs/rate-limits "Rate limits | Gemini API"