Paying for the Same Prompt Again and Again? Cut OpenAI, Claude and Gemini API Costs by Up to 90% with Prompt Caching

Last updated: July 2026

Many AI applications pay to process the same content repeatedly.

A customer-support assistant may send the same system instructions and knowledge base with every message. A coding agent may resend its tool definitions, repository guidelines and architecture notes at every step. A document assistant may attach the same long PDF whenever the user asks a new question.

Without prompt caching, these repeated tokens are billed as normal input again and again.

Prompt caching allows an API provider to reuse previously processed prompt content. When a later request begins with the same stable prefix, the provider can avoid recomputing that portion of the prompt and charge a lower cached-input rate.

Depending on the provider, model and workload, cached input can cost up to 90% less than normal input. Prompt caching may also reduce time to first token because the model does not need to process the entire repeated prefix again.

OpenAI, Anthropic and Google all provide caching, but they implement it differently:

• OpenAI primarily uses automatic prefix caching.
• Anthropic supports explicit cache breakpoints and selectable retention periods.
• Gemini offers implicit caching and, through supported APIs, explicitly managed context caches.

The best provider is not simply the one advertising the largest cache discount. Your real savings depend on prompt structure, cache hit rate, expiration, storage charges and how frequently the same context is reused.

Quick Answer

Prompt caching is most valuable when requests repeatedly include large, stable content such as:

• System prompts
• Tool definitions
• Product documentation
• Coding standards
• Large reference documents
• Repeated conversation prefixes
• Examples and output schemas
• User-uploaded files queried multiple times

It is less useful when most of the prompt changes on every request.

A simplified prompt-caching cost formula is:

Total input cost = uncached input cost + cache-write cost + cache-read cost + cache-storage cost

Not every provider charges all four components.

OpenAI automatically caches eligible repeated prompt prefixes and requires no caching-specific code for normal use. Its documentation says prompt caching is enabled for recent models and can substantially reduce cached-input cost and latency. ([OpenAI API][1])

Anthropic lets developers mark stable sections with cache controls. Its standard five-minute cache writes cost more than normal input, but subsequent cache reads can cost only 10% of the standard input rate. Anthropic also supports longer cache retention at a higher write cost. ([Anthropic Cookbook][2])

Gemini supports automatic implicit caching and explicitly managed context caches in supported APIs. Google charges discounted rates for cached tokens, while explicit caches may also generate storage charges based on token volume and retention time. ([Google AI for Developers][3])

Prompt caching can reduce input spending dramatically, but it does not discount:

• Newly added user content
• Changed prompt sections
• Output tokens
• Search or tool charges
• Failed requests
• Content that no longer matches the cached prefix

What Is Prompt Caching?

When an LLM receives a prompt, it processes the input tokens before generating an answer.

Many production prompts contain a stable section followed by dynamic content:

Stable system instructions
Stable tool definitions
Stable product documentation
Stable output schema
Dynamic conversation history
Dynamic user request

The stable section may be thousands or hundreds of thousands of tokens long.

If every request starts with exactly the same stable prefix, the provider may reuse previously computed information instead of processing that prefix from the beginning.

Conceptually, the first request looks like this:

Process stable prefix
+ Process new request
+ Generate response

A later cache hit looks more like:

Reuse cached stable prefix
+ Process new request
+ Generate response

The model still receives the necessary context. The difference is that the provider avoids recomputing the repeated portion.

Prompt caching normally does not mean the model permanently learns your content. It is an inference optimization tied to repeated request content, cache rules and provider retention policies.

Prompt Caching vs Application Caching

Prompt caching should not be confused with caching completed answers.

Response Caching

Your application stores a previous answer and returns it again when another request is identical or sufficiently similar.

This can avoid an LLM request completely.

It works well for:

• Frequently asked questions
• Deterministic classification
• Repeated summaries
• Standard product information
• Identical generated assets

But response caching is risky when answers depend on changing data, user permissions or personalized context.

Prompt Caching

The model still processes the new request and generates a new response. Only the repeated prompt prefix is reused.

It works well when users ask different questions about the same:

• Document
• Codebase
• System instructions
• Tools
• Product knowledge
• Conversation history

Semantic Caching

A separate system decides that two requests have similar meanings and may reuse an earlier result.

Semantic caching can produce larger savings because it may avoid the model call, but it requires similarity thresholds, freshness rules and careful quality control.

These methods can be combined:

Exact response cache
→ Semantic cache
→ Prompt cache
→ Normal model inference

Why Prompt Caching Matters for Production Applications

Consider an AI support assistant with:

• A 10,000-token system prompt and policy guide
• 2,000 tokens of tool definitions
• 1,000 tokens of dynamic conversation content
• 500 output tokens

Without caching, every request processes approximately 13,000 input tokens.

Across 100,000 monthly requests:

13,000 × 100,000
= 1.3 billion input tokens

But 12,000 of the 13,000 tokens are repeated instructions and tools.

If those stable tokens achieve a high cache-hit rate, most of the 1.2 billion repeated tokens may be billed at a discounted cached-input rate rather than the standard input rate.

This is why prompt architecture matters as much as model price.

Changing providers may reduce a model's standard input price by 40%, but improving a near-zero cache-hit rate to a high hit rate can reduce the cost of eligible repeated tokens by substantially more.

The strongest cost strategy often combines:

1. A competitive base API price
2. High cache reuse
3. Shorter dynamic context
4. Controlled output length
5. Fewer agent loops

OpenAI Prompt Caching

OpenAI prompt caching works automatically for eligible requests. Developers generally do not need to create a cache object or add a special cache-control field.

OpenAI attempts to reuse an exact prompt prefix that was processed recently. Its documentation states that prompt caching applies to recent model families and can lower cached-input cost while improving time to first token. ([OpenAI API][1])

How OpenAI Cache Matching Works

The beginning of the prompt is the most important part.

A request might be structured as:

1. Stable system instructions
2. Stable tool definitions
3. Stable examples
4. Stable product context
5. Dynamic user-specific content
6. Current user request

When a later request begins with the same stable content, the provider may reuse that prefix.

Changing content near the beginning can invalidate the cache for everything that follows it.

For example, this structure is inefficient:

Current timestamp
Random request ID
User-specific metadata
Stable system instructions
Stable tools
Stable documentation

The timestamp and random ID change on every request, so the prompt stops matching before it reaches the expensive stable content.

A better structure is:

Stable system instructions
Stable tools
Stable documentation
User-specific metadata
Current timestamp
Current request

What OpenAI Can Cache

Eligible repeated content may include:

• Messages
• Images
• Tool definitions
• Structured-output schemas
• Stable prompt prefixes

Cache usage is reported through usage details in API responses, allowing developers to compare total input with cached input.

OpenAI Prompt Design Recommendations

To improve the probability of cache hits:

• Put static instructions first.
• Put changing user content last.
• Keep tool definitions in a stable order.
• Avoid generating tool schemas dynamically.
• Do not insert timestamps or request IDs into the prefix.
• Keep stable examples unchanged.
• Reuse consistent system-message wording.
• Monitor cached-token fields in every production request.

OpenAI's automatic approach is easy to adopt because there is little additional application logic. The trade-off is that developers have less direct control over cache creation and expiration than with explicitly managed caching systems.

Anthropic Prompt Caching

Anthropic provides explicit prompt caching for Claude.

Instead of relying only on automatic prefix detection, developers can mark selected prompt blocks as cacheable. The API creates cache breakpoints, and later requests can reuse matching content before those breakpoints.

Anthropic's published caching structure generally includes:

• Normal input tokens
• Cache-creation input tokens
• Cache-read input tokens
• A default short retention period
• An optional longer retention period

Anthropic's cookbook states that standard cache writes cost 1.25 times the base input rate, while cache reads cost 0.1 times the base input rate. ([Anthropic Cookbook][2])

Why Cache Writes Cost More

The first request must create the cache.

If normal input costs $3 per million tokens, a five-minute cache write using a 1.25 multiplier would cost:

$3 × 1.25 = $3.75 per million cached-write tokens

A subsequent cache read at 0.1 times the base rate would cost:

$3 × 0.1 = $0.30 per million cached-read tokens

The first request is therefore more expensive than ordinary input, but repeated reuse can recover that cost quickly.

Anthropic Break-Even Example

Assume a stable prompt contains one million tokens purely to make the arithmetic clear.

Without caching, four requests at a $3 standard input rate cost:

4 × $3 = $12

With a five-minute cache:

First cache write: $3.75
Three cache reads: 3 × $0.30 = $0.90
Total: $4.65

Estimated saving:

$12 − $4.65 = $7.35

That is a reduction of approximately 61.25% across the four requests.

As the number of cache reads increases, the effective reduction approaches the cache-read discount.

When Anthropic Caching Works Best

It is especially useful for:

• Long system instructions
• Large codebase context
• Repeated tool definitions
• Multi-turn conversations
• Agents performing many steps within a short period
• Multiple questions about the same document
• Standardized business workflows

Anthropic Cache Risks

Caching may save little or even increase costs when:

• The content is reused only once.
• The cache expires before the next request.
• The marked prefix changes frequently.
• Requests are spread too far apart.
• Dynamic content is placed inside the cacheable prefix.
• The cache is recreated repeatedly without enough reads.

Developers should monitor at least:

cache_creation_input_tokens
cache_read_input_tokens
uncached_input_tokens
cache_hit_rate
cost per task

Do not judge the implementation only by whether caching is technically enabled.

Gemini Context Caching

Google uses the term context caching.

Gemini provides two main approaches:

Implicit Caching

Eligible repeated prompt prefixes may be cached automatically.

Developers do not need to create a cache resource. Google detects reusable context and applies cached-token billing when requirements are met.

Google's current documentation describes implicit caching as a built-in cost and performance optimization. ([Google AI for Developers][3])

Explicit Caching

In supported Gemini APIs, developers can create and manage a cache containing reusable content.

The application can then reference that cache in multiple requests.

Explicit caching is useful when developers know in advance that the same large context will be reused, such as:

• A collection of uploaded documents
• A long video
• A large codebase
• A fixed knowledge base
• Repeated multimodal context
• A large set of product instructions

Google's caching API describes context caching as a way to save and reuse precomputed input tokens across repeated requests. ([Google AI for Developers][4])

Gemini Storage Charges

Gemini pricing can include:

1. A discounted price for cached input tokens
2. A storage price based on cached token volume and time

For some current models, Google lists storage prices per one million tokens per hour. Exact rates depend on the selected Gemini model and processing tier. ([Google AI for Developers][5])

This means a cache that remains active for many hours but receives very few requests may not create meaningful savings.

Gemini Break-Even Formula

A simplified Gemini calculation is:

Total caching cost =
Initial processing
+ Cached-read tokens
+ Storage tokens × Storage duration

Compare it with:

No-cache cost =
Repeated context tokens
× Number of requests
× Standard input rate

Explicit caching makes sense when:

No-cache cost > Cache creation + Cache reads + Storage

The more frequently the context is reused during its retention period, the more attractive caching becomes.

OpenAI vs Claude vs Gemini Prompt Caching

There is no universal winner.

OpenAI is attractive when a team wants automatic caching with minimal code changes.

Anthropic is attractive when a team wants explicit control over which prompt sections are cached and can reuse them frequently enough to offset write costs.

Gemini is attractive for explicitly caching large documents, media or multimodal context, provided storage costs are managed carefully.

Real Cost Scenario 1: Customer-Support Assistant

Assume each request contains:

• 20,000 stable instruction and policy tokens
• 2,000 dynamic input tokens
• 1,000 output tokens
• 100,000 requests per month

Total uncached input without caching:

22,000 × 100,000
= 2.2 billion input tokens

Of those:

20,000 × 100,000
= 2 billion tokens

are repeated stable content.

Suppose the application achieves an 80% cache-hit rate on the stable prefix.

Then:

Cached repeated tokens:
2B × 80% = 1.6B

Uncached repeated tokens:
2B × 20% = 400M

Dynamic tokens:
200M

Instead of paying the normal input rate for all 2.2 billion input tokens, the application pays:

• Cached rate for 1.6 billion
• Normal or write rate for the remaining repeated input
• Normal input rate for 200 million dynamic tokens

Depending on the model, this can reduce total input spending by thousands of dollars per month.

However, output cost remains unchanged.

Real Cost Scenario 2: Coding Agent

Assume a coding agent repeatedly sends:

• 30,000 tokens of repository instructions and selected code
• 5,000 tokens of tool definitions
• 5,000 dynamic tokens per step
• Eight model calls per task
• 10,000 tasks per month

Without caching, stable content processed per task is:

35,000 × 8
= 280,000 tokens

Across 10,000 tasks:

280,000 × 10,000
= 2.8 billion repeated input tokens

Caching the stable repository context and tool definitions can materially reduce that repeated cost.

But the prompt must be designed carefully.

If the agent inserts a changing execution plan before the codebase context, the expensive codebase prefix may fail to match.

The recommended order is:

Stable system prompt
Stable repository instructions
Stable tool definitions
Stable coding standards
Dynamic task state
Dynamic tool results
Current action

Do not place frequently changing error logs or timestamps before the stable cacheable content.

Real Cost Scenario 3: Document Question Answering

Assume a user uploads a 200,000-token collection of documents and asks 20 questions.

Without caching:

200,000 × 20
= 4 million repeated document tokens

With explicit caching:

• The documents are processed and cached once.
• Each new question references the same cached context.
• The application pays cached-read pricing and, where applicable, storage.

This is one of the strongest prompt-caching use cases because the reusable context is large and the user asks multiple questions within a defined period.

Caching may be less effective when every user uploads a document and asks only one question.

In that case, the cache-write and storage overhead may produce no return.

Why Your Cache Hit Rate Is Low

Enabling caching is not enough. Many applications accidentally invalidate the reusable prefix.

Dynamic Content Appears Too Early

Bad:

Current timestamp
User ID
Session ID
Stable system prompt
Stable tools
Stable documentation

Better:

Stable system prompt
Stable tools
Stable documentation
User ID
Session ID
Current timestamp
Current request

Tool Order Changes

If tool definitions appear in a different order, the prompt prefix may no longer match.

Use a deterministic tool ordering strategy.

Tool Schemas Are Generated Dynamically

Small schema changes can invalidate a large prefix.

Version schemas deliberately and avoid adding dynamic descriptions.

Whitespace or Formatting Changes

Depending on provider behavior, even minor changes can prevent an exact-prefix match.

Keep prompt templates centralized and version controlled.

Documents Are Reordered

Two prompts containing the same document sections in a different order may not reuse the same cache.

Use stable document ordering.

Model IDs Change

Caches are generally model-specific. Switching models should not be expected to preserve an existing cache.

The Cache Expires

A five-minute cache is ineffective when requests arrive every 20 minutes.

Measure actual time between repeated requests before selecting retention behavior.

The Prompt Is Too Short

Providers may apply minimum requirements before content becomes eligible for caching.

Caching is primarily intended for substantial repeated prefixes, not tiny system messages.

How to Design Prompts for Better Cache Reuse

Use a stable-first architecture:

1. Stable role and rules
2. Stable tools
3. Stable examples
4. Stable documents or product knowledge
5. Semi-stable conversation history
6. Dynamic tool output
7. Current user message

Additional recommendations:

• Keep the system prompt identical.
• Do not inject current dates unless required.
• Place request IDs outside the model prompt when possible.
• Keep tools sorted consistently.
• Load only relevant tools, but keep each tool set deterministic.
• Store raw execution logs outside the active prompt.
• Summarize old conversation turns.
• Separate cacheable documents from changing task state.
• Track prompt-template versions.
• Avoid unnecessary A/B prompt variants in production traffic.
• Send high-volume workloads through consistent model configurations.

Prompt caching rewards consistency.

Measure Cache Economics Correctly

Track these metrics for every model and workflow:

Cache Hit Rate

Cache hit rate =
Cached eligible tokens
÷ Total eligible repeated tokens

Effective Input Price

Effective input price =
Total input spending
÷ Total input tokens

Cost per Request

Useful for simple chat and generation endpoints.

Cost per Successful Task

Essential for agents and multi-step workflows:

Cost per successful task =
All model and tool spending
÷ Successfully completed tasks

Time to First Token

Caching can improve perceived responsiveness even when total generation time changes little.

Cache Waste

Measure cache writes that expire without being read.

A high number of unused cache writes indicates that the caching strategy may be increasing rather than reducing costs.

Prompt Caching and LumeAPI

LumeAPI provides lower standard real-time prices for supported GPT, Claude and Gemini models through one OpenAI-compatible API.

However, LumeAPI's current public documentation does not publish a complete provider-specific prompt-caching compatibility and billing matrix.

Developers should not assume that every upstream caching feature, parameter or discount is passed through automatically.

Before designing a cache-dependent LumeAPI workload, confirm:

• Whether the selected model supports cached input through LumeAPI
• Whether caching is automatic or explicit
• Whether cache-control parameters are accepted
• Whether cached tokens appear in usage data
• How cache writes and reads are billed
• Whether explicit cache storage is supported
• Whether provider-native cache objects can be managed
• What retention period applies

If native prompt caching is essential to the economics of your application, the official model provider may currently be the safer route.

LumeAPI is most compelling when its lower standard real-time rates produce savings without requiring undocumented provider-specific caching features.

A hybrid architecture may be appropriate:

LumeAPI
→ Lower-cost standard real-time requests

Official Anthropic API
→ Claude prompt caching and Batch workloads

Official Gemini API
→ Explicit context caches and Gemini-native features

Official OpenAI API
→ Automatic caching and OpenAI-native APIs

The right routing depends on the workload rather than loyalty to one provider.

For document Q&A, agent loops and high-volume chat where caching may not apply, see the [RAG API](/rag-api) and [High-volume LLM API](/high-volume-llm-api) commercial pages for LumeAPI catalog pricing and integration paths.

When Prompt Caching Is Worth Using

Caching is usually worthwhile when:

• The repeated prefix is large.
• The content remains unchanged.
• Requests occur within the cache lifetime.
• The cache is read many times.
• Normal input prices are significant.
• Time to first token matters.
• The application can maintain stable prompt ordering.

Strong use cases include:

• Document chat
• Coding agents
• Research agents
• Customer-support systems
• Large tool-enabled agents
• Repeated multimodal analysis
• Enterprise policy assistants
• Long-running user sessions

When Prompt Caching Is Not Worth Using

Avoid overengineering caching when:

• Prompts are short.
• Every request is substantially different.
• Content is reused only once.
• Requests arrive after the cache expires.
• Storage charges exceed read savings.
• Prompt templates change constantly.
• Most cost comes from output rather than input.
• Batch processing or response caching is more appropriate.

A 90% cached-token discount does not mean a 90% reduction in the total API bill.

Suppose input represents only 30% of spending and half of that input becomes cached at a 90% discount.

The total saving would be:

30% input share
× 50% eligible cached tokens
× 90% cached discount
= 13.5% total bill reduction

Always calculate the effect on the full workload.

Frequently Asked Questions

Prompt caching allows an AI provider to reuse previously processed prompt content when later requests contain the same stable prefix. Cached input is generally cheaper and may be processed faster than ordinary input.

Prompt caching is intended to reuse prompt computation rather than return a saved answer. The model still generates a new response for the current request.

Cached input can cost up to 90% less than normal input on some models and providers. Total application savings are lower when only part of the prompt is cacheable or when output dominates the bill.

OpenAI says eligible prompt caching works automatically for supported recent models. Developers should still structure prompts with stable content first and inspect cached-token usage. ([OpenAI API][1])

Anthropic allows developers to mark cacheable prompt sections with cache controls. Cache writes cost more than normal input, while successful cache reads can cost substantially less. ([Anthropic Cookbook][2])

Yes. Gemini provides implicit caching for eligible repeated context. Supported Gemini APIs can also provide explicitly managed caches. ([Google AI for Developers][3])

Explicit Gemini context caches may incur storage charges based on the cached token volume and retention period. Rates vary by model. ([Google AI for Developers][5])

Common causes include changing content near the beginning of the prompt, inconsistent tool ordering, expired caches, model changes, dynamic schemas and insufficient repeated content.

They solve different problems.

RAG selects relevant external information. Prompt caching reduces the cost of repeatedly processing stable information. A RAG system can cache frequently reused retrieved context.

Caching is suitable for repeated real-time context. Batch processing is suitable for asynchronous workloads. They can sometimes be combined, depending on provider support.

LumeAPI's current public documentation does not provide a complete caching feature and pricing matrix for every model. Confirm support and billing for the exact model before relying on cached-input savings.

Final Recommendation

Prompt caching is one of the most effective ways to reduce LLM input costs, but only when the application repeatedly processes stable content.

The most important principles are:

1. Put stable content at the beginning of the prompt.
2. Put dynamic content at the end.
3. Keep tool definitions and schemas deterministic.
4. Monitor actual cached tokens.
5. Measure unused cache writes.
6. Include expiration and storage in the calculation.
7. Optimize for cost per successful task, not only cost per token.

Use OpenAI when automatic prompt caching and OpenAI-native features fit the workload.

Use Anthropic when explicit cache breakpoints and controlled retention are valuable for Claude workflows.

Use Gemini when large reusable text, document or multimodal contexts justify implicit or explicitly managed caching.

Use LumeAPI when lower standard real-time prices for supported mainstream models create a stronger saving than provider-specific caching—and verify any caching dependency before production migration.

Prompt caching does not eliminate API costs. It eliminates the unnecessary cost of repeatedly processing the same stable context.

For chatbots, document assistants, coding agents and tool-heavy AI systems, that difference can transform an expensive prototype into a sustainable production product.

[1]: https://developers.openai.com/api/docs/guides/prompt-caching "Prompt caching | OpenAI API" [2]: https://github.com/anthropics/anthropic-cookbook/blob/main/misc/prompt_caching.ipynb "Anthropic prompt caching cookbook" [3]: https://ai.google.dev/gemini-api/docs/caching "Context caching | Google AI for Developers" [4]: https://ai.google.dev/api/caching "Caching | Gemini API" [5]: https://ai.google.dev/gemini-api/docs/pricing "Gemini Developer API pricing"