You ship a feature that calls an LLM on every user request. System prompt, tool definitions, few-shot examples — the same 2,000 tokens, processed from scratch, every single call. A month later the bill arrives and you are paying full input price for content that literally never changes between requests.
Prompt caching fixes this. Both Anthropic and OpenAI cache repeated prompt prefixes. On cache hits you pay a fraction of the normal input cost — 10% on Anthropic, 50% on OpenAI. For workloads with large, stable system prompts that gets you from "ouch" to "manageable" fast.
How Prompt Caching Works
The idea is simple. When you send a request, the provider checks whether a prefix of your prompt matches something recently processed. If it does, those tokens are read from cache. You pay the cache-read rate (much cheaper) instead of the full input rate.
Both providers cache the full prefix — system prompt, tools, and messages up to a designated breakpoint. If the prefix is new, they process it normally and cache it for the next call.
The cache is ephemeral. Anthropic defaults to 5 minutes (free refresh every use), OpenAI to roughly 30 minutes. Think of it as "repeated requests close together in time."
Anthropic: Automatic Caching (One Line)
Anthropic offers the simplest setup. Add one field to your request:
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-sonnet-4-5-20250901", # or claude-opus-4-8
max_tokens=1024,
cache_control={"type": "ephemeral"},
system="You are a code reviewer. Check for security issues, performance problems, and style violations. Always suggest concrete fixes.",
messages=[{"role": "user", "content": "Review this: def foo(x): return eval(x)"}],
)
# Check cache usage in the response
print(response.usage.cache_read_input_tokens) # tokens read from cache
print(response.usage.cache_creation_input_tokens) # tokens written to cache
That cache_control={"type": "ephemeral"} is all you need. Anthropic automatically caches everything up to the last cacheable block. On the first call it writes to cache (at 1.25× the normal input rate). On subsequent calls with the same prefix, it reads from cache (at 0.1× the normal input rate).
For a system prompt with 2,000 tokens, the math works out like this:
| First call | Second call | 100th call (total) | |
|---|---|---|---|
| Without caching | 2,000 × $15/MTok = $0.03 | $0.03 | $3.00 |
| With caching | 2,000 × $18.75/MTok = $0.038 | 2,000 × $1.50/MTok = $0.003 | ~$0.34 |
That is roughly 90% savings after the first call hits the same prefix. The numbers use Claude Opus 4.8 pricing ($15/MTok input, $18.75/MTok cache write, $1.50/MTok cache read). Cheaper models like Claude Sonnet 4.5 have the same multiplier structure at lower absolute prices.
Cache lifetime is 5 minutes by default. Each time the cached content is used, the timer resets — free. You can also opt into a 1-hour TTL at 2× the normal input rate for cache writes.
Anthropic: Explicit Cache Breakpoints
Automatic caching is convenient but hands control to the system. If you want to cache specific blocks and leave others uncached, use explicit breakpoints:
response = client.messages.create(
model="claude-sonnet-4-5-20250901",
max_tokens=1024,
system=[
{
"type": "text",
"text": "You are a code reviewer. Check for security issues.",
"cache_control": {"type": "ephemeral"}, # Cache the system prompt
},
{
"type": "text",
"text": f"Review code in the following language: {user_language}",
# No cache_control — this changes per request, not worth caching
},
],
messages=[{"role": "user", "content": user_code}],
)
Only the block with cache_control gets cached. The dynamic language hint does not waste cache space. This matters when you have a mix of static and dynamic content — cache what repeats, skip what changes.
You can also mark the last message in a conversation as the breakpoint so the entire history is cached:
messages = conversation_history + [
{
"role": "user",
"content": [
{"type": "text", "text": latest_question},
{"type": "text", "text": "End of conversation turn.",
"cache_control": {"type": "ephemeral"}},
],
}
]
On the next turn, the entire history up to and including that marker is read from cache. Long conversations get substantially cheaper.
OpenAI: Automatic Caching (Zero Code Changes)
OpenAI takes a different approach. Prompt caching is automatic for prompts longer than 1,024 tokens on supported models (GPT-4o, GPT-4o-mini, o1, o3, o4-mini). You do not need to change your code at all — the service handles it.
Check your usage response to see if caching kicked in:
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "You are a SQL expert. Given a question, output only the SQL query. No explanations."},
{"role": "user", "content": "Find all users who signed up last month"},
],
)
usage = response.usage
print(f"Input tokens: {usage.prompt_tokens}")
print(f"Cached tokens: {usage.prompt_tokens_details.cached_tokens}")
# If cached_tokens > 0, you saved money on this call
On cache hits, OpenAI applies a 50% discount to the cached input tokens. Cache writes cost 1.25× the normal rate on newer models (GPT-4.1+, GPT-5.6+). On older models (GPT-4o, o1), cache writes are free — you only benefit from reads without paying a write penalty.
Prompt cache key. If multiple requests share the same long prefix, use prompt_cache_key to improve hit rates:
response = client.chat.completions.create(
model="gpt-4.1",
messages=[...],
extra_body={
"prompt_cache_key": "tenant:acme:legal-reviews-v1",
},
)
This helps OpenAI route requests with the same key to the same machine, where the cache is more likely to be warm. Use it when you have a multi-tenant app and each tenant has their own system prompt or knowledge base.
Real Patterns That Save Money
Here are three patterns where prompt caching makes a measurable difference.
Pattern 1: Large System Prompts
A customer support bot with a 4,000-token system prompt (brand guidelines, product catalog, escalation rules, tone-of-voice examples). Without caching: $0.06 per call on GPT-4o ($2.50/MTok × 4K tokens × 6 calls/min = ~$21/hr). With caching: 50% off the input tokens, roughly $10.50/hr.
Same workload on Claude Sonnet with a 90% cache-hit discount: the system prompt cost drops to near-zero after the first call.
Pattern 2: Tool-Heavy Agents
Agents that call LLMs with 20 tool definitions (often 3,000+ tokens of JSON schema). Those definitions are identical across hundreds of calls in the same session:
tools = [
{"name": "search_kb", "description": "Search the knowledge base...", "input_schema": {...}},
{"name": "create_ticket", "description": "Create a support ticket...", "input_schema": {...}},
# ... 18 more
]
# First call: tools cached at 1.25× rate
# Subsequent calls in the same 5-min window: tools read from cache at 0.1× rate
response = client.messages.create(
model="claude-sonnet-4-5-20250901",
max_tokens=1024,
cache_control={"type": "ephemeral"},
tools=tools,
messages=[{"role": "user", "content": "Find the VPN setup guide"}],
)
Pattern 3: Multi-Turn Conversations
A chat app where each new message appends to a growing history. On turn 20, the conversation might be 8,000 tokens. Without caching, every turn processes the full history at the full input rate. With Anthropic explicit breakpoints marking the latest message, only the new user message and the assistant response are processed at full rate — the first 7,500 tokens hit the cache at 10% cost.
OpenAI handles this automatically for conversations above 1,024 tokens.
When to Use Anthropic vs OpenAI Caching
Neither is strictly better. It depends on your workload.
Use Anthropic when:
- Your system prompt or tool definitions are large and stable (the 90% discount on reads wins)
- You need explicit control over what gets cached
- You are running agentic loops with tool calls (the same tools and system get reused)
- You want the guaranteed 5-minute TTL with free refresh
Use OpenAI when:
- You want zero code changes (it just works for prompts >1,024 tokens)
- Your workload has moderate prefix reuse (50% discount is solid for low-effort setup)
- You are on GPT-4o or o-series models where cache writes are free
- You prefer the 30-minute TTL without needing to manage it
Use both if you are routing between providers. The patterns are similar enough that you can enable caching on both sides with minimal code.
Common Pitfalls
Short prompts do not cache. OpenAI requires a minimum prefix of 1,024 tokens. Anthropic requires at least 1,024 tokens for Claude Sonnet and 2,048 tokens for Claude Opus. A 500-token system prompt will not be cached by either provider.
Cache writes cost more than normal input. On Anthropic, a cache write costs 1.25× the normal rate. On newer OpenAI models, same multiplier. If you send a different prefix on every request, you pay the write penalty without ever getting a read discount. That is strictly worse than no caching.
Dynamic prefixes break cache hits. If your system prompt includes a timestamp or a user ID at the beginning, the prefix changes on every call. The cache never matches. Put dynamic content after the static content (at the end of the prompt, not the start). Or use explicit breakpoints to mark only the static part as cacheable.
Too many cache breakpoints dilute the benefit. Each breakpoint splits the prefix. On OpenAI, only the longest matching prefix up to 50 breakpoints is considered. On Anthropic, only content before the designated breakpoint is cached. Pick one or two strategic breakpoints, not a dozen.
Monitoring is free — use it. Both providers expose cache hit/miss data in the usage response. Set up a dashboard or log. If your cache-hit rate is below 30%, your caching strategy is not working. The most common cause: the prefix changes between calls and you did not notice.
What We Did Not Cover
Prompt caching also reduces latency. Cached tokens are not re-processed through the full model forward pass, which shaves 100-500ms off response times for large prefixes. The latency win matters most in real-time chat UIs where every millisecond counts.
Anthropic also offers batch processing (50% discount, 24-hour turnaround) and OpenAI has a Batch API (same idea). These stack with caching — you can batch requests that share cached prefixes and compound the savings. Different tradeoff (throughput vs latency), but worth knowing about.
Next Steps
- Read Anthropic prompt caching docs for the full API reference, including 1-hour TTL setup and cache diagnostics
- Read OpenAI prompt caching guide for the Responses API variant and prompt cache key details
- Check your current API bills. Look for calls where input tokens dwarf output tokens — those are your caching candidates
- Enable caching on one endpoint and measure the difference over a week. The numbers will tell you more than any blog post can