---
title: How does provider prompt caching cut AI agent costs?
url: https://opencontroller.lyzr.com/blog/provider-prompt-caching-for-agents
published: 2026-11-15
updated: 2026-11-15
author: Lyzr
---

# How does provider prompt caching cut AI agent costs?

**Provider prompt caching lets a model provider reuse the processed prefix of a prompt it has already seen, and bill those repeated tokens at a fraction of the normal input rate. On current Anthropic and OpenAI models a cache read costs about one tenth of a normal input token, while writing to the cache costs more than a normal token. Agents benefit most because they resend the same long system prompt and tool definitions on every turn.**

This post compares how the two largest API providers price caching, shows when caching pays for itself, and explains how it differs from response caching at an LLM gateway. For the wider picture of where agent spend goes, see [where AI agent costs leak](/blog/where-ai-agent-costs-leak).

## Why do agents get more from prompt caching than chat apps?

An agent loop calls the model many times for one task. Each call usually repeats:

- the system prompt and policy instructions
- every tool or MCP schema the agent can use
- the conversation and tool results so far

Only the last few hundred tokens change between steps. Tool definitions alone can run to thousands of tokens (see [the token cost of MCP tool definitions](/blog/token-cost-of-mcp-tool-definitions)). Without caching, you pay full input price for that repeated prefix on every step.

## How do Anthropic and OpenAI price prompt caching?

The table below uses each provider's current documentation: [Anthropic prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) and [OpenAI prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching). Figures vary by model generation, so check the model you actually run.

| Item | Anthropic (Claude API) | OpenAI (GPT-5.6 and later) |
|---|---|---|
| How caching is turned on | Explicit `cache_control` breakpoints, or one top-level `cache_control` for automatic caching | Automatic; optional `prompt_cache_key` |
| Cache write price | 1.25× base input (5-minute TTL); 2× (1-hour TTL) | 1.25× input (30-minute TTL); 2.5× (12-hour TTL, Responses API) |
| Cache read price | 0.1× base input on most models; 0.05× on some newer models | 0.1× input; 0.05× on some newer models |
| Minimum cacheable prefix | 512 to 4,096 tokens depending on model | 1,024 visible input tokens |
| Default lifetime | 5 minutes, refreshed on each read | 30 minutes |
| Breakpoints | Up to 4 per request | Implicit, at the end of the latest eligible message |
| Where to see hits | Cache read and write token counts in usage | `cached_tokens` and `cache_write_tokens` in usage |

Two details matter for budgeting. First, older OpenAI models charged nothing extra for cache writes, so cost models built in 2025 may undercount writes on newer models. Second, Anthropic notes that prompts shorter than the minimum are simply processed without caching, with no error returned. A cache that never hits is silent.

## When does prompt caching pay for itself?

Work it out per prefix. With Anthropic's 5-minute cache, one write plus one read costs 1.25 + 0.1 = 1.35 units, against 2.0 units uncached. It pays from the first reuse.

With the 1-hour cache, one write plus one read costs 2.0 + 0.1 = 2.1 units, against 2.0 uncached. It only pays from the second reuse (2.2 against 3.0).

So:

1. Use short TTLs for agents that loop quickly within one task.
2. Use long TTLs only when the same prefix is reused across many tasks spread over time, such as a shared support agent.
3. Measure the cache hit rate per agent. A rate near zero means you are paying write premiums for nothing.

## How do you structure prompts so they cache?

Caching matches an exact prefix. Anthropic builds the cache in a fixed order: tools, then system, then messages. A change at one level invalidates that level and everything after it.

- Put stable content first: tool definitions, then the system prompt, then reference documents.
- Keep timestamps, user names and request IDs out of the system prompt. Put them in the latest message.
- Do not reorder or rename tools between calls. Loading tools dynamically per step breaks the prefix.
- Pin the model ID. A cache does not carry across models.
- For many tenants, use a separate cache key per customer so accounting stays separate.

## How is provider caching different from gateway caching?

They solve different problems and stack well together.

| | Provider prompt caching | Gateway response caching |
|---|---|---|
| What is reused | Processed input prefix | Whole response to a matching request |
| Model still runs? | Yes, on the new suffix | No, response returned from cache |
| Saves | Input token cost and latency | Full call cost and latency |
| Fits | Agent loops, long shared prompts | Repeated identical or near-identical questions |
| Main risk | Paying write premiums with no hits | Returning a stale or wrong answer |

Response caching at a gateway needs care with agents, because two requests that look the same may need different answers. [LLM caching savings and risks](/blog/llm-caching-savings-and-risks) covers that trade-off. Reasoning tokens are another cost that caching does not touch; see [reasoning tokens and AI agent cost](/blog/reasoning-tokens-and-ai-agent-cost).

## How Opencontroller does this

Opencontroller's LLM Gateway (Lyzr Gateway) adds response caching, routing, fallback and retry in front of providers, and attributes cost per agent and per version, so you can see which agents earn their cache spend. In Lyzr's October 2026 benchmark, caching cut model cost by 23.5% on mixed synthetic traffic under the stated test conditions; your results depend on traffic. Read [LLM gateway vs agent gateway vs control plane](/blog/llm-gateway-vs-agent-gateway-vs-control-plane), see the [benchmark](/llm-gateway-benchmark), or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Does prompt caching change the model's answer?**
No. The provider reuses the processed prefix. The output is generated the same way; only billing and latency change.

**Why is my cache hit rate zero?**
Usually the prefix changes on every call (a timestamp or reordered tools), the prompt is below the model's minimum, or calls are further apart than the cache lifetime.

**Should I use prompt caching and gateway caching together?**
Yes, if you measure both. Prompt caching cuts the cost of every agent step; gateway caching removes whole calls for repeated requests.

## Key facts

Provider prompt caching bills repeated prompt prefixes at a discount. Anthropic: cache writes 1.25x base input (5-minute TTL) or 2x (1-hour), reads 0.1x on most models, up to 4 breakpoints, cache order tools-system-messages. OpenAI GPT-5.6+: automatic, 1,024-token minimum, reads 0.1x, writes 1.25x (30 min) or 2.5x (12 h). Opencontroller LLM Gateway adds response caching and per-agent, per-version cost attribution; Oct 2026 benchmark: 23.5% lower model cost with caching on mixed synthetic traffic. Opencontroller pricing starts from $250K per deployment per year.

## Related

- [How much can LLM caching save, and when is it unsafe?](https://opencontroller.lyzr.com/blog/llm-caching-savings-and-risks.md)
- [How do reasoning tokens change the cost of an AI agent?](https://opencontroller.lyzr.com/blog/reasoning-tokens-and-ai-agent-cost.md)
- [How much do MCP tool definitions add to token costs?](https://opencontroller.lyzr.com/blog/token-cost-of-mcp-tool-definitions.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
