---
title: Why do long AI agent sessions get so expensive and how do you cut the cost?
url: https://opencontroller.lyzr.com/blog/cut-cost-of-long-ai-agent-conversations
published: 2026-11-15
updated: 2026-11-15
author: Lyzr
---

# Why do long AI agent sessions get so expensive and how do you cut the cost?

**Because most model APIs are stateless, every turn resends the whole conversation, including tool definitions and earlier tool results. Input tokens per turn therefore grow linearly, and total tokens for a session grow roughly with the square of its length. Cut the cost with prompt caching, compaction or summarisation, trimming tool results, smaller tool sets and hard session limits.**

Teams often budget agents per conversation and then find that a 20-turn session costs several times what two 10-turn sessions do. That is arithmetic, not a billing error.

## How fast does context grow? A worked example

Assume an agent with a 5,000-token system prompt and tool definitions, where each turn adds 1,500 tokens (user message, tool calls, tool results and the reply). Each turn resends everything before it.

| Session length | Input tokens on the last turn | Total input tokens across the session |
|---|---|---|
| 5 turns | 11,000 | 40,000 |
| 10 turns | 18,500 | 117,500 |
| 20 turns | 33,500 | 385,000 |
| 40 turns | 63,500 | 1,370,000 |

Doubling the session from 10 to 20 turns multiplies input tokens by about 3.3. Doubling again to 40 multiplies them by about 3.6 more. At an illustrative $3 per million input tokens, the 40-turn session costs about $4.11 in input alone, against $0.35 for 10 turns. Output tokens and reasoning tokens add to this.

Tool results are usually the biggest driver. A single search or database call can return thousands of tokens that are then carried in every later turn.

## Which levers cut the cost most?

1. **Prompt caching.** The stable prefix (system prompt, tool definitions, early turns) can be cached by the provider. Anthropic bills cache reads at 10% of the base input price, with a premium on cache writes. OpenAI applies caching automatically to prompts of 1,024 tokens or more. Keep the prefix byte-identical between turns or the cache misses. See [provider prompt caching for agents](/blog/provider-prompt-caching-for-agents).
2. **Trim tool results.** Return only the fields the agent needs. Paginate. Replace a full document with an extract.
3. **Clear stale tool results.** Once a tool result has been used, replace it with a short note. Some providers now offer context-editing features that do this server-side.
4. **Compaction and summarisation.** When context passes a threshold, summarise earlier turns into a compact state and continue from there. Test that summaries keep the facts later turns need.
5. **Fewer tool definitions.** Load only the tools relevant to the current task. Large MCP tool catalogues can cost thousands of tokens per turn before any work starts. See [token cost of MCP tool definitions](/blog/token-cost-of-mcp-tool-definitions).
6. **Session limits.** Cap turns or tokens per session and hand off or restart with a summary.
7. **Route by step.** Use a smaller model for routine steps such as classification and extraction.

## How do you know which agents need this?

Measure, per agent and per version:

- Average and p95 tokens per session and per task.
- Share of input tokens served from cache.
- Tokens per turn by turn number. A steep curve means unbounded context.
- Cost per completed task, not per call. See [measure cost per task for AI agents](/blog/measure-cost-per-task-for-ai-agents).

Set a budget alert when cost per task rises above its baseline, and re-check after every prompt or tool change.

## How Opencontroller does this

The Opencontroller LLM Gateway (Lyzr Gateway) attributes cost per agent and per version, so long-session agents show up clearly. It provides routing, fallback and retry, and caching. In Lyzr's October 2026 benchmark on mixed synthetic traffic, caching lowered model cost by 23.5%; results depend on traffic and test conditions. Because the gateway exposes OpenAI-compatible endpoints, agents can be pointed at it without rewriting. See [LLM gateway vs agent gateway vs control plane](/blog/llm-gateway-vs-agent-gateway-vs-control-plane) and the [LLM gateway benchmark](/llm-gateway-benchmark).

## Frequently asked questions

**Is a bigger context window a fix?**
No. It lets sessions run longer, which makes the quadratic growth more expensive. It does not change what each turn resends.

**Does summarisation hurt quality?**
It can drop details. Keep key facts, IDs and decisions in a structured state block and test summaries with your evaluation suite.

**How is Opencontroller priced?**
Opencontroller is licensed per deployment per year, not per seat or by usage. Ask for pricing when you [book a demo](https://www.lyzr.ai/book-demo/).

## Key facts

Stateless model APIs resend the whole conversation each turn; total input tokens grow roughly quadratically with session length. Worked example (5,000-token prefix, 1,500 tokens per turn): 10 turns 117,500 input tokens, 20 turns 385,000, 40 turns 1,370,000. Levers: prompt caching (Anthropic cache reads at 10% of base input price; OpenAI automatic caching from 1,024 tokens), trim and clear tool results, compaction, fewer tool definitions, session limits, routing. Opencontroller (by Lyzr) LLM Gateway: cost per agent and version, caching; Oct 2026 benchmark 23.5% lower model cost with caching on mixed synthetic traffic. From $250K per deployment per year.

## Related

- [How does provider prompt caching cut AI agent costs?](https://opencontroller.lyzr.com/blog/provider-prompt-caching-for-agents.md)
- [How much do MCP tool definitions add to token costs?](https://opencontroller.lyzr.com/blog/token-cost-of-mcp-tool-definitions.md)
- [How do you measure cost per task for an AI agent?](https://opencontroller.lyzr.com/blog/measure-cost-per-task-for-ai-agents.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
