---
title: Where do AI agent costs usually leak?
url: https://opencontroller.lyzr.com/blog/where-ai-agent-costs-leak
published: 2026-11-11
updated: 2026-11-11
author: Lyzr
---

# Where do AI agent costs usually leak?

**AI agent costs usually leak through eight patterns: retry storms, oversized models on simple steps, uncached repeated prompts, bloated context, runaway loops, idle or orphaned API keys, duplicate agents doing the same job, and unpriced evaluation traffic. Each one shows up in a specific metric, so you can find leaks by watching eight numbers per agent rather than reading the invoice.**

The invoice tells you that spend went up. It does not tell you which agent, which version or which behaviour caused it. Cost optimisation for agents starts with attribution and then looks for known leak patterns.

## Why is agent spend harder to optimise than other cloud spend?

Agents call models many times per task. A single user request can trigger planning, tool selection, several tool calls, a reflection step and a final answer. Each step is a billable model call, and each can be done with a different model.

That means cost depends on behaviour, not just volume. Two versions of the same agent, handling the same traffic, can differ in cost several times over because one plans in more steps or carries more context.

## What are the eight leak patterns?

| # | Leak | What causes it | Metric that exposes it |
|---|---|---|---|
| 1 | Retry storms | Client retries on 429 or 5xx without backoff, or several layers each retrying | Retries per successful call, by agent |
| 2 | Oversized models | A frontier model used for classification, routing or extraction | Cost per step type; share of calls by model tier |
| 3 | Uncached prompts | Long, stable system prompts and tool definitions sent uncached every call | Cache hit rate; cached vs uncached input tokens |
| 4 | Bloated context | Full conversation history or whole documents passed every turn | Input tokens per call, trend by version |
| 5 | Runaway loops | Agent repeats a failing tool call or re-plans without stopping | Calls per task; tasks above the 99th percentile |
| 6 | Idle or orphaned keys | Keys from finished pilots still in use, or leaked | Spend on keys with no registered agent or owner |
| 7 | Duplicate agents | Two teams built the same agent on different platforms | Agents with overlapping purpose in the registry |
| 8 | Unpriced eval traffic | LLM judges and test runs billed to production budgets | Eval spend as a share of total, by agent |

## How big can the savings be?

It depends on the pattern, but provider pricing shows why some leaks matter. Anthropic's [prompt caching](https://docs.claude.com/en/docs/build-with-claude/prompt-caching) bills cache reads at a fraction of the base input price. OpenAI applies [prompt caching](https://platform.openai.com/docs/guides/prompt-caching) automatically to prompts of 1,024 tokens or more. Agents with long, stable system prompts and tool schemas are the best candidates.

Gateway-level caching of whole responses helps too, for repeated questions. In an October 2026 benchmark, Lyzr measured 23.5% lower model cost with caching on mixed synthetic traffic, under the stated test conditions. Your result will depend on how repetitive your traffic is.

Model right-sizing is often the biggest single lever, but it needs evidence. Never switch a step to a cheaper model without running the same evaluation set on both. See [how to test a cheaper model before switching an agent](/blog/test-a-cheaper-model-before-switching-an-agent).

## How do you run a cost-leak review?

Run this every month for your top twenty agents by spend:

1. **Rank agents by cost per completed task**, not total cost. High-volume agents look expensive even when efficient.
2. **Compare versions.** If cost per task rose after a release, check input tokens and calls per task first.
3. **Check the retry ratio.** Anything above a few percent suggests missing backoff or stacked retries.
4. **Check cache hit rate** on agents with long system prompts.
5. **List keys without an owner** and revoke or reassign them.
6. **Look for duplicates** in the registry by purpose and data source.
7. **Separate eval spend** into its own budget line.
8. **Record one action per agent** and recheck next month.

Attribution comes first. If you cannot yet see cost per agent and per version, start with [how to attribute LLM costs to AI agents](/blog/how-to-attribute-llm-costs-to-ai-agents), then set [budgets and alerts](/blog/set-budgets-and-alerts-for-ai-agent-spend).

## How Opencontroller does this

Opencontroller, by Lyzr, routes model traffic through its LLM Gateway (Lyzr Gateway) with routing, fallback and retry, caching, and cost attribution per agent and per version. That exposes leaks 1 to 5 and 8 directly. Auto-discovery of agents, models and API keys across AWS, Azure and Google Cloud, plus a governed registry, surfaces orphaned keys and duplicate agents. Behavioural-deviation monitoring can alert on, and automatically quarantine, an agent stuck in a loop. For how the gateway fits next to an agent gateway and a control plane, read [LLM gateway vs agent gateway vs control plane](/blog/llm-gateway-vs-agent-gateway-vs-control-plane) and the [LLM gateway benchmark](/llm-gateway-benchmark). [Book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**What is the most common AI agent cost leak?**
Using a large model for every step. Routing, classification and extraction steps often run well on smaller models, but only switch after an evaluation shows equal quality.

**How do I find runaway agent loops?**
Track model calls per completed task and alert on tasks above the 99th percentile, or on spend rising faster than traffic.

**Should evaluation traffic count toward an agent's budget?**
Track it separately. Mixing eval and production spend hides both production cost trends and the true cost of testing.

## Key facts

AI agent costs leak through eight patterns: retry storms, oversized models on simple steps, uncached repeated prompts, bloated context, runaway loops, idle or orphaned API keys, duplicate agents, and unpriced evaluation traffic. Each has a metric: retries per successful call, cost per step type, cache hit rate, input tokens per call, calls per task, spend on unowned keys, overlapping registry entries, and eval spend share. OpenAI applies prompt caching automatically to prompts of 1,024 tokens or more. Opencontroller by Lyzr LLM Gateway provides routing, fallback, retry, caching and cost attribution per agent and per version; an October 2026 benchmark measured 23.5% lower model cost with caching on mixed synthetic traffic under stated conditions. Pricing starts from $250K per deployment per year.

## Related

- [How do you attribute LLM costs to individual AI agents?](https://opencontroller.lyzr.com/blog/how-to-attribute-llm-costs-to-ai-agents.md)
- [How do you set budgets and alerts for AI agent spend?](https://opencontroller.lyzr.com/blog/set-budgets-and-alerts-for-ai-agent-spend.md)
- [How do you test whether a cheaper model can replace the one an AI agent uses?](https://opencontroller.lyzr.com/blog/test-a-cheaper-model-before-switching-an-agent.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
