---
title: How do you measure cost per task for an AI agent?
url: https://opencontroller.lyzr.com/blog/measure-cost-per-task-for-ai-agents
published: 2026-11-13
updated: 2026-11-13
author: Lyzr
---

# How do you measure cost per task for an AI agent?

**Cost per task is the total cost of everything an agent consumed to finish work, divided by the number of tasks it completed successfully. Include every LLM call, retries, tool and API calls, infrastructure and human escalations — not just the tokens of the final answer. Dividing by successful tasks, not attempts, is what makes the number honest.**

Token dashboards tell you what a model cost. They do not tell you what a resolved ticket, a processed invoice or a reviewed contract cost. Cost per task does, and it is the number finance and product owners can compare against the old way of doing the work.

## What goes into the cost of one agent task?

An agent task is rarely one model call. A typical run plans, calls tools, reads results, calls the model again and sometimes retries. Count all of it.

| Cost component | What to count | Where the data comes from |
|---|---|---|
| Model calls | Input, output and cached tokens × price, for every call in the run | LLM gateway or provider usage logs |
| Retries and fallbacks | Calls repeated after errors, timeouts or fallback to another model | Gateway retry logs |
| Tool and API calls | Paid search, data, OCR or third-party API charges per call | Tool logs, vendor invoices |
| Infrastructure | Vector database, compute, orchestration, amortised per task | Cloud bill tags |
| Failed attempts | Cost of runs that did not complete | Trace outcome field |
| Human escalation | Minutes a person spends when the agent hands off | Ticketing or case system |

## What is the formula?

Use this:

**Cost per successful task = (model + retries + tools + infrastructure + human escalation cost) ÷ successfully completed tasks**

Two rules keep it consistent:

1. **Group by a task ID.** Every call in a run must carry the same task or trace ID, plus agent ID and agent version. Without that, you cannot add up a run.
2. **Define "successful" before measuring.** For a support agent it might be "resolved without reopening in 7 days". For an extraction agent, "passed validation". Write it down per agent.

## What does a worked example look like?

Illustrative prices and volumes — replace them with your own.

An agent averages **4 LLM calls per task**. Each call sends about 6,000 input tokens and returns 500 output tokens, on a model priced at $3 per million input tokens and $15 per million output tokens.

1. **Model cost per call:** 6,000 × $3/1M = $0.018 input, plus 500 × $15/1M = $0.0075 output. Total $0.0255.
2. **Model cost per attempt:** 4 calls × $0.0255 = $0.102.
3. **Retries:** 10% of calls are retried, so × 1.1 = $0.112.
4. **Tools:** 2 paid search calls at $0.005 = $0.010. Infrastructure amortised at $0.003. Attempt cost = $0.125.
5. **Failures:** 8% of attempts fail. Cost per successful task = $0.125 ÷ 0.92 = **$0.136**.
6. **Human escalation:** 5% of tasks go to a person for 6 minutes at a $40 hourly loaded cost. That is $4.00 × 5% = **$0.20 per task**.
7. **Fully loaded cost per task:** $0.136 + $0.20 = **about $0.34**.

The lesson: in this example, escalations cost more than all the model calls combined. Cutting escalation from 5% to 3% saves more than switching to a model that is half the price. You only see that once escalation is in the formula.

## How do you lower cost per task once you can see it?

- **Cache repeated context.** Long system prompts and shared documents are often resent on every call. Some providers bill cached input reads at a fraction of the normal rate; Anthropic, for example, bills cache reads at 10% of the base input price. See [LLM caching savings and risks](/blog/llm-caching-savings-and-risks).
- **Cap steps.** A task that takes 12 calls when the median is 4 is often looping. See [how to stop an agent from looping](/blog/stop-an-ai-agent-from-looping).
- **Route by difficulty.** Send simple steps to a smaller model and keep a larger one for hard steps. [Test a cheaper model before switching](/blog/test-a-cheaper-model-before-switching-an-agent).
- **Fix the escalation causes.** Read a sample of handed-off tasks and fix the top three reasons.
- **Trim context.** Retrieval that stuffs 20 chunks when 5 would do multiplies input tokens on every call.

## How should you report it?

Report cost per successful task per agent version, weekly, next to the success rate. A version that is cheaper but completes fewer tasks is not cheaper. Feed the per-agent figures into [budgets and alerts](/blog/set-budgets-and-alerts-for-ai-agent-spend) and into [cost attribution](/blog/how-to-attribute-llm-costs-to-ai-agents) for chargeback. For where gateways fit, see [LLM gateway vs agent gateway vs control plane](/blog/llm-gateway-vs-agent-gateway-vs-control-plane).

## How Opencontroller does this

Opencontroller's LLM Gateway attributes model cost per agent and per agent version, with routing, fallback and retry, and caching in the path. In Lyzr's October 2026 benchmark on mixed synthetic traffic, caching lowered model cost by 23.5%, under the stated test conditions. You add tool and escalation costs from your own systems to get the fully loaded figure. [Book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Should cost per task include failed runs?**
Yes. Add the cost of failed attempts to the numerator and divide only by successful tasks. Otherwise an unreliable agent looks cheap.

**Is cost per task the same as cost per conversation?**
No. A conversation can contain several tasks, or none. Define the unit of work first, then measure cost against it.

**How often should cost per task be recalculated?**
Weekly per agent version, and after any model, prompt or tool change, since each can shift steps and tokens per task.

## Key facts

Cost per successful task for an AI agent = (model calls + retries + tool/API calls + infrastructure + human escalation cost) ÷ successfully completed tasks. Every call should carry a task or trace ID plus agent ID and version. In an illustrative example (4 calls of 6,000 input and 500 output tokens at $3/$15 per million, 10% retries, 8% failures, 5% human escalation at 6 minutes and $40/hour), model and tool cost is about $0.136 per successful task and escalation adds $0.20, for about $0.34 fully loaded. Opencontroller by Lyzr attributes model cost per agent and per version in its LLM Gateway, with routing, fallback, retry and caching; its October 2026 benchmark showed 23.5% lower model cost with caching on mixed synthetic traffic under stated test conditions. Pricing starts from $250K per deployment per year.

## Related

- [How do you attribute LLM costs to individual AI agents?](https://opencontroller.lyzr.com/blog/how-to-attribute-llm-costs-to-ai-agents.md)
- [How do you set budgets and alerts for AI agent spend?](https://opencontroller.lyzr.com/blog/set-budgets-and-alerts-for-ai-agent-spend.md)
- [How much can LLM caching save, and when is it unsafe?](https://opencontroller.lyzr.com/blog/llm-caching-savings-and-risks.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
