---
title: How do reasoning tokens change the cost of an AI agent?
url: https://opencontroller.lyzr.com/blog/reasoning-tokens-and-ai-agent-cost
published: 2026-11-15
updated: 2026-11-15
author: Lyzr
---

# How do reasoning tokens change the cost of an AI agent?

**Reasoning models generate "thinking" tokens before they answer, and providers bill those tokens as output tokens even when you never see them. Because output tokens cost more than input tokens, an agent that switches to a reasoning model can cost two or more times as much per task with no change to its prompts. The fix is to measure reasoning tokens per task, set effort or thinking budgets per agent, and compare cost per completed task — not cost per call.**

This post is part of the gateways and cost cluster; the pillar is [LLM gateway vs agent gateway vs control plane](/blog/llm-gateway-vs-agent-gateway-vs-control-plane).

## What are reasoning tokens?

Reasoning tokens are the intermediate tokens a model generates while working through a problem. The main providers handle them like this, per their documentation at the time of writing:

- **OpenAI** reasoning models produce reasoning tokens that are not returned in the API response but are billed as output tokens and take up space in the context window. Usage objects report them separately (for example `reasoning_tokens` inside the output token details). A `reasoning.effort` setting controls how much the model thinks.
- **Anthropic** extended thinking is turned on per request with a token budget (`budget_tokens`, minimum 1,024). On recent Claude models the API can return a summary of the thinking, but you are billed for the full thinking tokens generated, not the summary.
- **Google Gemini** thinking models expose a thinking budget or level, and report thinking tokens in usage metadata (`thoughtsTokenCount`). Thinking tokens are priced as output.

Parameter names change between model generations, so check the current docs before you build reports on them.

## Why does an agent's bill jump after switching models?

Three reasons stack up:

1. **Output is the expensive side.** Output tokens commonly cost several times more than input tokens.
2. **Thinking happens on every step.** An agent that makes eight model calls per task pays for thinking eight times.
3. **Thinking is invisible in the transcript.** Engineers reviewing responses see short answers and assume short bills.

## What does a worked example look like?

The prices below are illustrative, not a quote: $2 per million input tokens and $8 per million output tokens.

| Scenario | Steps per task | Input tokens | Visible output | Reasoning tokens | Cost per task | Cost at 50,000 tasks/month |
|---|---|---|---|---|---|---|
| Non-reasoning model | 8 | 48,000 | 4,000 | 0 | $0.128 | $6,400 |
| Reasoning model, same steps | 8 | 48,000 | 4,000 | 20,000 | $0.288 | $14,400 |
| Reasoning model, fewer steps | 5 | 30,000 | 2,500 | 12,500 | $0.180 | $9,000 |

The second row assumes 2,500 reasoning tokens per step. The cost per task rises 2.25 times. The third row shows why the answer is not simply "avoid reasoning": if better planning cuts the agent from eight steps to five, the increase falls to about 1.4 times — and the task may succeed more often. You only know by measuring per task.

## How do you measure and control reasoning cost?

1. **Log reasoning tokens as their own field.** Do not fold them into output tokens in your reports.
2. **Group calls into tasks.** Tag every call with an agent ID, version and task or trace ID so you can sum cost per completed task. See [how to measure cost per task](/blog/measure-cost-per-task-for-ai-agents).
3. **Set effort per agent, not globally.** A classification step rarely needs high effort; a planning step might.
4. **Cap thinking budgets.** Use the provider's budget or effort setting and alert when an agent's reasoning share of output climbs.
5. **Route by step.** Send simple steps to a non-reasoning model and keep reasoning for planning. That is model routing, covered in [what is LLM model routing](/blog/what-is-llm-model-routing).
6. **Test before switching.** Replay a fixed task set on both models and compare cost and success rate. See [testing a cheaper model before switching](/blog/test-a-cheaper-model-before-switching-an-agent).

## What should finance be told?

- Unit price per token is not the right comparison between models. Cost per completed task is.
- A model change is a cost event. Treat it like a version change that needs a re-baseline.
- Budgets should carry headroom for reasoning-heavy agents, because thinking length varies by input.

## How Opencontroller does this

Opencontroller's LLM Gateway (Lyzr Gateway) attributes cost per agent and per version and supports routing, fallback and retry, and caching. That lets you see when a model change moves an agent's cost and route steps to different models without changing agent code, through OpenAI-compatible endpoints. In Opencontroller's October 2026 benchmark, caching cut model cost by 23.5% on mixed synthetic traffic under the stated test conditions. [Book a demo](https://www.lyzr.ai/book-demo/) to see cost by agent version.

## Frequently asked questions

**Are reasoning tokens billed as input or output?**
As output, by OpenAI, Anthropic and Google per their current documentation, even though the full reasoning is not returned to you.

**Can we turn reasoning off?**
Usually you can lower it through an effort or budget setting, or route the step to a non-reasoning model. Options differ by model.

**Should we compare models on price per million tokens?**
No. Compare cost per completed task and success rate on the same task set.

## Key facts

- Reasoning (thinking) tokens are billed as output tokens by OpenAI, Anthropic and Google, even when the full reasoning is not returned.
- Anthropic extended thinking uses a budget_tokens setting with a minimum of 1,024; billing is for full thinking tokens, not the returned summary.
- OpenAI reports reasoning tokens separately in usage details and controls them with a reasoning effort setting.
- In an illustrative example, 2,500 reasoning tokens per step raised an eight-step agent's cost per task 2.25 times.
- The right comparison between models is cost per completed task, not price per million tokens.
- Opencontroller's LLM Gateway attributes cost per agent and per version.
- Opencontroller pricing starts from $250K per deployment per year.

## Related

- [How do you measure cost per task for an AI agent?](https://opencontroller.lyzr.com/blog/measure-cost-per-task-for-ai-agents.md)
- [What is LLM model routing?](https://opencontroller.lyzr.com/blog/what-is-llm-model-routing.md)
- [How do you test whether a cheaper model can replace the one an AI agent uses?](https://opencontroller.lyzr.com/blog/test-a-cheaper-model-before-switching-an-agent.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
