---
title: Does extended thinking work through an LLM gateway?
url: https://opencontroller.lyzr.com/blog/does-extended-thinking-work-through-an-llm-gateway
published: 2026-11-17
updated: 2026-11-17
author: Lyzr
---

# Does extended thinking work through an LLM gateway?

**Yes, if the gateway passes reasoning content through untouched. With Claude, every thinking block in a tool-use turn must go back to the API complete and unmodified, with its signature; modified blocks are rejected with a 400 error. Gateways that translate formats, trim history or strip unknown block types are the usual reason reasoning models break behind a gateway.**

Reasoning models change what a gateway has to preserve. A plain chat request is stateless text in, text out. An agent using extended thinking with tools is a multi-step protocol, and the gateway sits in the middle of every step.

## What exactly must the gateway preserve?

Anthropic's documentation sets out the rules for thinking with tool use:

1. **Pass thinking blocks back within a tool-use turn.** Each `thinking` block returns complete and unmodified, alongside the `tool_use` block it accompanied.
2. **Keep the signature.** The `signature` field is an encrypted copy of the full reasoning. The API uses it to verify the block came from Claude. It is opaque and must not be parsed or changed.
3. **Keep `redacted_thinking` blocks.** When parts of the reasoning are safety-redacted, the API returns `redacted_thinking` blocks with encrypted `data`. Code that filters on `type == "thinking"` silently drops them and breaks the protocol.
4. **Keep the order.** Consecutive thinking blocks in the latest assistant message must match what the model generated: no rearranging, editing or partial dropping.

Outside tool use you may omit earlier turns' thinking, but Anthropic recommends passing everything back and letting the API filter.

## Where do gateways break this?

| Gateway behaviour | What goes wrong |
|---|---|
| Translating Anthropic Messages to an OpenAI Chat Completions shape | Chat Completions has no field for signed thinking blocks, so they are lost on the return trip |
| Rewriting or redacting assistant messages | A modified thinking block is rejected with a 400 error |
| Filtering content blocks by known types | `redacted_thinking` blocks are dropped |
| Trimming conversation history to save tokens | Thinking blocks are removed mid tool loop |
| Toggling thinking per request inside one turn | The API does not error; it silently disables thinking for that request |
| Forcing a specific tool | Manual extended thinking accepts only `tool_choice` auto or none |

The format problem matters most. If agents speak an OpenAI-compatible format to the gateway and the gateway speaks Anthropic or Bedrock Converse upstream, the gateway must carry reasoning content across the translation or keep it server-side. The trade-offs of each format are in [Bedrock Converse vs OpenAI format in LLM gateways](/blog/bedrock-converse-vs-openai-format-in-llm-gateways).

## Do thinking blocks work across Claude API, Bedrock and Google Cloud?

Mostly. Anthropic states that signature values are compatible across the Claude API, Amazon Bedrock (the bedrock-mantle endpoint) and Claude models on Google Cloud, so a gateway that fails over between them can usually replay a conversation. The exception is a set of newer models whose thinking blocks work only in the account that produced them or a linked account. Check this before relying on cross-provider failover for long agent conversations. Failover design is covered in [LLM fallback and retry policies](/blog/llm-fallback-and-retry-policies-for-agents).

Model versions also change the request shape. Newer Claude models use adaptive thinking (`thinking: {type: "adaptive"}`) with an `effort` setting, and return a 400 error for the older `type: "enabled"` with `budget_tokens`. A gateway that injects a fixed thinking configuration for every model will break when a new model is routed in.

## How does thinking affect cost and caching at the gateway?

- **Billing.** Thinking tokens are billed as output tokens. When the API returns summarised thinking, or you set display to omitted, you are still charged for the full thinking tokens, so the visible text undercounts the bill. Use the usage fields, not the response text, for attribution. See [how reasoning tokens change agent cost](/blog/reasoning-tokens-and-ai-agent-cost).
- **Budget.** In manual mode, `budget_tokens` must be at least 1,024 and below `max_tokens`.
- **Caching.** Changing the thinking budget or switching thinking modes between requests invalidates prompt cache breakpoints. Keep the configuration stable for a conversation, as described in [provider prompt caching for agents](/blog/provider-prompt-caching-for-agents).

## What should you test after a gateway upgrade?

Run this regression set whenever the gateway or a model route changes:

1. A multi-step tool loop with thinking enabled, completing without a 400 error.
2. A conversation that returns a `redacted_thinking` block, if you can trigger one in test.
3. Failover from one Claude platform to another mid-conversation.
4. A new model version routed with the correct thinking configuration.
5. Usage reconciliation: gateway-recorded output tokens against provider usage fields.

The role of the gateway alongside agent gateways and control planes is set out in [LLM gateway vs agent gateway vs control plane](/blog/llm-gateway-vs-agent-gateway-vs-control-plane).

## How Opencontroller does this

The Lyzr Gateway, part of Opencontroller, exposes OpenAI-compatible endpoints and provides routing, fallback and retry, caching, and cost attribution per agent and per version. Opencontroller has runtime adapters for Bedrock AgentCore, Vertex AI and Microsoft Foundry. In the October 2026 benchmark, under stated test conditions, in-gateway routing added under 0.7 ms at p95 per routing method. See the [LLM gateway benchmark](/llm-gateway-benchmark) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**What happens if a gateway modifies a thinking block?**
Anthropic rejects modified thinking blocks with a 400 error. The gateway must return them byte-for-byte, including the signature.

**Are summarised thinking tokens cheaper?**
No. You are billed for the full thinking tokens generated, not the summary returned, so attribute cost from usage fields.

**Can I force a specific tool while extended thinking is on?**
With manual extended thinking, only tool_choice auto or none is supported. Some newer models also reject forced tool use under adaptive thinking.

## Key facts

- In Claude tool-use turns, thinking blocks must be returned complete and unmodified with their signature; modified blocks are rejected with a 400 error; redacted_thinking blocks must also be kept and order preserved.
- Toggling thinking within a turn silently disables thinking for that request; manual extended thinking supports only tool_choice auto or none.
- Anthropic says signatures are compatible across the Claude API, Amazon Bedrock (bedrock-mantle) and Claude on Google Cloud, except for some newer models whose thinking blocks are account-bound.
- Newer Claude models use adaptive thinking with an effort setting and reject type enabled with budget_tokens; manual budget_tokens must be at least 1,024 and below max_tokens.
- Thinking tokens bill as output tokens, including full thinking when summarised or omitted; changing thinking configuration invalidates prompt cache.
- Lyzr Gateway in Opencontroller offers OpenAI-compatible endpoints, routing, fallback/retry, caching and cost per agent and version; Oct 2026 benchmark under stated conditions: under 0.7 ms added p95 per routing method.
- Opencontroller pricing starts from $250K per deployment per year.

## Related

- [How do reasoning tokens change the cost of an AI agent?](https://opencontroller.lyzr.com/blog/reasoning-tokens-and-ai-agent-cost.md)
- [Why do Bedrock requests fail through an OpenAI-compatible LLM gateway?](https://opencontroller.lyzr.com/blog/bedrock-converse-vs-openai-format-in-llm-gateways.md)
- [How does provider prompt caching cut AI agent costs?](https://opencontroller.lyzr.com/blog/provider-prompt-caching-for-agents.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
