---
title: Why do Bedrock requests fail through an OpenAI-compatible LLM gateway?
url: https://opencontroller.lyzr.com/blog/bedrock-converse-vs-openai-format-in-llm-gateways
published: 2026-11-15
updated: 2026-11-15
author: Lyzr
---

# Why do Bedrock requests fail through an OpenAI-compatible LLM gateway?

**Most failures come from three mismatches: the gateway signs requests the wrong way for Bedrock, it sends a model name Bedrock does not recognise, or it sends an OpenAI-shaped body to an API that expects Converse or a model-native format. Bedrock now accepts OpenAI Chat Completions and Responses requests on `/openai/v1` paths, but only for models that support those endpoints, so check each model before you route it.**

This question came up in a pilot where routing to Bedrock models stalled on authorization and request-format errors while other providers worked. The fixes are mechanical once you know which layer is failing.

## Which Bedrock APIs can a gateway call?

Bedrock exposes several APIs on two endpoints. Your gateway must pick one per model.

| API | Body shape | `bedrock-runtime` | `bedrock-mantle` |
|---|---|---|---|
| InvokeModel | Model-native JSON (differs per provider) | Yes | No |
| Converse / ConverseStream | One AWS-defined shape for all models | Yes | No |
| Chat Completions | OpenAI-compatible | Yes (`/openai/v1`) | Yes |
| Responses API | OpenAI-compatible | Yes (`/openai/v1`) | Yes |
| Messages API | Anthropic-native | Yes | Yes |

Source: the AWS Bedrock endpoints documentation. On `bedrock-runtime`, the Responses API is synchronous only, server-side tools such as web search are unavailable, and only the default project is supported.

## Why does authorization fail?

Bedrock accepts two credential types: AWS SigV4 signing, and Bedrock API keys. A gateway built for OpenAI sends a bearer token. That works only with a Bedrock API key on the OpenAI-compatible paths.

Even with valid credentials, the IAM principal needs permission. On `bedrock-runtime` that is `bedrock:InvokeModel`, plus `bedrock:InvokeModelWithResponseStream` for streaming. On `bedrock-mantle` it is `bedrock-mantle:CreateInference`. A missing streaming permission is a common cause of requests that work in testing and fail once the client turns streaming on.

Short-lived role credentials also expire. If the gateway caches STS credentials without refreshing them, calls start failing hours after deployment. Refresh before expiry, or use workload identity so the platform handles it.

## Why does the model ID fail?

OpenAI clients send names like `gpt-4o`. Bedrock expects its own model IDs, and many newer models must be called through an inference profile ID (for example a cross-Region profile) rather than the base model ID. Calling the base ID for a profile-only model returns a validation error that looks like a format problem.

Keep an explicit alias table in the gateway: the name agents send, the Bedrock ID or inference profile ARN, the Region, and the API to use. Our guide to [Bedrock model access and inference profiles](/blog/bedrock-model-access-and-inference-profiles-for-agents) covers the profile rules.

## Why does the request body fail?

Three patterns account for most body errors:

1. **OpenAI body sent to Converse or InvokeModel.** Converse uses `messages` with content blocks and an `inferenceConfig` object, not `max_tokens` and `temperature` at the top level. Translate, or use the OpenAI-compatible path.
2. **OpenAI-compatible path used for a model that does not support it.** Check the model's row in the Bedrock endpoint availability table first.
3. **Tool schemas.** Converse expects tools under `toolConfig` with `toolSpec` and `inputSchema.json`. OpenAI tools use `tools` with `function.parameters`. A gateway that passes tools through untranslated breaks agent tool calls even when plain chat works.

## What is a quick troubleshooting order?

1. Call the model directly with the AWS CLI using the same role. If this fails, the problem is IAM or model access, not the gateway.
2. Confirm the model ID or inference profile in the alias table matches the Region.
3. Confirm which API the gateway uses for that model, and that the model supports it.
4. Send a request without tools, then with tools, then with streaming. The step that fails tells you which translation is broken.
5. Check credential refresh by running a test after the STS session would have expired.

Document the working path for each model before you enforce routing. Our post on [routing Bedrock models through an LLM gateway](/blog/route-bedrock-models-through-an-llm-gateway) covers the rollout side.

## How Opencontroller does this

The Lyzr LLM Gateway in Opencontroller exposes OpenAI-compatible endpoints to agents and handles routing, fallback and retry, caching and cost attribution per agent and per version. Opencontroller also has runtime adapters for Bedrock AgentCore, Vertex AI and Microsoft Foundry. In our October 2026 benchmark, under its stated test conditions, in-gateway routing added under 0.7 ms at p95 per routing method. Read how this fits an [AI agent control plane](/blog/what-is-an-ai-agent-control-plane), see the [LLM gateway benchmark](/llm-gateway-benchmark), or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Does Bedrock support the OpenAI SDK?**
Yes, for supported models. Point the OpenAI SDK at the `bedrock-runtime` `/openai/v1` base URL and authenticate with a Bedrock API key.

**Why does streaming fail when non-streaming works?**
The IAM principal probably lacks `bedrock:InvokeModelWithResponseStream`.

**Should a gateway use Converse or the OpenAI-compatible path?**
Use the OpenAI-compatible path where the model supports it, and Converse where it does not. Record the choice per model.

## Key facts

Amazon Bedrock endpoints: bedrock-runtime supports InvokeModel, Converse/ConverseStream, OpenAI-compatible Chat Completions and Responses (on /openai/v1 paths) and Anthropic Messages; bedrock-mantle supports Chat Completions, Responses and Messages. Auth: SigV4 or Bedrock API keys (API keys work with the OpenAI SDK). IAM: bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream on bedrock-runtime; bedrock-mantle:CreateInference on bedrock-mantle. Responses API on bedrock-runtime is synchronous only. Opencontroller LLM Gateway (Lyzr Gateway) exposes OpenAI-compatible endpoints with routing, fallback, caching and per-agent cost attribution; benchmark Oct 2026: under 0.7 ms added p95 per routing method. Pricing starts from $250K per deployment per year.

## Related

- [Why do Amazon Bedrock calls fail through an LLM gateway, and how do you fix them?](https://opencontroller.lyzr.com/blog/route-bedrock-models-through-an-llm-gateway.md)
- [Why can an agent not invoke a Bedrock model you can see in the console?](https://opencontroller.lyzr.com/blog/bedrock-model-access-and-inference-profiles-for-agents.md)
- [What is an OpenAI-compatible endpoint and why does it matter for governance?](https://opencontroller.lyzr.com/blog/what-is-an-openai-compatible-endpoint.md)
- [More on Platform Guides](https://opencontroller.lyzr.com/blog/cluster/platform-guides)
