---
title: Do structured outputs work the same across LLM providers behind a gateway?
url: https://opencontroller.lyzr.com/blog/structured-outputs-across-llm-providers
published: 2026-11-18
updated: 2026-11-18
author: Lyzr
---

# Do structured outputs work the same across LLM providers behind a gateway?

**No. OpenAI, Anthropic and Google Gemini all offer schema-constrained JSON output, but each accepts a different subset of JSON Schema and handles unsupported keywords differently. A schema that works on one provider can be rejected, or silently loosened, on another, so a gateway fallback can break an agent that depends on strict output.** Design one portable schema, test it on every route, and validate in your own code.

## How does each provider enforce structured output?

- **OpenAI** uses `response_format` with `type: "json_schema"` and `strict: true`, or `strict: true` on function definitions. In strict mode every property must be listed in `required` and objects must set `additionalProperties: false`. Optional fields are emulated with a union such as `["string", "null"]`.
- **Anthropic** uses `output_config.format` with `type: "json_schema"` for JSON responses, and `strict: true` on tools for validated tool inputs. Anthropic documents structured outputs as stable on its API, Amazon Bedrock, Google Cloud and Microsoft Foundry. Using an unsupported keyword returns a 400 error.
- **Gemini** uses a response MIME type of `application/json` plus `response_json_schema` (`responseJsonSchema` in JavaScript and Go). Google states that only a subset of JSON Schema is supported and that unsupported properties are ignored.

That last difference matters most for a gateway. A rejected schema fails loudly. An ignored keyword fails quietly.

## Which JSON Schema features differ?

Based on each provider's current documentation:

| Feature | OpenAI strict mode | Anthropic structured outputs | Gemini |
|---|---|---|---|
| All properties required | Yes, mandatory | No | No |
| `additionalProperties` | Must be `false` | Only `false` allowed | Boolean or schema |
| `minimum` / `maximum` | Not supported | Not supported | Supported |
| `minLength` / `maxLength` | Not supported | Not supported | Not listed |
| `pattern` (regex) | Not supported | Supported, with limits (no lookaround or backreferences) | Not listed |
| String `format` | Not supported | `date-time`, `email`, `uri`, `uuid` and others | `date-time`, `date`, `time` |
| `minItems` / `maxItems` | Not supported | `minItems` of 0 or 1 only | Supported |
| Recursive schemas | Supported via `$ref` | Not supported | Shown in examples via `$ref` |
| Unsupported keyword | Request rejected | 400 error | Ignored |

OpenAI also documents caps on the total number of object properties and nesting depth, and Azure OpenAI notes strict function calling is not supported with parallel tool calls. Check current limits before relying on very large schemas.

## What goes wrong during a gateway fallback?

A typical failure path:

1. The primary route is OpenAI with a recursive schema for a nested approval tree.
2. The provider returns 429s, so the gateway falls back to an Anthropic model.
3. Anthropic rejects the recursive schema with a 400. The fallback fails, and the agent errors anyway.

A quieter variant: the fallback goes to Gemini, which ignores an unsupported keyword. The JSON parses, the agent continues, and a constraint you thought was enforced is not.

Other differences to plan for:

- **Refusals and truncation.** Any model can stop early on `max_tokens` or refuse a request, producing output that does not match the schema. Handle these as explicit outcomes.
- **First-request latency.** Anthropic compiles a grammar for each new schema and caches it for 24 hours after last use, so the first call on a fallback route can be slower.
- **Prompt caching.** On Anthropic, changing the output format invalidates the prompt cache for that conversation.
- **Semantics.** Google states plainly that structured output guarantees valid JSON matching the schema, not correct values. The same holds everywhere.

## How do you make schemas portable?

Use this checklist before any model is added to a fallback chain:

1. **Write to the intersection.** Objects with `additionalProperties: false`, every property required, nullable unions for optional fields, `enum` for closed sets, no recursion.
2. **Move value constraints into code.** Ranges, lengths and patterns go in your validator (Pydantic, Zod, JSON Schema validator), not in the provider schema.
3. **Validate every response,** whichever provider served it, and treat a failure as a retryable error.
4. **Test each route.** Run your schema suite against every model in the fallback chain in CI, not just the primary.
5. **Restrict fallbacks per agent.** Agents with strict output contracts should only fall back to models that passed the suite.
6. **Log the route.** Record which provider and model produced each structured response, so a parsing incident can be traced.

The [LLM fallback and retry policies guide](/blog/llm-fallback-and-retry-policies-for-agents) covers the routing side, and [MCP output schemas](/blog/mcp-output-schemas-and-structured-tool-results) covers structured results from tools rather than models.

## Does an OpenAI-compatible endpoint solve this?

Only at the request-shape level. An [OpenAI-compatible endpoint](/blog/what-is-an-openai-compatible-endpoint) can translate `response_format` into each provider's native parameter, but it cannot make a provider enforce a keyword it does not support. Translation hides the difference; it does not remove it. For where a gateway sits relative to other controls, see [LLM gateway vs agent gateway vs control plane](/blog/llm-gateway-vs-agent-gateway-vs-control-plane).

## How Opencontroller does this

Opencontroller's LLM Gateway (Lyzr Gateway) exposes OpenAI-compatible endpoints with routing, fallback and retry, and attributes cost per agent and per version, so you can see which route served each call. Fallback chains are configured per agent, which lets you keep strict-output agents on tested models. [Book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Will a schema that works on OpenAI work on Claude?**
Often, but not always. Recursive schemas, for example, are supported by OpenAI strict mode but rejected by Anthropic structured outputs.

**Why did my Gemini output ignore a constraint?**
Gemini supports a subset of JSON Schema and ignores unsupported properties rather than rejecting the request. Enforce such constraints in your own validator.

**Is application-side validation still needed with strict mode?**
Yes. Strict modes guarantee schema shape, not correct values, and outputs can still be cut short by token limits or refusals.

## Key facts

Structured outputs differ by provider. OpenAI strict mode (json_schema, strict: true) requires all properties in required and additionalProperties: false, supports recursion, and does not support minimum/maximum, minLength/maxLength, pattern or format. Anthropic structured outputs (output_config.format, strict tools) are stable on the Claude API, Bedrock, Google Cloud and Microsoft Foundry; they reject recursive schemas and numeric/length constraints with a 400 error but support limited regex and string formats; grammars are cached 24 hours. Gemini (response_json_schema) supports minimum/maximum and minItems/maxItems and ignores unsupported properties. Gateway fallbacks between providers can therefore fail or silently loosen constraints.

## Related

- [How should an LLM gateway handle fallback and retries for AI agents?](https://opencontroller.lyzr.com/blog/llm-fallback-and-retry-policies-for-agents.md)
- [What is an OpenAI-compatible endpoint and why does it matter for governance?](https://opencontroller.lyzr.com/blog/what-is-an-openai-compatible-endpoint.md)
- [What are MCP output schemas, and why do they matter for governance?](https://opencontroller.lyzr.com/blog/mcp-output-schemas-and-structured-tool-results.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
