---
title: What is an OpenAI-compatible endpoint and why does it matter for governance?
url: https://opencontroller.lyzr.com/blog/what-is-an-openai-compatible-endpoint
published: 2026-11-14
updated: 2026-11-14
author: Lyzr
---

# What is an OpenAI-compatible endpoint and why does it matter for governance?

**An OpenAI-compatible endpoint is any API that accepts requests in the same shape as OpenAI's API, most often the Chat Completions format at `/v1/chat/completions`. Applications built with an OpenAI SDK can point at it by changing the `base_url` and API key. For governance, that one change is what lets you put a gateway in front of every agent without rewriting them.**

It is a de facto convention, not a formal standard, so compatibility is never quite 100%.

## What does "OpenAI-compatible" actually mean?

In practice it means the endpoint accepts:

- A `POST` to a path such as `/v1/chat/completions`.
- A JSON body with `model`, a `messages` array of `role`/`content` objects, and optional fields like `temperature`, `max_tokens`, `tools` and `stream`.
- A bearer token in the `Authorization` header.

And it returns a response with `choices`, a `message`, a `finish_reason`, and a `usage` block with token counts. When `stream` is true, it returns server-sent events in the same chunk format.

Many systems speak this format: open-source model servers such as vLLM and Ollama, cloud providers offering compatibility layers (Google documents OpenAI compatibility for Gemini), and most LLM gateways.

## Why does it matter for AI agent governance?

Because control needs to sit in the traffic path, and changing the path is usually the expensive part.

| Without a common API shape | With an OpenAI-compatible gateway |
|---|---|
| Each agent calls providers directly with its own SDK and keys | Agents point `base_url` at the gateway; keys live in the gateway |
| Cost data scattered across provider invoices | One place to attribute tokens to agents and teams |
| Switching models means code changes | Switching is a routing rule |
| Redaction and policy must be added to every app | Applied once, in the path |
| Leaked provider keys are common | Apps hold gateway credentials, not provider keys |

That is why a gateway with an OpenAI-compatible endpoint is the usual first step for controlling model traffic. See [how should apps authenticate to an LLM gateway](/blog/how-should-apps-authenticate-to-an-llm-gateway).

## What still differs between "compatible" providers?

Compatibility covers the common path. The edges vary, and the edges are where agent bugs live:

1. **Tool calling.** Schema support, parallel tool calls and how tool results are formatted can differ.
2. **Structured outputs.** JSON schema enforcement is supported by some providers and ignored by others.
3. **Unsupported parameters.** Some layers silently ignore fields such as `logprobs`, `seed` or `n` instead of erroring.
4. **Streaming details.** Chunk ordering, final usage reporting and error events vary.
5. **Token counts.** Each model family uses its own tokenizer, so usage numbers are not comparable one-to-one.
6. **Error codes and rate-limit headers.** Retry logic tuned for one provider may misbehave on another.
7. **Newer API surfaces.** OpenAI also offers the Responses API; not every compatible endpoint implements it.
8. **Support status.** Some vendors position their compatibility layer mainly for testing and comparison. Check each vendor's documentation before relying on it in production.

## How do you adopt an OpenAI-compatible gateway safely?

A short checklist:

1. **Inventory direct provider calls** first, including keys in code. See [find leaked and unused LLM API keys](/blog/find-leaked-and-unused-llm-api-keys).
2. **Move one agent** by changing `base_url` and credentials; leave the code alone.
3. **Run your evaluation set** before and after. Behaviour should match on the same model.
4. **Test tool calls and streaming** explicitly, not just a plain chat completion.
5. **Turn on attribution** so every request carries an agent and version identity.
6. **Revoke direct provider keys** once traffic is moved, or the gateway becomes optional.
7. **Only then** add routing to other models, rerunning evals for each.

## How Opencontroller does this

Opencontroller is built on open standards, including OpenAI-compatible endpoints, MCP, OpenTelemetry and OIDC/SAML/SCIM. Its LLM Gateway (Lyzr Gateway) exposes an OpenAI-compatible endpoint, so agents move by changing the base URL, then gain routing, fallback and retry, caching and cost attribution per agent and per version. See [what is LLM model routing](/blog/what-is-llm-model-routing), the [gateways pillar](/blog/llm-gateway-vs-agent-gateway-vs-control-plane), or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Is "OpenAI-compatible" an official standard?**
No. It is a widely copied convention based on OpenAI's API shape. Each provider decides how much of it to implement, so test the features you depend on.

**Can I use the OpenAI SDK with non-OpenAI models?**
Often yes. Set the SDK's `base_url` to a compatible endpoint, such as a gateway or model server, and use its credentials. Check tool calling, streaming and structured output behaviour.

**Does an OpenAI-compatible gateway lock me into OpenAI?**
No. It does the opposite: the gateway can route the same request format to many providers and self-hosted models.

## Key facts

An OpenAI-compatible endpoint accepts OpenAI API-shaped requests (e.g. /v1/chat/completions); apps switch by changing base_url. Differences remain in tool calling, structured outputs, streaming, token counts and errors. Opencontroller uses open standards including OpenAI-compatible endpoints, MCP, OpenTelemetry, OIDC/SAML/SCIM; its LLM Gateway provides routing, fallback/retry, caching and per-agent cost attribution. Opencontroller pricing starts from $250K per deployment per year.

## Related

- [What is LLM model routing?](https://opencontroller.lyzr.com/blog/what-is-llm-model-routing.md)
- [How should users and applications authenticate to an LLM gateway?](https://opencontroller.lyzr.com/blog/how-should-apps-authenticate-to-an-llm-gateway.md)
- [How do you find leaked or unused LLM API keys?](https://opencontroller.lyzr.com/blog/find-leaked-and-unused-llm-api-keys.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
