---
title: What does Databricks AI Gateway govern for agents?
url: https://opencontroller.lyzr.com/blog/databricks-ai-gateway-for-agents
published: 2026-11-12
updated: 2026-11-12
author: Lyzr
---

# What does Databricks AI Gateway govern for agents?

**Databricks AI Gateway governs traffic to Databricks model serving endpoints: usage tracking in system tables, request and response logging to inference tables, rate limits, guardrails for safety and PII, traffic splitting and fallbacks. Since August 2026 Databricks also offers Unity AI Gateway, which extends that control to external agents, coding assistants, MCP servers, skills and models. Both govern what passes through Databricks; neither sees agents and model calls that never touch it.**

Data teams often ask whether the Databricks gateway is "enough" to govern their agents. The honest answer depends on where the agents run and which features apply to which endpoint type.

## What does AI Gateway do on model serving endpoints?

Databricks documents these features for AI Gateway on serving endpoints (now labelled the legacy experience):

| Feature | What it does | Limits from Databricks docs |
|---|---|---|
| Usage tracking | Logs token counts per request to `system.serving.endpoint_usage` | Needs Unity Catalog; only account admins can query; tokens estimated if the model does not return them |
| Inference tables | Writes requests and responses to Unity Catalog Delta tables | Payloads over 1 MiB are not logged; data usually appears within an hour |
| Rate limits | Queries per minute or tokens per minute at endpoint, per-user default, and custom user, service principal or group levels | Up to 20 limits, 5 group-specific; TPM limits cannot be applied to custom model or agent endpoints |
| AI guardrails (preview) | Safety filter and PII detection that can block or mask | OpenAI-compatible Chat Completions and Completions, non-streaming; no output guardrails for streaming; native Anthropic Messages and Gemini formats not evaluated |
| Traffic splitting | Sends a percentage of traffic to each served model | 0% makes a fallback-only model |
| Fallbacks | Retries the next served entity after a 429 or 5XX | External models only; up to two fallbacks |

Two limits matter most for agents. Token-per-minute rate limits do not apply to agent endpoints. And guardrails only evaluate certain request formats, so an agent using streaming or a native provider format may pass through without content checks.

## What does Unity AI Gateway add?

Databricks announced Unity AI Gateway as generally available on 4 August 2026 and now recommends it as the control plane. Databricks describes it as one control point for agents, models, MCP servers, skills and tools, including external agents and coding assistants. Features named in the announcement include permissions through Unity Catalog, runtime guardrails and policies, cost attribution by model, provider, team and application, budgets with hard spend caps, and Smart Routing (beta) based on quality, cost, performance, availability and budget.

That is a real step up for estates centred on Databricks. Check the current docs for which features apply to which traffic before you rely on them.

## What does a Databricks gateway not cover?

1. **Agents that never call Databricks.** Agentforce, Copilot Studio, Bedrock AgentCore or a team's own service using a provider directly are invisible to it.
2. **Cost per agent version across clouds.** Attribution is by what passes through the gateway; spend on other providers needs joining up elsewhere. See [where AI agent costs hide in data platforms](/blog/where-ai-agent-costs-hide-in-data-platforms).
3. **Behaviour over time.** Rate limits and guardrails act on single requests; they do not flag an agent whose pattern of tool calls has changed.
4. **Answer quality.** Safety and PII filters do not score groundedness or task adherence.
5. **Estate-wide evidence.** Framework evidence needs every agent, not only the Databricks ones.

## How should you set it up for agents?

1. **Turn on usage tracking and inference tables** for every serving endpoint an agent calls.
2. **Set per-user or per-service-principal rate limits**, and give each agent its own service principal so limits and logs separate cleanly.
3. **Use OpenAI-compatible, non-streaming requests** where you need guardrails to apply, or add checks elsewhere.
4. **Configure fallbacks** for external models that agents depend on.
5. **Evaluate Unity AI Gateway** for external agents and MCP servers that already sit in your Databricks estate.
6. **Register Databricks agents in a cross-platform inventory**; see [governing Databricks agents](/blog/govern-databricks-agents) and the [cross-platform governance guide](/blog/how-to-govern-ai-agents-across-platforms).

## How Opencontroller does this

Opencontroller, Lyzr's open control plane for enterprise AI agents, connects to Databricks as a third-party agent platform and brings its agents into a governed registry alongside agents discovered across AWS, Azure and Google Cloud. Its LLM Gateway (Lyzr Gateway) offers routing, fallback and retry, caching and cost attribution per agent and per version for traffic you route through it, on OpenAI-compatible endpoints. In Lyzr's October 2026 benchmark, under its stated test conditions, it added under 0.7 ms p95 per in-gateway routing method. It works alongside a platform gateway; see [LLM gateway vs agent gateway vs control plane](/blog/llm-gateway-vs-agent-gateway-vs-control-plane) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Can Databricks AI Gateway rate-limit agent endpoints?**
Yes, by queries per minute. Databricks documents that tokens-per-minute limits cannot be applied to custom model or agent endpoints.

**Do Databricks AI guardrails check streaming responses?**
No. Output guardrails are not supported for streaming, and native Anthropic Messages and Gemini formats are not evaluated.

**What is the difference between AI Gateway and Unity AI Gateway?**
AI Gateway on serving endpoints is now the legacy experience. Unity AI Gateway, generally available since 4 August 2026, extends control to external agents, coding assistants, MCP servers, skills and models.

## Key facts

Databricks AI Gateway on model serving endpoints (now the legacy experience) provides usage tracking in system.serving.endpoint_usage, inference tables in Unity Catalog (payloads over 1 MiB not logged), QPM/TPM rate limits (up to 20, 5 group-specific; TPM not available for custom model or agent endpoints), AI guardrails in preview (safety and PII; OpenAI-compatible non-streaming only; Anthropic Messages and Gemini formats not evaluated), traffic splitting, and fallbacks after 429/5XX for external models (up to two). Unity AI Gateway became generally available on 4 August 2026 and covers external agents, coding assistants, MCP servers, skills and models, with cost attribution, budgets and Smart Routing (beta). Opencontroller by Lyzr connects to Databricks and adds a cross-platform registry and an LLM Gateway with cost per agent and version. Pricing starts from $250K per deployment per year.

## Related

- [How do you govern AI agents built on Databricks?](https://opencontroller.lyzr.com/blog/govern-databricks-agents.md)
- [How do you see AI costs inside data platforms like Snowflake and Databricks?](https://opencontroller.lyzr.com/blog/where-ai-agent-costs-hide-in-data-platforms.md)
- [LLM gateway vs agent gateway vs AI control plane: what's the difference?](https://opencontroller.lyzr.com/blog/llm-gateway-vs-agent-gateway-vs-control-plane.md)
- [More on Platform Guides](https://opencontroller.lyzr.com/blog/cluster/platform-guides)
