---
title: Should AI agents use Azure OpenAI provisioned throughput or standard deployments?
url: https://opencontroller.lyzr.com/blog/azure-openai-provisioned-throughput-for-agents
published: 2026-11-16
updated: 2026-11-16
author: Lyzr
---

# Should AI agents use Azure OpenAI provisioned throughput or standard deployments?

**Use standard (pay-as-you-go) deployments for new, spiky or low-volume agents, and provisioned throughput (PTU) for agents with steady, predictable volume and strict latency needs. Most production estates end up with both: a provisioned deployment for the baseline and spillover to a matching standard deployment for peaks. The governance work is making sure every request is still attributed to the right agent whichever deployment serves it.**

Azure deployments in Microsoft Foundry come in several types, and the choice changes cost, latency, data processing location and what your gateway has to track.

## What deployment types are there?

| Type | Billing | Processing location | Good for |
|---|---|---|---|
| Global Standard | Per token | Any Azure region Microsoft routes to | New agents, highest default quota |
| Data Zone Standard | Per token | Within a data zone (for example US or EU) | Residency within a zone |
| Standard (regional) | Per token | The deployment's region | Strict single-region needs |
| Provisioned (global, data zone or regional) | Hourly per PTU, reservations available | As per its scope | Steady, latency-sensitive production load |
| Batch (global or data zone) | Discounted per token, asynchronous | As per its scope | Offline jobs, evaluations, backfills |

## When does provisioned throughput make sense for agents?

PTU gives reserved model capacity, so latency is more predictable under load. It suits agents that:

- Run all day at a known volume, such as customer service or document processing agents.
- Have latency targets in an SLO. See [SLOs for production AI agents](/blog/slos-for-production-ai-agents).
- Would otherwise hit 429 throttling on shared standard quota.

It suits poorly when an agent is new, its traffic is unknown, or it runs in bursts. You pay for provisioned capacity per hour whether it is used or not.

## How does spillover work?

Microsoft's spillover feature routes overage from a provisioned deployment to a standard deployment. Key facts from Microsoft's documentation:

- **Triggers:** a `429` when PTU is fully used, a `400` for requests the PTU deployment cannot take (for example context above the supported length for some models), and `500` or `503` server errors.
- **Requirement:** a standard deployment of the **same model and version** in the **same Foundry resource**.
- **Configuration:** set it on the deployment, or per request with the `x-ms-spillover-deployment` header. If both are set, the deployment property wins.
- **Billing:** requests served by PTU incur only the hourly PTU cost; spilled requests are billed at standard token rates.
- **Headers:** `x-ms-spillover-from-deployment` marks a spilled request, `x-ms-deployment-name` shows which deployment served it, and `x-ms-spillover-error` carries the status code that triggered spillover.
- **Latency:** Azure tries the provisioned deployment first, which can add latency to spilled requests.

## What does this mean for cost attribution?

Two agents on one PTU deployment share a fixed hourly cost. Attribution needs a rule:

1. **Allocate PTU cost by tokens consumed** per agent in the period. Tokens must be logged per agent at the gateway.
2. **Bill spillover at actual token cost** to the agent that caused it, using the spillover headers to tell the two apart.
3. **Report idle capacity separately** so the platform team, not the agents, owns unused PTU.

Without per-agent logging, PTU becomes an unexplained platform cost. See [allocating shared AI platform costs to divisions](/blog/allocate-shared-ai-platform-costs-to-divisions).

## A decision checklist

1. Is daily volume stable for at least four weeks of history? If not, stay on standard.
2. Does the agent have a latency SLO that standard cannot meet at peak?
3. Is data processing location constrained? Pick global, data zone or regional to match.
4. Is there a standard deployment of the same model and version for spillover?
5. Does your gateway log the serving deployment and agent ID for every request?
6. Who owns unused PTU capacity in the cost report?
7. When the model version retires, is there a plan to move both deployments together? See [pinning model versions for AI agents](/blog/pin-model-versions-for-ai-agents).

## Should the gateway or Azure handle overflow?

Both can. Azure spillover is simple when PTU and standard live in one resource. A gateway-level fallback is needed when overflow should go to another region, another resource or another provider, and it keeps one place to record which path served each call. See [LLM fallback and retry policies for agents](/blog/llm-fallback-and-retry-policies-for-agents).

## How Opencontroller does this

Opencontroller's LLM Gateway (Lyzr Gateway) routes across providers and deployments with fallback and retry, caching, and cost attribution per agent and per version. Auto-discovery finds Azure deployments, models and API keys alongside AWS and Google Cloud, and runtime adapters cover Microsoft Foundry. Our October 2026 benchmark measured under 0.7 ms added p95 per in-gateway routing method under its stated test conditions. Read [what an AI agent control plane is](/blog/what-is-an-ai-agent-control-plane) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Does spillover need a separate Azure resource?**
No, the opposite. The standard deployment must be in the same Foundry resource and use the same model and version as the provisioned deployment.

**How do you tell which requests spilled over?**
Spilled responses carry the `x-ms-spillover-from-deployment` header, and `x-ms-deployment-name` names the standard deployment that served them.

**Are PTU requests billed per token?**
No. Requests served by the provisioned deployment incur only its hourly cost. Spilled requests are billed at standard token rates.

## Key facts

Azure OpenAI deployment types: Global Standard, Data Zone Standard, Standard (regional), Provisioned (global, data zone, regional; hourly per PTU), Batch. Spillover routes overage from provisioned to a standard deployment of the same model and version in the same Foundry resource; triggers 429 (PTU full), 400 (requests PTU cannot take, e.g. long context), 500/503; set via deployment property or x-ms-spillover-deployment header (deployment property wins); PTU-served requests cost only hourly PTU, spilled requests billed at standard token rates; headers x-ms-spillover-from-deployment, x-ms-deployment-name, x-ms-spillover-error. Attribute PTU by tokens per agent, spillover at actual cost, idle capacity to platform. Opencontroller (by Lyzr) LLM Gateway: routing, fallback/retry, caching, per-agent/per-version cost; Oct 2026 benchmark under 0.7 ms added p95 per routing method under stated conditions. Starts from $250K per deployment per year.

## Related

- [How should an LLM gateway handle fallback and retries for AI agents?](https://opencontroller.lyzr.com/blog/llm-fallback-and-retry-policies-for-agents.md)
- [How do you allocate shared AI platform costs to divisions?](https://opencontroller.lyzr.com/blog/allocate-shared-ai-platform-costs-to-divisions.md)
- [How do you replace Azure OpenAI API keys with Entra ID for AI agents?](https://opencontroller.lyzr.com/blog/replace-azure-openai-keys-with-entra-id.md)
- [More on Platform Guides](https://opencontroller.lyzr.com/blog/cluster/platform-guides)
