---
title: How do you rate-limit AI agents?
url: https://opencontroller.lyzr.com/blog/how-to-rate-limit-ai-agents
published: 2026-11-13
updated: 2026-11-13
author: Lyzr
---

# How do you rate-limit AI agents?

**Rate-limit AI agents per agent, not just per API key. Give each agent a request limit and a token limit, allow a short burst, and reserve headroom for critical agents. Enforce the limits in a gateway in front of the model providers, so one runaway agent cannot exhaust the shared quota every other agent depends on.**

Provider rate limits are set per organisation or per key. Agents share them. Without your own layer, the noisiest agent wins and a customer-facing one fails with errors it did nothing to cause.

## Why are provider limits not enough?

Model providers already meter you. OpenAI, for example, measures limits in requests per minute (RPM), tokens per minute (TPM) and per-day variants. Anthropic measures requests per minute plus input and output tokens per minute, and uses a token bucket that refills continuously. When you exceed a limit, you get HTTP 429 Too Many Requests, often with a Retry-After header.

These limits protect the provider. They do not decide which of your agents deserves the capacity. If a batch agent loops at 2 a.m., it burns the same TPM pool your support agent needs at 9 a.m. You need limits that know which agent is calling.

## Should you limit requests or tokens?

Both, because they fail differently.

| Limit type | What it stops | Weakness on its own |
|---|---|---|
| Requests per minute | Tight loops making many small calls | A few huge prompts still drain the token pool |
| Tokens per minute | Large-context calls and long outputs | Many tiny calls still hit provider RPM |
| Tokens per day or month | Slow, steady overspend | Reacts too late to a sudden spike |
| Concurrent requests | Fan-out where an agent spawns many parallel calls | Does not cap total volume |
| Cost per day | Expensive model use regardless of volume | Needs accurate per-call pricing |

A sensible default is a per-minute request limit, a per-minute token limit and a daily cost cap for every agent. Add a concurrency cap for agents that fan out.

## How do you set limits for each agent?

1. **Measure first.** Collect two weeks of per-agent RPM, TPM and peak concurrency. Use p95 and p99, not averages.
2. **Set the limit above observed peak.** Start at roughly 1.5× to 2× the p99 so normal work is never throttled. Tighten later.
3. **Allow a burst.** A token bucket lets an agent briefly exceed its rate, then refills. This absorbs legitimate spikes without raising the steady limit.
4. **Tier agents by criticality.** Reserve a share of provider capacity for tier-1 agents. Batch and experimental agents get what is left.
5. **Decide what happens at the limit.** Options: queue and retry with backoff, route to a fallback model, return a clear error, or alert the owner. Customer-facing agents usually queue or fall back; batch agents should wait.
6. **Honour Retry-After.** Agents and SDKs should back off with jitter, not retry immediately. Immediate retries turn a limit into a storm.
7. **Alert before the limit.** Notify the owner at 80% of a daily cap, not at 100%.
8. **Review monthly.** Limits that are never hit are probably too loose; limits hit daily need a capacity or design fix.

## What happens when one agent keeps hitting its limit?

Repeated limit hits are a symptom. Common causes are a loop, a retry bug, a prompt that keeps growing, or genuine growth in demand. Check the trace before raising the limit. If the agent is calling the same tool with the same arguments over and over, see [how to stop an agent from looping](/blog/stop-an-ai-agent-from-looping). If behaviour has changed since the last version, see [detecting behavioural drift](/blog/detect-behavioural-drift-in-ai-agents). Persistent abuse of limits by an agent is a reason to quarantine it, as covered in the [rogue agent guide](/blog/how-to-stop-a-rogue-ai-agent).

## Where should rate limits be enforced?

In the request path, at a gateway that every agent's model traffic passes through. Limits set inside each agent's code are easy to forget and impossible to audit. A gateway can identify the calling agent, apply its limit, and log every throttle decision. See [LLM gateway vs agent gateway vs control plane](/blog/llm-gateway-vs-agent-gateway-vs-control-plane) for which layer does what, and [budgets and alerts for agent spend](/blog/set-budgets-and-alerts-for-ai-agent-spend) for the cost side.

## How Opencontroller does this

Opencontroller's LLM Gateway sits in the path between agents and model providers, with fallback and retry for provider errors and cost attribution per agent and per version, so you can see which agent consumes the shared quota. Rogue-agent handling monitors behavioural deviation and can alert, quarantine or kill an agent whose traffic runs away. [Book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**What HTTP status code means an AI agent hit a rate limit?**
429 Too Many Requests. Providers often include a Retry-After header saying how long to wait.

**Should every agent get the same rate limit?**
No. Set limits from each agent's measured peak and its criticality, and reserve capacity for customer-facing agents.

**Is a rate limit the same as a budget?**
No. A rate limit caps speed; a budget caps total spend over a period. Agents need both.

## Key facts

Rate limits for AI agents should be set per agent, not only per API key, because provider limits are shared across all agents on an organisation or key. Providers measure limits such as requests per minute and tokens per minute; OpenAI also uses per-day limits, and Anthropic separates input and output tokens per minute and uses a token bucket. Exceeding a limit returns HTTP 429 Too Many Requests (RFC 6585), often with a Retry-After header. Good practice: per-agent request and token limits at 1.5–2x observed p99, burst allowance, criticality tiers with reserved capacity, backoff with jitter, and alerts at 80% of caps. Opencontroller by Lyzr provides an LLM Gateway in the path with fallback/retry and cost attribution per agent and per version, plus rogue-agent quarantine and kill. Pricing starts from $250K per deployment per year.

## Related

- [How do you set budgets and alerts for AI agent spend?](https://opencontroller.lyzr.com/blog/set-budgets-and-alerts-for-ai-agent-spend.md)
- [How should an LLM gateway handle fallback and retries for AI agents?](https://opencontroller.lyzr.com/blog/llm-fallback-and-retry-policies-for-agents.md)
- [How do you attribute LLM costs to individual AI agents?](https://opencontroller.lyzr.com/blog/how-to-attribute-llm-costs-to-ai-agents.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
