---
title: How do you detect anomalies in AI and LLM spend?
url: https://opencontroller.lyzr.com/blog/detect-ai-spend-anomalies
published: 2026-11-17
updated: 2026-11-17
author: Lyzr
---

# How do you detect anomalies in AI and LLM spend?

**Detect AI spend anomalies by baselining each agent version's normal usage — cost per hour, tokens per task, calls per task and model mix — and alerting when live traffic departs from that baseline, not only when a budget is used up. Anomaly signals catch a runaway agent within an hour; monthly budget alerts often take days. Route each alert to the agent's owner, and quarantine automatically for extreme cases.**

Budgets and anomaly detection do different jobs. Budgets cap total exposure. Anomaly detection notices that something changed.

## Why are budget alerts too slow on their own?

Here is an illustrative example. An agent completes 2,000 tasks a day at about 18,000 tokens and $0.09 per task: $180 a day, with a monthly budget of $5,400.

On day 10 (spend so far $1,800), version 5 ships with a retry loop. Tokens per task jump to 70,000 and cost to about $0.35 per task: $700 a day.

| Detection method | When it fires | Spend before alert |
|---|---|---|
| Monthly budget alert at 80% ($4,320) | About 3.6 days after the release | About $1,900 extra |
| Hourly cost above 3× baseline ($7.50 → $29 per hour) | Within the first hour | About $22 extra |
| Tokens per task above 2× baseline | Within the first few dozen tasks | A few dollars |

The budget did its job, eventually. The anomaly signal saved most of the money. See [setting budgets and alerts for agent spend](/blog/set-budgets-and-alerts-for-ai-agent-spend) for the budget side.

## Which signals should you watch?

Watch ratios as well as totals. Totals rise when business grows; ratios rise when behaviour changes.

1. **Cost per hour, per agent version.** The basic spike detector.
2. **Tokens per task (or per session).** Catches loops, oversized context and verbose prompts. See [measuring cost per task](/blog/measure-cost-per-task-for-ai-agents).
3. **Model calls per task.** Catches retry storms and agents that plan in circles.
4. **Model mix.** A rising share of calls to an expensive model, for example after a fallback rule kicks in.
5. **Spend by caller.** One user or key suddenly responsible for a large share, a sign of misuse or a [denial-of-wallet attack](/blog/denial-of-wallet-attacks-on-ai-agents).

## How do you build a baseline?

1. **Baseline per agent version,** not per team. A new version starts with the previous version's baseline and earns its own after a week.
2. **Use the right window.** Hour-of-day and day-of-week patterns matter; compare Monday 10:00 to recent Mondays at 10:00.
3. **Use robust statistics.** Median and percentile bands resist the occasional spike better than averages.
4. **Exclude known events.** Planned batch jobs, evaluation runs and load tests should be tagged so they do not trigger alerts or distort the baseline.
5. **Recompute weekly** so normal growth does not become a constant alert.

## What thresholds and responses work?

| Severity | Example rule | Response |
|---|---|---|
| Info | Cost per hour above the 95th percentile band for 2 hours | Daily digest to owner |
| Warning | Tokens per task above 2× baseline for 30 minutes | Ticket to agent owner |
| High | Cost per hour above 3× baseline, or a new model appears in the mix | Page owner; consider throttling |
| Critical | Cost per hour above 10× baseline, or spend from an unknown caller | Automatic quarantine of the agent version; page on-call |

Tune multiples to your traffic. Low-volume agents need absolute floors (for example, ignore anything under $5 an hour) to avoid noise.

## Where should detection run?

Provider dashboards and cloud billing anomaly tools work on billing data, which usually arrives hours later and is grouped by key, project or account. They rarely know which agent version made a call. Detection at the gateway, where each request is tied to an agent and version in real time, is faster and more precise. Use billing tools as a backstop and reconciliation check. OWASP describes the risk of uncontrolled usage as [LLM10:2025 Unbounded Consumption](https://genai.owasp.org/llmrisk/llm102025-unbounded-consumption/). For architecture choices, see the [gateway comparison pillar](/blog/llm-gateway-vs-agent-gateway-vs-control-plane).

## How Opencontroller does this

Opencontroller's LLM Gateway (Lyzr Gateway) attributes cost per agent and per version, which is the basis for per-version baselines. Rogue-agent handling monitors behavioural deviation, raises alerts, and can automatically quarantine and kill an agent, with policies applied across the estate. Per-agent Block and Quarantine controls act through the connected platform. See the [capability reference](/capability-reference) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Can we rely on our cloud provider's cost anomaly tool for AI spend?**
As a backstop, yes. It works from billing data that lags and is grouped by account or key, so it is slower and less precise than detection at the gateway per agent version.

**Should an anomaly automatically stop an agent?**
Only at the critical tier, and preferably by quarantine rather than deletion, so the owner can investigate. Lower tiers should notify the owner.

**How is Opencontroller priced?**
Opencontroller is licensed annually per deployment, not per seat or usage, starting from $250K per deployment per year.

## Key facts

AI spend anomaly detection baselines each agent version on cost per hour, tokens per task, model calls per task, model mix and spend by caller, then alerts on deviation. Illustrative example: a regression raising tokens per task from 18,000 to 70,000 is caught within an hour by a 3x hourly-cost rule, versus about 3.6 days for an 80% monthly budget alert. OWASP LLM10:2025 covers unbounded consumption. Opencontroller attributes cost per agent and per version, monitors behavioural deviation, and can automatically quarantine and kill rogue agents. Licensing starts from $250K per deployment per year.

## Related

- [How do you set budgets and alerts for AI agent spend?](https://opencontroller.lyzr.com/blog/set-budgets-and-alerts-for-ai-agent-spend.md)
- [What is a denial-of-wallet attack on AI agents?](https://opencontroller.lyzr.com/blog/denial-of-wallet-attacks-on-ai-agents.md)
- [How do you measure cost per task for an AI agent?](https://opencontroller.lyzr.com/blog/measure-cost-per-task-for-ai-agents.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
