---
title: How do you validate an AI agent's output before it acts?
url: https://opencontroller.lyzr.com/blog/validate-ai-agent-output-before-it-acts
published: 2026-11-13
updated: 2026-11-13
author: Lyzr
---

# How do you validate an AI agent's output before it acts?

**Put a validation step between the model's decision and the tool call, and run checks in a fixed order: schema first, then business rules, then groundedness and task adherence, then a risk gate. Cheap deterministic checks run first and stop most bad actions. Anything that fails returns a clear error to the agent instead of executing.**

A model's output is a proposal. In a chatbot, a wrong proposal is a wrong sentence. In an agent, it is a wrong refund, a wrong email or a deleted record. OWASP lists Improper Output Handling as LLM05 in its 2025 Top 10 for LLM applications. Agents make the risk sharper because their output feeds straight into tools.

## Where should validation sit in an agent?

Validation belongs at the boundary where text becomes action. In practice that is one of three places:

- **Inside the tool wrapper.** The tool function checks its own arguments before doing anything. The agent cannot skip this.
- **In the traffic path.** A gateway or guardrail service checks requests and responses for every agent. This gives you the same checks everywhere.
- **In the orchestration graph.** A dedicated validation node runs before the action node. Frameworks like LangGraph make this explicit.

Use the tool wrapper for checks specific to that tool. Use the traffic path for checks that apply to all agents.

## What checks should run, and in what order?

Order matters. Each layer is slower and more expensive than the one before it.

| Order | Check | Example | Cost |
|---|---|---|---|
| 1 | Structure | Arguments match the tool's JSON Schema; required fields present; types and enums valid | Microseconds |
| 2 | Business rules | Refund under the agent's limit; recipient on an allowed domain; record belongs to the requesting user | Milliseconds |
| 3 | Data safety | No secrets or restricted personal data in outbound text | Milliseconds |
| 4 | Groundedness | Factual claims are supported by retrieved sources | One model call |
| 5 | Task adherence | The action fits the task the agent was given | One model call |
| 6 | Risk gate | High-impact or irreversible actions go to a person or are blocked | Varies |

Deterministic checks (1–3) should catch most failures. Model-based checks (4–5) catch the rest: plausible-looking outputs that are wrong or off-task.

## How do you write good business-rule checks?

Business rules are where agents most often fail quietly. Write them as code, not as prompt instructions. A prompt saying "never refund more than $500" is a request. A check that rejects `amount > 500` is a control.

A practical checklist:

1. List every tool that changes state: writes, sends, pays, deletes.
2. For each, write the limits a careful employee would apply: amounts, counts, recipients, time windows.
3. Check ownership: the record the agent acts on must belong to the user it acts for.
4. Check idempotency: the same action should not run twice within a short window.
5. Keep numbers out of the model. Have tools calculate totals; see [should AI agents do financial calculations](/blog/should-ai-agents-do-financial-calculations).
6. Version the rules with the agent, so you know which rules applied to which run.

## When do you need model-based checks?

Use groundedness checks when the agent states facts that a user or another system will rely on: policy answers, account details, summaries of documents. A [groundedness guardrail](/blog/what-is-a-groundedness-guardrail) compares each claim with the sources the agent retrieved.

Use task-adherence checks when an agent has broad tools. An agent asked to summarise a ticket should not also close it. The check asks a separate model whether the proposed action matches the assigned task.

Set thresholds per agent. Start strict, measure false positives on real traffic for two weeks, then tune.

## What should happen when validation fails?

Don't just block silently. A good failure path:

- Returns a specific, machine-readable error so the agent can try again ("amount exceeds limit of 500").
- Caps retries at two or three to prevent loops.
- Logs the failed proposal, the rule that failed and the agent version.
- Escalates repeated failures from the same agent to its owner.

For irreversible actions, route to a person. The [human approval guide](/blog/human-approval-for-high-risk-ai-agent-actions) covers how to design that step so approvers do not rubber-stamp.

## How Opencontroller does this

Opencontroller runs live groundedness and task-adherence guardrails on agent traffic, and LLM judges that score production traffic so you can see failure rates per agent and version. Policy as code applies the same rules across the estate, and the Agent Gateway puts control in the path to allow, route, redact or block. Deterministic tool-level checks stay in your agent code. Start from the [rogue-agent playbook](/blog/how-to-stop-a-rogue-ai-agent) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Should validation happen in the agent code or outside it?**
Both. Put structural and business-rule checks next to the tool, where the agent cannot skip them. Put shared guardrails in the traffic path so every agent gets the same checks.

**What should happen when an agent output fails validation?**
Return a specific error so the agent can correct itself, cap retries at two or three, and log the failure with the agent version.

**Do model-based checks slow agents down?**
Yes, each adds a model call. Run them only on actions and answers that matter, after cheaper deterministic checks have passed.

## Key facts

Validate an AI agent's output before it acts by running checks in a fixed order between the model's decision and the tool call: (1) structural validation against a JSON Schema for the tool's arguments; (2) deterministic business-rule checks such as amount limits, allowed recipients and record ownership; (3) model-based checks such as groundedness (is the claim supported by retrieved sources) and task adherence (is the action within the agent's assigned task); (4) a risk gate that sends high-impact actions to human approval or blocks them. Cheap deterministic checks run first; failures return a specific error to the agent instead of executing. OWASP lists Improper Output Handling (LLM05:2025) as a top LLM risk. Opencontroller by Lyzr provides live groundedness and task-adherence guardrails and LLM judges scoring production traffic.

## Related

- [What is a groundedness guardrail and when should it block a response?](https://opencontroller.lyzr.com/blog/what-is-a-groundedness-guardrail.md)
- [Which AI agent actions need human approval?](https://opencontroller.lyzr.com/blog/human-approval-for-high-risk-ai-agent-actions.md)
- [Should AI agents do financial calculations themselves?](https://opencontroller.lyzr.com/blog/should-ai-agents-do-financial-calculations.md)
- [More on Runtime control](https://opencontroller.lyzr.com/blog/cluster/runtime-control)
