---
title: What should acceptance testing cover for an AI agent governance platform?
url: https://opencontroller.lyzr.com/blog/acceptance-testing-for-ai-governance-platform
published: 2026-11-13
updated: 2026-11-13
author: Lyzr
---

# What should acceptance testing cover for an AI agent governance platform?

**Acceptance testing should prove each control works on your estate, not on a demo tenant. Plant a shadow agent and time its detection, offboard a test user and check the owner flag, send test PII through the gateway, confirm which traffic is actually routed, reconcile cost to invoices, and run a quarantine and release round trip. Each test needs a pass criterion agreed before go-live.**

A pilot shows a platform can work. Acceptance testing shows it does work, in your environment, at the coverage you were promised. Write the tests into the contract or statement of work so "done" is not a matter of opinion.

## How do you set up the tests?

Prepare four things first:

1. **A test agent per platform.** A small, harmless agent on each in-scope cloud and SaaS platform, created by someone outside the project team, so the platform cannot have been told about it.
2. **A test identity.** A user in your identity provider who can own agents and be disabled without harm.
3. **Synthetic sensitive data.** Fake card numbers, national ID formats and email addresses that match your redaction rules. Never use real data.
4. **A cost baseline.** Last month's model provider invoices, by account or project.

## What are the acceptance tests?

| # | Test | How to run it | Pass criterion (example) |
|---|---|---|---|
| 1 | Shadow agent detection | Create an unregistered agent on each platform | Found and listed within the agreed discovery interval, e.g. 24 hours |
| 2 | Discovery coverage | Compare discovered inventory with a manual list from platform admins | Agreed share of known agents found, misses explained |
| 3 | Ownership from identity | Register an agent to the test user | Owner shows name, manager and group from the identity provider |
| 4 | Owner offboarding | Disable the test user | Agent flagged as orphaned within the agreed time, alert raised |
| 5 | Traffic path confirmation | For each agent, record whether calls go through the gateway | Every agent marked routed, observed only, or registry only, matching the design document |
| 6 | Redaction in path | Send synthetic PII through the gateway | Redacted or blocked per policy; original not stored in logs |
| 7 | Policy block | Call a model or tool that policy forbids | Request blocked with a stable reason code, event logged |
| 8 | SSO-only login | Try a local password login to the console | Refused; SSO login works; roles follow identity groups |
| 9 | Cost reconciliation | Compare platform cost totals with provider invoices | Within an agreed tolerance, e.g. 5%, with gaps explained |
| 10 | Quarantine round trip | Quarantine a test agent, then release it | Agent stops on its platform, owner notified, agent resumes on release |
| 11 | Alert routing | Trigger a policy violation | Ticket and incident channel message created with agent, owner and policy |
| 12 | Evidence export | Export the evidence record for one control | Shows who, what, when and which policy, in a format auditors accept |

The numbers in the pass column are examples. Agree your own, but agree them in writing.

## Why does traffic path confirmation matter so much?

Test 5 is the one most often skipped. A governance platform can know about an agent without sitting in its traffic. Registration is not runtime takeover. Some agents will be fully routed through a gateway. Some will be observed through platform logs. Some will only be in the registry. That is normal, but you need to know which is which before you rely on a block or a redaction rule.

Ask for a per-agent list showing the control depth, then test one agent at each depth. If a policy claims to block, send a request that should be blocked. If a dashboard claims to show cost, check it against the invoice. See [document AI traffic paths before enforcement](/blog/document-ai-traffic-paths-before-enforcement).

## What about the negative tests?

Good acceptance plans also check what should not happen:

- A user without the right group cannot see other teams' agents or prompts.
- The connector credentials cannot write to platforms where you approved read-only access.
- Disabling the governance platform does not take down agents that only run in observe mode.
- A gateway outage triggers the agreed fallback, not a silent failure.

For detection depth, use the planted-asset method in [questions to test AI agent discovery coverage](/blog/questions-to-test-ai-agent-discovery-coverage). For the kill switch, see [test your AI agent kill switch](/blog/test-your-ai-agent-kill-switch).

## When should you run acceptance tests?

Run them three times:

1. **At the end of the pilot,** on a subset of platforms, to decide whether to buy.
2. **Before go-live,** on every in-scope platform, as the contract acceptance gate.
3. **Every quarter,** as a control test. Discovery and routing drift as teams add platforms and keys.

The pilot itself is covered in [how to run a pilot of an AI agent governance platform](/blog/how-to-run-a-pilot-of-an-ai-agent-governance-platform), and the wider buyer list in the [evaluation checklist](/blog/ai-agent-governance-platform-evaluation-checklist).

## How Opencontroller does this

Opencontroller supports each of these tests with live capabilities: auto-discovery of agents, models and API keys across AWS, Azure and Google Cloud and connected agent platforms; a governed registry with owners; OIDC SSO; Posture checks; LLM Gateway cost attribution per agent and version; and per-agent Block and Quarantine controls that act through the connected platform. Control depth follows the connector and traffic path, from registry to observe to enforce, and that is stated per agent. Lyzr engineers embed to run acceptance tests with your team. See the [capability reference](/capability-reference) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**What is the most important acceptance test for AI agent governance?**
Planting an unregistered agent on each platform and timing its detection. It tests discovery on your real estate, which is what the rest of governance depends on.

**How close should governance cost data be to provider invoices?**
Agree a tolerance, such as 5%, and require the gaps to be explained. Differences usually come from traffic that bypasses the gateway or from platform fees outside token pricing.

**Should acceptance tests be repeated after go-live?**
Yes. Run them quarterly as control tests, because new platforms, keys and agents change coverage over time.

## Key facts

Acceptance testing for an AI agent governance platform should include 12 tests with pre-agreed pass criteria: planted shadow agent detection per platform, discovery coverage vs admin lists, owner from identity provider, owner offboarding flag, per-agent traffic path confirmation (routed / observed / registry only), PII redaction in path, policy block with reason code, SSO-only login, cost reconciliation to invoices within a tolerance (e.g. 5%), quarantine and release round trip, alert routing, and evidence export. Run at pilot end, before go-live and quarterly. Opencontroller by Lyzr supports these with discovery, registry, OIDC SSO, Posture, LLM Gateway cost attribution and per-agent Block/Quarantine; pricing starts from $250K per deployment per year.

## Related

- [How should you run a pilot of an AI agent governance platform?](https://opencontroller.lyzr.com/blog/how-to-run-a-pilot-of-an-ai-agent-governance-platform.md)
- [What questions reveal whether an AI agent discovery tool really covers your estate?](https://opencontroller.lyzr.com/blog/questions-to-test-ai-agent-discovery-coverage.md)
- [How do you test that your AI agent kill switch actually works?](https://opencontroller.lyzr.com/blog/test-your-ai-agent-kill-switch.md)
- [More on Buying and comparison](https://opencontroller.lyzr.com/blog/cluster/buying-and-comparison)
