---
title: How to evaluate an AI agent governance platform: 30 questions for your RFP (2026)
url: https://opencontroller.lyzr.com/blog/ai-agent-governance-platform-evaluation-checklist
published: 2026-11-11
updated: 2026-11-11
author: Lyzr
---

# How to evaluate an AI agent governance platform

**Judge an AI agent governance platform on five things: how much of your agent estate it can see, whether every agent gets its own identity, whether it can stop an action *before* it happens, whether it produces audit-ready evidence, and whether it works without moving your agents.** Below are 30 questions to put in your RFP, with what a strong answer looks like.

## Before you write the RFP

List where your agents run today (clouds, SaaS platforms, custom code, developer devices) and pick two real workflows to test, one read-only and one that changes data. Ask every vendor to demonstrate both on your environment, not a sandbox story.

## 1. Discovery and inventory

| # | Question | Strong answer |
| --- | --- | --- |
| 1 | Which clouds and agent platforms do you discover automatically? | Named list covering your estate, with what is discovered on each |
| 2 | Do you discover agents built outside your platform? | Yes, including custom code and SaaS agents |
| 3 | Can you find model API keys and map them to agents? | Yes, across secret stores and code |
| 4 | Do you cover coding agents and MCP servers on developer devices? | Yes, via an IT-deployed endpoint component |
| 5 | How often does discovery run? | Continuously or at least daily, with alerts on new agents |
| 6 | What does every registry record contain? | Owner, purpose, risk tier, version, dependencies, status |

## 2. Identity and access

| # | Question | Strong answer |
| --- | --- | --- |
| 7 | Does each agent get its own identity? | Yes, federated to your IdP, no shared keys |
| 8 | How do agents act on behalf of users? | Token exchange carrying user and agent (RFC 8693) |
| 9 | Are tokens audience-restricted and short-lived? | Yes (RFC 8707), per resource |
| 10 | How are background agents authorized? | Explicit, time-bound delegation grants with revocation |
| 11 | Which IdPs and provisioning standards do you support? | OIDC, SAML, SCIM with your IdP |
| 12 | How fast can one agent's access be revoked? | Seconds, without affecting users or other agents |

## 3. Enforcement in the path

| # | Question | Strong answer |
| --- | --- | --- |
| 13 | Can you block a tool or API call before it executes? | Yes, at a gateway in the path, with allow / block / hold decisions |
| 14 | Can consequential actions be held for human approval? | Yes, with the approval linked to the request |
| 15 | Do you enforce per-agent tool and MCP allow-lists? | Yes |
| 16 | Do you govern model calls too? | Budgets, routing, fallback, caching, redaction per agent |
| 17 | What happens if your policy service is unavailable? | Configurable fail-closed per risk tier |
| 18 | Can you quarantine or kill a misbehaving agent? | Yes, automatically on defined signals and manually |
| 19 | What latency do you add? | Published benchmark with test conditions, and a test on your traffic |

## 4. Evaluation, quality and improvement

| # | Question | Strong answer |
| --- | --- | --- |
| 20 | Are evaluations tied to a specific agent version? | Yes, as release evidence |
| 21 | Do you support multi-turn and red-team evaluation? | Yes |
| 22 | Do you score production traffic, not just test sets? | Yes, with LLM judges and thresholds |
| 23 | Do you detect behavioural drift per agent? | Yes, against a per-agent baseline |

## 5. Evidence and policy

| # | Question | Strong answer |
| --- | --- | --- |
| 24 | Can you show why a specific action was allowed? | One trace: agent, version, caller, policy, decision, outcome |
| 25 | Which frameworks are mapped out of the box? | e.g. OWASP, MITRE ATLAS, NIST, ISO 42001, described as evidence, not certification |
| 26 | Can we add our own policies? | Yes, as policy as code |
| 27 | Can model and tool spend be attributed per agent and version? | Yes |

## 6. Deployment and lock-in

| # | Question | Strong answer |
| --- | --- | --- |
| 28 | Do agents have to move to your runtime? | No; registration, observation and enforcement work where agents run |
| 29 | Can it run in our own cloud account? | Yes, without exporting telemetry or source to the vendor |
| 30 | Which open standards do you use? | OpenTelemetry, MCP, OpenAI-compatible endpoints, OIDC/SCIM, REST |

## Red flags

- "Compliant with the EU AI Act" claims. Platforms provide evidence; organisations achieve compliance.
- Governance that only works for agents built on the vendor's own builder.
- Dashboards with no way to block, hold or revoke.
- Per-seat or per-call pricing that makes discovering more agents more expensive.
- A demo that can't show one end-to-end trace from user request to tool call.

## Scoring

Weight sections by your risk. A common split: enforcement 30%, identity 20%, discovery 20%, evidence 15%, evaluation 10%, deployment 5%. Score only what the vendor demonstrated on your workflows.

## Where Opencontroller fits

[Opencontroller](/blog/what-is-opencontroller) by Lyzr was built around these questions: discovery across clouds and agent platforms, per-agent identity with on-behalf-of delegation, Agent and LLM gateways in the path, evaluations and production judges, quarantine and kill, and policy as code with framework mappings. It is licensed per deployment, not per seat or call. See the [capability reference](/capability-reference) and the [control plane comparison](/compare/controlplane), or [book a demo](https://www.lyzr.ai/book-demo/) to run your two workflows.

## Frequently asked questions

**Should we build this ourselves?**
Teams can build pieces (a gateway, a registry). The cost is in keeping connectors, identity flows and evidence current as platforms change.

**How long should a proof of concept take?**
Scope it by outcome, not calendar: discover your real estate and enforce one consequential workflow end to end. Agree that exit criterion with the vendor up front.

**Who should own the evaluation?**
Security and the AI platform team jointly, with a business owner for the test workflows.

## Related

- [What is an AI agent control plane?](https://opencontroller.lyzr.com/blog/what-is-an-ai-agent-control-plane.md)
- [LLM gateway vs agent gateway vs AI control plane: what's the difference?](https://opencontroller.lyzr.com/blog/llm-gateway-vs-agent-gateway-vs-control-plane.md)
- [What is Opencontroller? The open control plane for enterprise AI agents](https://opencontroller.lyzr.com/blog/what-is-opencontroller.md)
- [More on Buying and comparison](https://opencontroller.lyzr.com/blog/cluster/buying-and-comparison)
