---
title: How do you govern LlamaIndex agents in production?
url: https://opencontroller.lyzr.com/blog/govern-llamaindex-agents
published: 2026-11-12
updated: 2026-11-12
author: Lyzr
---

# How do you govern LlamaIndex agents in production?

**Govern LlamaIndex agents by controlling three things: which tools each agent is given, which model endpoint it calls, and where its instrumentation events go. LlamaIndex has a built-in instrumentation module and OpenTelemetry export that make every step visible. Policy enforcement, ownership and an inventory across teams are not part of the framework, so you add them around it.**

LlamaIndex started as a retrieval framework and now includes agents (such as `FunctionAgent`, `ReActAgent` and `AgentWorkflow`) built on its event-driven Workflows. Many enterprise RAG assistants are LlamaIndex agents that have quietly gained tools. That makes them a common blind spot.

## Where are the control points in a LlamaIndex agent?

| Layer | What LlamaIndex provides | Control point |
|---|---|---|
| Model | LLM classes for many providers, including OpenAI-compatible endpoints | Point every agent at an approved gateway endpoint; restrict models |
| Tools | `FunctionTool`, tool specs from LlamaHub, MCP tools | Approve each tool and tool spec; deny write tools by default |
| Retrieval | Indexes, retrievers and query engines over your data | Check that the index only holds data the agent's users may see |
| Workflow steps | Event-driven steps with typed events | Insert approval or validation steps before side effects |
| Instrumentation | Dispatcher, events, spans and handlers | Send everything to a trace store you own |

The retrieval row deserves attention. A LlamaIndex agent usually answers from an index built in advance. If that index was built from a shared drive with broad access, the agent can surface documents to people who could not open them directly. Governance starts with what went into the index.

## How does LlamaIndex instrumentation work?

The instrumentation module has five parts, documented by LlamaIndex:

1. **Events**: single moments, such as `LLMChatStartEvent` and `LLMChatEndEvent`. Custom events subclass `BaseEvent`.
2. **Spans**: periods of execution that contain events. Custom spans subclass `BaseSpan`.
3. **Event handlers**: subclass `BaseEventHandler` and implement `handle()`.
4. **Span handlers**: subclass `BaseSpanHandler` and implement `new_span()`, `prepare_to_exit_span()` and `prepare_to_drop_span()`.
5. **Dispatchers**: created with `instrument.get_dispatcher(__name__)`. They form a hierarchy like Python's logging library, and propagate to their parent by default. Handlers on the root dispatcher act as global handlers.

For Workflows, the runtime "tracks the input and output of every workflow step." Internal `StepStateChanged` events report each step as `PREPARING`, `RUNNING` or `NOT_RUNNING`; pass `expose_internal=True` to see them.

For export, install `llama-index-observability-otel`, configure `LlamaIndexOpenTelemetry` with a span exporter, and call `start_registering()`. Spans include step names, timings, input and output attributes, and parent-child links. LlamaIndex documents integrations with Arize Phoenix and Langfuse, and Opik through OTLP.

## What should a LlamaIndex governance checklist include?

1. **Register the agent.** Record owner, version, model, tools, indexes and the data sources behind each index.
2. **Review index sources.** List every source an index was built from. Remove data the agent's audience should not see, or filter retrieval by the user's permissions.
3. **Approve tools.** Keep an allow-list of tools and tool specs per agent. Treat MCP tools as third-party integrations with their own review.
4. **Use one model endpoint.** Configure the LLM class with your gateway's base URL so routing, budgets and model allow-lists are enforced outside the code.
5. **Attach a root handler.** Add one span handler on the root dispatcher that exports to OpenTelemetry. Do not rely on each team to add its own.
6. **Control what traces contain.** Span attributes can include inputs and outputs. Decide where prompts and retrieved text may be stored, and for how long. See [how long to retain AI agent logs](/blog/how-long-to-retain-ai-agent-logs).
7. **Gate side effects in the workflow.** Add an explicit step that validates or requests approval before any write, payment or message send.
8. **Evaluate groundedness.** RAG agents fail by answering beyond their sources. Score production answers for groundedness, not just test sets.

## What does LlamaIndex not cover?

LlamaIndex shows you what one agent did. It does not show how many LlamaIndex agents exist, which ones were deployed outside your platform team, which API keys they use, or how their behaviour compares with agents on other frameworks. It also has no estate-wide policy engine or kill switch. Those belong in a control plane. See [what an AI agent control plane is](/blog/what-is-an-ai-agent-control-plane) and the [cross-platform governance guide](/blog/how-to-govern-ai-agents-across-platforms).

## How Opencontroller does this

Opencontroller by Lyzr discovers agents, models and API keys across AWS, Azure and Google Cloud and keeps them in a governed registry with versions, tools and MCP servers. LlamaIndex agents can call models through the OpenAI-compatible LLM Gateway, which adds routing, fallback, caching and cost attribution per agent and per version. Live groundedness and task-adherence guardrails and LLM judges score production traffic, and traces use OpenTelemetry. Control depth depends on the traffic path: agents that route through the gateway can be enforced in the path; others are registered and observed. [Book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Does LlamaIndex support OpenTelemetry?**
Yes. The llama-index-observability-otel package exports workflow and agent spans to any OpenTelemetry span exporter.

**What is the LlamaIndex root dispatcher?**
It is the top of the dispatcher hierarchy, returned by instrument.get_dispatcher() with no name. Handlers attached to it receive events and spans from the whole application.

**Can a LlamaIndex agent leak documents through retrieval?**
It can if the index holds documents the user could not otherwise open. Review index sources or filter retrieval by user permissions.

## Key facts

LlamaIndex agents (FunctionAgent, ReActAgent, AgentWorkflow) run on event-driven Workflows. Its instrumentation module uses dispatchers (hierarchical, like Python logging), events (e.g. LLMChatStartEvent), spans, event handlers and span handlers; handlers on the root dispatcher are global. The llama-index-observability-otel package exports spans via OpenTelemetry; documented integrations include Arize Phoenix, Langfuse and Opik. Key governance risk: indexes built from broadly shared data. Opencontroller by Lyzr provides discovery across AWS, Azure and Google Cloud, a governed registry, an OpenAI-compatible LLM Gateway with cost per agent, and live groundedness guardrails. Pricing starts from $250K per deployment per year.

## Related

- [How do you govern LangGraph agents in production?](https://opencontroller.lyzr.com/blog/govern-langgraph-agents-in-production.md)
- [How do you export AI agent traces with OpenTelemetry?](https://opencontroller.lyzr.com/blog/export-ai-agent-traces-with-opentelemetry.md)
- [How do you govern CrewAI agents in production?](https://opencontroller.lyzr.com/blog/govern-crewai-agents-in-production.md)
- [More on Platform Guides](https://opencontroller.lyzr.com/blog/cluster/platform-guides)
