---
title: How should data classification labels control what AI agents can read?
url: https://opencontroller.lyzr.com/blog/data-classification-labels-for-ai-agents
published: 2026-11-15
updated: 2026-11-15
author: Lyzr
---

# How should data classification labels control what AI agents can read?

**Data classification labels should act as policy inputs at three points: when an agent retrieves data, when it sends data to a model, and when it writes output. Retrieval should filter by label and by the user the agent acts for; the gateway should route or block requests by the highest label in the context; and output should inherit the highest label it was built from. Labels only work if the agent and its tools can read them, so check that first.**

Most enterprises already label documents and tag data catalogue columns. The gap is that many agents ignore those labels because retrieval runs under a broad service identity. This post shows how to close that gap. For identity design, start with the pillar on [on-behalf-of delegation](/blog/ai-agent-identity-on-behalf-of-delegation).

## Why don't existing labels protect agent access by default?

Labels describe sensitivity. They do not enforce anything on their own. Enforcement happens in the system that reads the label. Three common failures:

1. **The agent's identity can read everything.** A shared index built with an admin account returns "Highly Confidential" files to anyone who asks the agent.
2. **Labels are lost in chunking.** Documents are split into vector chunks, and the label metadata is not copied to each chunk.
3. **The model provider sees everything.** Even if the user may see the data, policy may forbid sending it to an external model.

## How does Microsoft 365 Copilot handle sensitivity labels?

Microsoft 365 Copilot is a useful reference because Microsoft documents the behaviour in detail. According to [Microsoft Purview guidance](https://learn.microsoft.com/purview/ai-m365-copilot-considerations):

- Copilot and agents honour the **EXTRACT** (Copy) usage right on encrypted content. If a user has VIEW but not EXTRACT, Copilot will not summarise the content but can link to it.
- A Purview DLP policy can stop Copilot and agents summarising labelled files and emails, whether encrypted or not.
- New content created from labelled items inherits the label, and an inherited label replaces a lower-priority label applied manually.

The pattern generalises: enforce on the user's rights, add a policy layer for what AI may process, and carry labels through to outputs.

## What policy should each classification level get?

Start with a matrix like this and adjust to your scheme. The point is to make the rule explicit and testable.

| Classification | Agent may retrieve? | Allowed model destinations | Output handling |
|---|---|---|---|
| Public | Yes | Any approved model | No label required |
| Internal | Yes, for authenticated users | Approved external models under enterprise terms | Label "Internal" |
| Confidential | Only on behalf of a user who can access it | Approved models with no training on data and agreed retention | Inherit label; no external sharing tools |
| Highly confidential / regulated | Only for named agents with a documented need | Internal or self-hosted models only, or blocked | Inherit label; log every access |

## How do you implement label-aware agents?

Follow these steps in order:

1. **Inventory labelled sources.** List every data source each agent can reach, and whether the source exposes labels or tags through its API.
2. **Retrieve as the user.** Use delegated, on-behalf-of access so retrieval returns only what the user can see. See [control AI agent retrieval from vector databases](/blog/control-ai-agent-retrieval-from-vector-databases).
3. **Copy labels to chunks.** At indexing time, store the source label and access list on every chunk, and filter on them at query time.
4. **Compute the context label.** Before each model call, take the highest label among the retrieved items.
5. **Route by context label.** At the gateway, allow, route to an internal model, redact or block based on that label.
6. **Label the output.** Write outputs with the inherited label, and block tools that would send them outside allowed destinations.
7. **Test it.** Seed a test document at each level and confirm an unauthorised user's agent cannot retrieve or summarise it.

## What should you log as evidence?

Record, per agent request: agent and version, acting user, labels of retrieved items, the computed context label, the routing decision and the model destination. This turns "our agents respect classification" into something an auditor can sample. The [least privilege checklist for AI agents](/blog/least-privilege-checklist-for-ai-agents) lists the related access evidence.

## How Opencontroller does this

Opencontroller sits in the traffic path where agents call models and tools, and policy as code can allow, route, redact or block based on request attributes, including classification metadata you pass in. Delegated on-behalf-of identity keeps retrieval scoped to the user, and the LLM Gateway can route sensitive requests to internal models. Posture flags agents with broad data permissions. Enforcement depth depends on the connector and traffic path. See [how to redact PII in LLM prompts at the gateway](/blog/how-to-redact-pii-in-llm-prompts-at-the-gateway) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Do AI agents automatically respect sensitivity labels?**
Only if the system they read through enforces them. Microsoft 365 Copilot honours EXTRACT rights; custom agents using a shared index often do not.

**Should highly confidential data go to external models?**
Many enterprises block it or route it to internal models. Make the rule explicit per classification and enforce it at the gateway.

**What label should agent output get?**
The highest label among the inputs it was built from, so a summary of a confidential file stays confidential.

## Key facts

Data classification labels should drive agent policy at retrieval (filter by label and acting user), model call (route or block by highest context label) and output (inherit highest label). Microsoft 365 Copilot honours the EXTRACT usage right; Purview DLP can stop Copilot summarising labelled content; inherited labels override lower-priority manual labels. Opencontroller enforces in the traffic path (allow, route, redact, block) with policy as code, delegated on-behalf-of identity and LLM Gateway routing; Posture flags broad permissions. Pricing starts from $250K per deployment per year.

## Related

- [How do you control what an AI agent can retrieve from a vector database?](https://opencontroller.lyzr.com/blog/control-ai-agent-retrieval-from-vector-databases.md)
- [What does least privilege look like for an AI agent?](https://opencontroller.lyzr.com/blog/least-privilege-checklist-for-ai-agents.md)
- [How do you redact PII from LLM prompts at the gateway?](https://opencontroller.lyzr.com/blog/how-to-redact-pii-in-llm-prompts-at-the-gateway.md)
- [More on Agent identity and access](https://opencontroller.lyzr.com/blog/cluster/agent-identity-and-access)
