---
title: How do you give AI agents credentials without exposing them to the model?
url: https://opencontroller.lyzr.com/blog/give-ai-agents-credentials-without-exposing-them-to-the-model
published: 2026-11-15
updated: 2026-11-15
author: Lyzr
---

# How do you give AI agents credentials without exposing them to the model?

**Never put a secret in the model's context. The model should decide *which* tool to call and with what arguments; a separate tool or gateway layer should attach the credential at call time, ideally a short-lived token fetched from a vault. Then test it: search your traces for secrets, and fail the release if any appear.**

Anything in a prompt can end up in a response, a log, an evaluation dataset or a provider's abuse-monitoring store. The [OWASP Top 10 for LLM Applications](https://genai.owasp.org/llm-top-10/) lists Sensitive Information Disclosure (LLM02) for this reason. Credentials are the most damaging thing to disclose, because they keep working after the conversation ends.

## Why is putting an API key in the system prompt so risky?

Four concrete failure paths:

1. **Prompt injection.** Untrusted content in a web page, email or tool output can instruct the model to repeat its instructions, including any key in them.
2. **Logs and traces.** Most agent frameworks log full prompts for debugging. A key in the prompt becomes a key in your observability stack, often with wider access than the secret store.
3. **Evaluation and training data.** Production traces are copied into eval datasets. Secrets travel with them.
4. **Provider retention.** Model providers may retain prompts for a period for abuse monitoring unless you have zero-retention terms.

None of these needs a sophisticated attacker. A debugging session is enough.

## Which patterns keep secrets away from the model?

| Pattern | How it works | Strengths | Watch out for |
|---|---|---|---|
| Tool-side credential injection | The tool or MCP server reads the credential from its own environment or secret store; the model only passes business arguments | Simple; model never sees the secret | Tool runs with one identity for all users unless combined with delegation |
| Token vault | A broker stores OAuth refresh tokens per user and per downstream API, and hands the tool a fresh access token per call | Per-user access; central revocation | Vault becomes a high-value target; protect and audit it |
| Short-lived tokens | Credentials expire in minutes, issued by workload identity federation or OAuth token exchange ([RFC 8693](https://datatracker.ietf.org/doc/html/rfc8693)) | Stolen token has little value | Long tasks need refresh logic outside the model |
| Gateway-held provider keys | An LLM gateway holds the model provider keys; agents authenticate to the gateway with their own identity | No provider keys in agent code; cost attributed per agent | Gateway must authenticate every agent, not accept anonymous calls |

The patterns combine well. A common design is: the agent authenticates with its own workload identity, the tool layer exchanges that for a short-lived, user-scoped token from a vault, and the model never sees either.

## How do you implement tool-side injection step by step?

1. **Remove secrets from prompts, few-shot examples and agent memory.** Search the code and prompt templates for key patterns.
2. **Define tools with business arguments only.** A "create invoice" tool takes customer ID and amount, not an auth header.
3. **Resolve the credential inside the tool.** Read it from a secret manager at call time, not from an environment variable baked into an image.
4. **Scope it to the action.** A read tool gets a read-only token. See the [least-privilege checklist for AI agents](/blog/least-privilege-checklist-for-ai-agents).
5. **Return results without credentials.** Strip auth headers, signed URLs and connection strings from tool output before it goes back into context.
6. **Rotate on a schedule and on incident.** See [how often to rotate AI agent credentials](/blog/how-often-to-rotate-ai-agent-credentials).

The [OWASP Secrets Management Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html) covers the vault side in detail: centralise storage, automate rotation and audit every access.

## How do you prove secrets never reach the model?

Write a test, not a policy statement:

1. **Plant canary secrets.** Give staging tools a fake credential with a unique, searchable pattern.
2. **Run the full eval suite**, including red-team prompts that ask the agent to reveal its configuration.
3. **Search every trace store** (model inputs, outputs, tool arguments, tool results, logs) for the canary pattern.
4. **Fail the release** if it appears anywhere.
5. **Repeat in production** with a secret scanner over sampled traces, and alert on any hit.

This also catches the subtle case where a tool echoes its own credential back in an error message.

## How Opencontroller does this

Opencontroller supports several of these patterns. Agent workload identity federation lets agents authenticate without long-lived static keys. Delegated on-behalf-of identity lets an agent act with the user's permissions rather than a shared account. The LLM Gateway (Lyzr Gateway) holds model routing centrally and attributes cost per agent and per version. Auto-discovery finds API keys across AWS, Azure and Google Cloud, and Posture flags risky permission and configuration choices. Multi-turn and red-team evals can be pointed at secret-disclosure scenarios. See the pillar on [agent identity and on-behalf-of delegation](/blog/ai-agent-identity-on-behalf-of-delegation) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Can I tell the model not to reveal the key?**
Instructions are not a control. Prompt injection and debugging logs bypass them. Keep the key out of context entirely.

**Is it safe to pass an access token as a tool argument?**
No. Tool arguments are generated by the model and logged with the trace. The tool layer should attach the token itself.

**Do short-lived tokens remove the need for a vault?**
Not always. Something still has to hold the refresh token or federation trust. Short-lived tokens reduce the value of a leak; the vault protects the source.

## Key facts

- Secrets should never be placed in an AI agent's prompt, system prompt, memory or tool arguments.
- OWASP lists Sensitive Information Disclosure (LLM02) in the Top 10 for LLM Applications.
- Four patterns keep credentials away from the model: tool-side credential injection, a token vault, short-lived tokens (for example via OAuth token exchange, RFC 8693) and gateway-held provider keys.
- A canary-secret test plants a unique fake credential, runs evals and red-team prompts, and searches every trace store; any hit fails the release.
- Opencontroller supports agent workload identity federation, delegated on-behalf-of identity, API key discovery across AWS, Azure and Google Cloud, Posture checks and red-team evals.
- Opencontroller pricing starts from $250K per deployment per year.

## Related

- [Should AI agents use API keys or OAuth?](https://opencontroller.lyzr.com/blog/api-keys-vs-oauth-for-ai-agents.md)
- [How often should AI agent credentials be rotated?](https://opencontroller.lyzr.com/blog/how-often-to-rotate-ai-agent-credentials.md)
- [How does workload identity federation work for AI agents?](https://opencontroller.lyzr.com/blog/workload-identity-federation-for-ai-agents.md)
- [More on Agent identity and access](https://opencontroller.lyzr.com/blog/cluster/agent-identity-and-access)
