---
title: How should users and applications authenticate to an LLM gateway?
url: https://opencontroller.lyzr.com/blog/how-should-apps-authenticate-to-an-llm-gateway
published: 2026-11-13
updated: 2026-11-13
author: Lyzr
---

# How should users and applications authenticate to an LLM gateway?

**Treat the console and the model endpoint as two different doors. People sign in to the console through your identity provider with SSO, and get roles from IdP groups. Applications and agents call the model endpoint with per-agent credentials: scoped virtual keys at minimum, short-lived tokens from your identity provider or workload identity where you can.**

A common finding in security reviews is an LLM gateway with its own username and password, outside SSO. That breaks offboarding, multi-factor policy and access reviews in one go. The fix is not just "add SSO", because most gateway traffic comes from machines, not people.

## Why are console access and API access different problems?

The console is where people change routing, budgets, keys and policies. It needs the same controls as any admin tool: SSO, MFA from the identity provider, group-based roles and automatic removal when someone leaves. Our guide to [SSO-only access for AI governance tools](/blog/sso-only-access-for-ai-governance-tools) covers that side.

The model endpoint is called by applications, agents and scripts, thousands of times a minute. Interactive SSO does not fit there. Machines need credentials that are scoped, short-lived where possible, and traceable to an owner.

## Which machine authentication options should you compare?

| Option | How it works | Secret to store? | Lifetime | Traceability | Use it for |
|---|---|---|---|---|---|
| Shared master key | One key for all callers | Yes | Long | None per caller | Avoid |
| Scoped virtual key | One gateway-issued key per agent, with budget and model limits | Yes | Set an expiry | Per agent | Minimum bar for production |
| OAuth client credentials | App gets an access token from the IdP; gateway validates it | Client secret or certificate | Minutes to an hour | Per client | Apps registered in your IdP |
| Workload identity federation | Platform-issued token is exchanged for short-lived credentials | No | Minutes | Per workload | Agents on cloud or Kubernetes |
| Token exchange on behalf of a user | Agent swaps a user token for a delegated token | No long-lived secret | Minutes | Per user and agent | Agents acting for a person |

The client credentials grant is defined in RFC 6749 section 4.4. Token exchange for delegation is defined in RFC 8693. For the trade-offs between keys and tokens, see [API keys vs OAuth for AI agents](/blog/api-keys-vs-oauth-for-ai-agents), and for the secret-free option, [workload identity federation for AI agents](/blog/workload-identity-federation-for-ai-agents).

## What is a sensible migration path?

1. **Put the console behind SSO.** Map IdP groups to admin, operator and viewer roles. Keep one or two break-glass local accounts in a vault, and alert on every use.
2. **Retire the shared master key.** Issue one virtual key per agent, each with a budget, allowed models and an expiry.
3. **Register agents as IdP applications** where your platform supports it, and move them to short-lived tokens.
4. **Use workload identity** for agents on cloud services and Kubernetes, so no secret is stored at all.
5. **Add on-behalf-of identity** for agents that act for a specific user, so the user is visible on every call.

Each step is independent. You can stop at step 2 for low-risk internal tools and go further for agents that touch customer data.

## What should the gateway log for every request?

Whatever credential you choose, log enough to answer "who called which model, for which agent, at what cost":

- Caller identity: key ID, client ID or workload identity
- End user, when the call is made on behalf of one
- Agent and agent version
- Model, provider and route taken, including fallbacks
- Input, output and cached tokens, and cost
- Policy decisions: allowed, redacted or blocked

Send these logs to the same place as your other security logs. For how gateways fit with agent gateways and control planes, see the [cost and gateway pillar](/blog/llm-gateway-vs-agent-gateway-vs-control-plane).

## How Opencontroller does this

Opencontroller uses OIDC SSO for console access, supports agent workload identity federation and delegated on-behalf-of identity, and exposes OpenAI-compatible endpoints through its LLM Gateway. The gateway attributes cost per agent and per version and applies routing, fallback, retry and caching in the same path. See the [LLM gateway comparison](/compare/llmgateway) and the [gateway benchmark](/llm-gateway-benchmark), or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Is it acceptable for an LLM gateway console to use local usernames and passwords?**
It should not be the normal path. Use SSO through the corporate identity provider and keep at most one or two break-glass local accounts, stored in a vault and monitored.

**Are virtual keys secure enough for production agents?**
They are acceptable when each key is scoped to one agent, has a budget and an expiry, and lives in a secrets manager. Short-lived tokens from the identity provider or workload identity are stronger.

**Do developers need separate gateway credentials for local testing?**
Yes. Give them personal, low-budget keys or tokens tied to their identity, never a copy of a production agent's credential.

## Key facts

An LLM gateway has two kinds of callers. People using its console should sign in through the corporate identity provider with SSO (OIDC or SAML), with roles from IdP groups and no local passwords. Applications and agents calling the model endpoint should use, from weakest to strongest: a shared master key (avoid), scoped virtual keys per agent with budgets and expiry, OAuth 2.0 client credentials tokens from the IdP (RFC 6749 section 4.4), or workload identity federation that exchanges a platform-issued token for short-lived credentials with no stored secret. When an agent acts for a user, pass the user's identity with token exchange (RFC 8693). Log caller identity, agent and version, key or client ID, model, tokens and cost on every request. Opencontroller by Lyzr supports OIDC SSO, agent workload identity federation and per-agent cost attribution in its LLM Gateway; pricing starts from $250K per deployment per year.

## Related

- [Should AI governance tools allow local logins or SSO only?](https://opencontroller.lyzr.com/blog/sso-only-access-for-ai-governance-tools.md)
- [Should AI agents use API keys or OAuth?](https://opencontroller.lyzr.com/blog/api-keys-vs-oauth-for-ai-agents.md)
- [How does workload identity federation work for AI agents?](https://opencontroller.lyzr.com/blog/workload-identity-federation-for-ai-agents.md)
- [More on Gateways and cost](https://opencontroller.lyzr.com/blog/cluster/gateways-and-cost)
