---
title: How do you keep LLM and agent traffic off the public internet on AWS, Azure and Google Cloud?
url: https://opencontroller.lyzr.com/blog/keep-llm-traffic-private-across-clouds
published: 2026-11-13
updated: 2026-11-13
author: Lyzr
---

# How do you keep LLM and agent traffic off the public internet on AWS, Azure and Google Cloud?

**Use each cloud's private connectivity for model APIs: AWS PrivateLink interface endpoints for Amazon Bedrock, private endpoints with public network access disabled for Microsoft Foundry, and Private Service Connect or Private Google Access inside a VPC Service Controls perimeter for Vertex AI. Then control the other half: agent egress to tools, grounding services and gateways, which private model endpoints do not cover.**

Security reviews usually ask two questions: does the model call leave our network, and does anything the agent does leave it? The first has a clean answer in each cloud. The second needs more work.

## What are the private options in each cloud?

| | AWS (Amazon Bedrock) | Azure (Microsoft Foundry) | Google Cloud (Vertex AI) |
|---|---|---|---|
| Private inbound path | Interface VPC endpoints (AWS PrivateLink) | Private endpoint on the Foundry resource (target sub-resource "account") | Private Service Connect endpoint for Google APIs, or Private Google Access |
| Endpoints or names | `bedrock`, `bedrock-runtime`, `bedrock-mantle`, `bedrock-agent`, `bedrock-agent-runtime`, plus FIPS variants in some Regions | Private DNS zone for the `privatelink` subdomain | A /32 internal IP mapped to a DNS name such as `aiplatform-genai1.p.googleapis.com` |
| Turn off public access | Deny calls not from your VPC endpoint with IAM conditions | Set public network access to Disabled | Use a VPC Service Controls perimeter |
| Restrict what goes through | Endpoint policy (an IAM resource policy on the endpoint) | Network rules and RBAC | Perimeter and ingress or egress rules |
| On-premises access | Direct Connect or VPN into the VPC | ExpressRoute or VPN into the VNet | Cloud Interconnect, Cross-Cloud Interconnect, HA VPN over Interconnect or SD-WAN |

Notes from the vendor docs:

- **Bedrock.** With private DNS enabled, calls to `bedrock-runtime.{region}.amazonaws.com` route through the endpoint with no code change. Without it, set the VPC endpoint URL in the SDK. The default endpoint policy allows full Bedrock access, so attach a custom one that allows only the actions you use, such as `bedrock:InvokeModel`. The `bedrock-mantle` endpoint covers the OpenAI-compatible API.
- **Foundry.** The private endpoint must be in the same region and subscription as the VNet, and only Approved endpoints carry traffic. Removing private endpoints does not make the resource public; that is a separate networking setting.
- **Vertex AI.** For on-premises access, advertise the Private Google Access range (`private.googleapis.com` 199.36.153.8/30 or `restricted.googleapis.com` 199.36.153.4/30) or your endpoint IP from Cloud Router, and resolve `REGION-aiplatform.googleapis.com` to the private address.

## What do private model endpoints not cover?

A private endpoint protects the call into the model service. Agents also make outbound calls, and those follow different paths:

1. **Tool and MCP calls.** An agent calling a SaaS API or a remote MCP server leaves your network unless you route it through a proxy or private link.
2. **Platform-managed grounding.** Microsoft documents that Bing Grounding, Websearch and SharePoint Grounding use public endpoints even in network-isolated Foundry setups.
3. **Hosted agent egress.** Foundry Agent Service needs outbound VNet injection: a subnet delegated to `Microsoft.App/environments`, /27 or larger. It cannot be added to an existing deployment; you redeploy. Microsoft recommends Azure Firewall with FQDN allowlists for egress.
4. **Gateways.** Microsoft notes AI gateways created in the Foundry portal are public by default and need separate isolation.
5. **Direct provider APIs.** Calls to a model provider's public API go over the internet unless a cloud marketplace or private connectivity option exists for that provider.

## A checklist for a private agent estate

1. List every model endpoint each agent calls, by cloud and Region.
2. Create private endpoints for each, and turn on private DNS.
3. Disable public access, or deny it with policy, for every model resource.
4. Attach least-privilege endpoint policies.
5. Put agent egress behind a firewall or proxy with an FQDN allowlist.
6. Route model traffic through one gateway that is itself private; see the [self-hosted LLM gateway network checklist](/blog/self-hosted-llm-gateway-network-checklist).
7. Document which services stay public, such as web grounding, and record the risk acceptance.
8. Test from a host with no internet route. Every agent call should still succeed.

Private networking is not the same as data residency. A private path can still reach a model that processes data in another Region. See [how to enforce data residency for LLM and agent traffic](/blog/how-to-enforce-data-residency-for-llm-and-agent-traffic).

## How Opencontroller does this

Opencontroller can run on Lyzr SaaS or in customer-controlled infrastructure. Its LLM Gateway and Agent Gateway put control in the path, so you can allow, route, redact or block traffic, with routing, fallback and retry, and cost attribution per agent and per version. Auto-discovery across AWS, Azure and Google Cloud finds agents, models and API keys, so you can see which endpoints exist before you lock them down. Read the [gateway comparison](/blog/llm-gateway-vs-agent-gateway-vs-control-plane), the [cross-platform governance guide](/blog/how-to-govern-ai-agents-across-platforms), or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Does AWS PrivateLink work for Bedrock Agents?**
Yes. Bedrock offers `bedrock-agent` and `bedrock-agent-runtime` interface endpoints alongside `bedrock` and `bedrock-runtime`.

**Is Foundry web grounding private if I use a private endpoint?**
No. Microsoft documents that Bing Grounding, Websearch and SharePoint Grounding use public endpoints even in isolated environments.

**Does a private endpoint guarantee data residency?**
No. It controls the network path, not where the model processes data. Choose Regions and routing separately.

## Key facts

Private model access: Amazon Bedrock supports AWS PrivateLink interface endpoints (bedrock, bedrock-runtime, bedrock-mantle, bedrock-agent, bedrock-agent-runtime, FIPS variants) with endpoint policies; Microsoft Foundry supports private endpoints (target sub-resource "account") with public network access disabled, and Agent Service outbound VNet injection via a subnet delegated to Microsoft.App/environments (/27 or larger); Vertex AI supports Private Service Connect endpoints, Private Google Access (private.googleapis.com 199.36.153.8/30, restricted.googleapis.com 199.36.153.4/30) and VPC Service Controls. Bing Grounding, Websearch and SharePoint Grounding use public endpoints even in isolated Foundry setups. Opencontroller (by Lyzr) runs on Lyzr SaaS or customer infrastructure with an LLM Gateway and Agent Gateway in the path. Pricing starts from $250K per deployment per year.

## Related

- [What network setup does a self-hosted LLM gateway need?](https://opencontroller.lyzr.com/blog/self-hosted-llm-gateway-network-checklist.md)
- [How do you enforce data residency for LLM and AI agent traffic?](https://opencontroller.lyzr.com/blog/how-to-enforce-data-residency-for-llm-and-agent-traffic.md)
- [How do you manage IP allow-lists for an LLM gateway without infrastructure drift?](https://opencontroller.lyzr.com/blog/manage-llm-gateway-ip-allowlists-without-drift.md)
- [More on Platform Guides](https://opencontroller.lyzr.com/blog/cluster/platform-guides)
