Cloudflare's web_fetch tool takes six request limits that the app sets at creation
Cloudflare's Agents SDK documents a paired web_search and web_fetch workflow with six request limits the app sets when it creates the tool. A dev.to write-up says the wrappers do not guarantee fetched content is safe or true.
The Engineer · Build desk
Cloudflare documents request controls. The wrappers limit requests but don't guarantee safe or true content. The tool's creator sets the policy. The app or agent inspects results before fetching. Users see chosen URLs and can remove one. Apps open to any domain decide how access is logged.
- capability Cloudflare Controls for allowed and blocked hosts, response bytes, timeouts, redirects and characters per window, claim 8
- constraint The search and fetch wrappers Help choose results, limit requests and show provenance, but don't guarantee content is safe or true, claim 4
- decision Whoever creates the tool The post says to set the request policy at creation, the narrowest that fits the product, claim 9
- decision Application or agent The post says to inspect each search result's title, host and snippet before fetching, claim 21
- exposure Users The post says to show them the chosen URLs and let them remove a result, claim 15
- decision Apps open to arbitrary domains The post says to decide explicitly how that access is mediated and logged, claim 13
| Who | How | Kind | Claim |
|---|---|---|---|
| Cloudflare | Controls for allowed and blocked hosts, response bytes, timeouts, redirects and characters per window | capability | 8 |
| The search and fetch wrappers | Help choose results, limit requests and show provenance, but don't guarantee content is safe or true | constraint | 4 |
| Whoever creates the tool | The post says to set the request policy at creation, the narrowest that fits the product | decision | 9 |
| Application or agent | The post says to inspect each search result's title, host and snippet before fetching | decision | 21 |
| Users | The post says to show them the chosen URLs and let them remove a result | exposure | 15 |
| Apps open to arbitrary domains | The post says to decide explicitly how that access is mediated and logged | decision | 13 |
What happened
- Search returns a short numbered set of titles, URLs and descriptions, which the post treats as candidate sources and not proof that a result is relevant or authoritative.
- web_fetch converts HTML, PDF and supported Office documents to readable text, requests Markdown from sites that offer it, and formats JSON.
- Each fetch result reports the final URL, HTTP status, content type and character counts, so an app can say what it read and whether the response succeeded.
- The write-up recommends the narrowest policy that fits: an assistant that reads your own docs and a few vendor domains does not need to reach every public host.
Why it matters
- exposure An allowlist that checks only the URL the model requested leaves redirects open. The post says to check redirect destinations too and to keep private or local hosts unreachable without a reviewed reason.
- constraint The six limits bound what gets requested. None of them governs how returned text enters the prompt, so keeping it apart from trusted instructions is application code.
- cost The app team builds the provenance layer: a way for users to remove a chosen URL, and a log of why each result was selected, with the original URL preserved.
- decision Letting users name arbitrary domains is a product decision. The post says to settle explicitly how that access is mediated and logged instead of inheriting a permissive default.
The limits are set once, at tool creation. According to a dev.to write-up of the docs, Cloudflare's documentation covers which hosts are allowed, which are blocked, how many bytes a response may carry, timeouts, redirects, and how many characters come back in a single window [8][23]. The write-up says to set the request policy when the tool is created [9]. It also says to cap bytes and time, so that a response that is slow or unusually large cannot tie up resources without bound [12]. The excerpt does not list a default for any of the six.
The workflow the post describes is sequential:
1. The model states the question it is trying to answer, then searches with a modest result limit [20]. 2. The app or agent inspects the title, host and snippet of each result before fetching anything [21]. A result can be commentary, an obsolete page or a copy that omits important conditions [18]. 3. The selected pages are fetched under the limits set at creation [9]. 4. A further window is requested by offset only if the first window shows more text is needed. The loop stops when the evidence is enough [17]. 5. The assistant shows the source URLs and statuses [22]. If a page redirected, it keeps the final URL the fetch step returned, because that is the address whose content the model actually received [16].
Step 5 is cheap because the fetch result already carries the fields. The final URL, HTTP status, content type and character counts come back with the text [7].
The sequence has an exit for weak evidence. If the first results are poor, the post says to revise the query or say the evidence is thin, and to widen access only with a reason [19].
Search quality is the part the SDK cannot supply. The post treats a search result as a candidate source and steers the app toward first-party material: the vendor's documentation when the question is how a product behaves, the primary announcement when it is release details, and the vendor's own policy page when it is rules [18]. That ranking is application logic. So is the choice of which results the model may fetch.
This account rests on one dev.to write-up. Option names and defaults should be confirmed in Cloudflare's reference before anyone builds on them.
What to watch
- Whether Cloudflare's reference docs state default values for the six web_fetch controls, which the dev.to excerpt does not.
- Whether redirect hops are checked against the allowed-hosts list or only the first requested URL.
- Whether the Agents SDK adds any built-in separation between fetched text and trusted instructions, or leaves it to app code.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence55
- Adoption
- Insufficient
- Hype gap−10
- Incentives
- Insufficient
- Confidence55
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
Cloudflare's Agents SDK now documents a paired web_search and web_fetch workflow.
- [2]
Search can return a short numbered set of titles, URLs, and descriptions, which are structured result descriptions and not proof that every result is relevant or authoritative.
- [3]
web_fetch can read selected pages and return bounded text with response details.
- [4]
The wrappers help with the work of choosing results, limiting each request and presenting provenance, but they do not guarantee that retrieved content is safe or true.
- [5]
Cloudflare's web_fetch tool can convert HTML, PDF, and supported Office documents into readable text, request Markdown from sites that support it, and format JSON.
- [6]
Long pages are returned in windows that can be continued with an offset.
- [7]
Fetch results include details such as final URL, HTTP status, content type, and character counts.
- [8]
Cloudflare documents controls for allowed hosts, blocked hosts, response bytes, timeouts, redirects, and the number of characters returned in one window.
- [9]
The post says to set the request policy when you create the tool, using the narrowest policy that fits the product.
- [10]
A research assistant that should read your own documentation and a few approved vendor domains does not need permission to fetch every public host on the internet.
- [11]
Allowlisting is a policy boundary, not a substitute for validating the task; the post says to check redirect destinations as part of the effective policy and to keep private or local hosts unavailable unless the product has a specific, reviewed reason to reach them.
- [12]
The post says to enforce byte and timeout limits so a slow or unusually large response cannot consume unbounded resources.
- [13]
If users can request arbitrary domains, the post says to decide explicitly how that access is mediated and logged rather than inheriting a permissive default by accident.
- [14]
The post says to keep fetched material separate from trusted system instructions.
- [15]
The post says to show the chosen URLs and let the user remove a result when the assistant's interpretation of the question is off; for automated flows, record why each result was selected and preserve the original URL.
- [16]
If the source redirects, the post says to keep the final URL returned by the fetch step, because that is the address whose content the model actually received.
- [17]
The post says to use the returned offset to request another portion only when the first window shows that more text is needed, and to stop when there is enough evidence.
- [18]
A search result can point to commentary, an obsolete page, or a copy that omits important conditions, so the post says to treat it as a candidate source; it prefers vendor documentation for product behavior, a primary announcement for release details, and a first-party policy page for rules.
- [19]
If the initial results are weak, the post says to revise the query or say that the evidence is thin rather than broadening access without a reason.
- [20]
The post says to ask the model to state the question it is trying to answer, then issue a search with a modest result limit appropriate to the task.
- [21]
Before fetching, the post says to have the application or agent inspect the title, host, and snippet of each search result.
- [22]
The post says the assistant should show the resulting source URLs and statuses to the user.
- [23]
The controls Cloudflare documents for web_fetch number six: allowed hosts, blocked hosts, response bytes, timeouts, redirects, and characters per window.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- dev.toCloudflare web search and bounded fetch for SaaS agent research workflows
1 article · October 14, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Topics
- AI Agent SecurityFollow
- AI Agent ToolingFollow
- Web retrieval for LLMsFollow