Skip to content

Product1 publisher3 min readPublished

Amazon shuts OpenAI's shopping agent out of its own product pages

Amazon's robots.txt now disallows ChatGPT-User and OAI-SearchBot, and in ZDNET's tests the shopping agent sent a buyer to Walmart, eBay and a retailer called Soaplicity instead of Amazon listings.

The Product Desk · Product desk

Photograph accompanying Amazon shuts OpenAI's shopping agent out of its own product pages
Photo: zdnet.com

What happened

  • An earlier Disallow line in the same file already covered GPTBot, the crawler OpenAI uses to train its models on site content, according to analyst Juozas Kaziukėnas.
  • OpenAI shipped its shopping research tool in late November as a personal shopper that scours retailers for a product matching a user's features and price range.
  • Amazon's advertising business takes in around $56 billion a year, revenue that depends on people browsing its pages, as reported by Modern Retail.

Compiled by The Product DeskSomething wrong?How this is made

Why it matters

  • constraint The new rules hit live fetching, not just training, so an agent that promises a current Amazon price has to answer for a cached one every time a shopper asks.
  • decision Anyone building agent commerce now chooses between fetching open web pages and negotiating feeds or affiliate links retailer by retailer, and the second route puts a product roadmap behind a business development queue.
  • exposure Amazon enforced against Perplexity through counsel, so the company shipping a buying agent takes on a legal exposure that no robots.txt parser handles.
  • contradiction Kaziukėnas attributes the Amazon links that still surface to archived crawl data, and a user reading a chat reply cannot tell a live price from a stale one.

ZDNET's reporter asked ChatGPT for the best MagSafe chargers for iPhones at Amazon. He got a list of chargers with no links to Amazon product pages, and when he asked the agent to add Amazon links, the links it produced went elsewhere [8]. A second request, for an item he had bought on Amazon before, came back with an offer to list alternative sellers: Walmart, eBay, and a retailer called Soaplicity [9].

Three OpenAI user agents now sit in that file with Disallow next to them [5]. One of the three, GPTBot, is the training crawler [4]. The other two run at the moment a user asks for something, and Kaziukėnas said the new rules stop ChatGPT from browsing Amazon's site when you ask a question or when you run a web search [3]. A training block shapes a future model. A live-fetch block changes what the product does this afternoon.

A robots.txt file is instructions to crawlers [12]. Software that ignores it keeps working, and Amazon's answer in that case has been legal. The cease-and-desist it sent Perplexity in early November demanded the company block its Comet browser from buying items at Amazon on users' behalf [13], and the letter says transparency is critical because it helps a service provider limit conduct that degrades the shopping experience and creates security risks for customers [14]. It also asserts that an AI like Comet may not choose the best price, delivery method, or recommendations that Amazon itself would provide [15].

Amazon's advertising take is around $56 billion a year, and that revenue depends on people browsing the site, ZDNET reports, citing Modern Retail [16]. Spread across the year, that is roughly $153 million a day of inventory sold against human page views [17]. An agent that reads a listing and reports back in chat does not deliver the page view the advertiser bought.

The blocking is not airtight. ZDNET's second test did produce an Amazon link once the reporter pressed for one [10], and Kaziukėnas said ChatGPT still retains Amazon data from archived web crawls [11]. ZDNET discloses that its parent, Ziff Davis, filed an April 2025 lawsuit against OpenAI alleging infringement of its copyrights in training and operating OpenAI's AI systems [18].

For a team shipping a buying flow, the sort runs on two questions. Does the retailer's robots.txt name your agent, and do you have a channel that does not require fetching the page, such as a product feed or an affiliate link. Named with a channel is a business development problem. Named without one is the quadrant where the demo passes on cached data and production quotes a price from a crawl nobody dated. Unnamed without a channel is one commit away from the first case. Twenty product queries through your own agent, counting how many links land on a retailer you have no relationship with, gives you that number before a customer finds it.

What to watch

  • Whether Amazon adds other agent user agents to the Disallow list, or opens a paid channel for agents instead.
  • Whether OpenAI drops Amazon from shopping research results or keeps serving answers built on archived crawl data.
  • Whether Perplexity complies with the cease-and-desist over Comet's purchases at Amazon or contests it.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories