On October 2 the Wall Street Journal's Patrick Coffee reported that retailers are split on AI shopping agents, under the headline "Some Retailers Are Locking the Doors." QVC and HSN welcome them. Tapestry lets them browse Coach and Kate Spade but not buy. Kohl's stops them from buying. Amazon blocks them entirely, including Meta's new Muse agent.
Most of that story is about one step: the purchase. But before an agent can buy anything, it has to read the product page. That's the step a shopper's AI assistant takes every time they ask what to buy, and it's one anyone can check from the outside. Every big site publishes a robots.txt file that tells bots which pages they may read.
mapou pulled the file from 18 big US retailers and tested 11 AI agents against a product page on each. Only Amazon stops AI assistants from reading. Every one of the other 17 lets ChatGPT, Claude and Perplexity read a product page when a shopper asks about it.
The numbers
The bots fall into three jobs. Some collect pages to train AI models. Some build the index an assistant searches. And some fetch a page live, because a shopper just asked the assistant about that product. The table counts how many bots in each group a retailer allows.
| Retailer | Bots that train AI models | Bots that index for AI search | Bots that fetch for a shopper | Has rules for AI bots |
|---|---|---|---|---|
| Amazon | 0 of 5 | 0 of 2 | 0 of 4 | Yes |
| Best Buy | 5 of 5 | 2 of 2 | 4 of 4 | Yes |
| Chewy | 5 of 5 | 2 of 2 | 4 of 4 | No |
| Coach | 5 of 5 | 2 of 2 | 4 of 4 | Yes |
| Costco | 5 of 5 | 2 of 2 | 4 of 4 | No |
| CVS | 5 of 5 | 2 of 2 | 4 of 4 | Yes |
| eBay | 1 of 5 | 2 of 2 | 4 of 4 | Yes |
| Gap | 5 of 5 | 2 of 2 | 4 of 4 | Yes |
| HSN | 5 of 5 | 2 of 2 | 4 of 4 | No |
| Kate Spade | 5 of 5 | 2 of 2 | 4 of 4 | Yes |
| Kohl's | 5 of 5 | 2 of 2 | 4 of 4 | No |
| Neiman Marcus | 4 of 5 | 2 of 2 | 4 of 4 | Yes |
| QVC | 5 of 5 | 2 of 2 | 4 of 4 | Yes |
| Sephora | 5 of 5 | 2 of 2 | 4 of 4 | No |
| Target | 5 of 5 | 2 of 2 | 4 of 4 | No |
| Ulta | 5 of 5 | 2 of 2 | 4 of 4 | Yes |
| Walgreens | 5 of 5 | 2 of 2 | 4 of 4 | No |
| Walmart | 5 of 5 | 2 of 2 | 4 of 4 | No |
Train AI models: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, meta-externalagent. Index for AI search: OAI-SearchBot, Claude-SearchBot. Fetch for a shopper: ChatGPT-User, Claude-User, Perplexity-User, meta-externalfetcher. Counts what robots.txt permits, not what a site enforces.
Reading is open. Buying is where it closes.
Amazon is the only one that says no to reading. Training, search, live fetch, all of it, every agent named one by one. Googlebot still gets in. This matches the Journal's reporting exactly.
Blocked, and still cited. In mapou's September study of allergy medicine answers, amazon.com still showed up 53 times out of 2,609 citations (2%). Why is an open question. It could be an older index, copies of Amazon pages elsewhere, or a search layer that treats robots.txt differently. mapou hasn't tested which yet.
Tapestry wrote browse, not buy, into the file. Coach and Kate Spade name every major AI agent, let them read product pages, and shut them out of the shopping bag and checkout. Best Buy, which has announced a partnership with Muse, does something similar: assistants can read product pages but not the cart, pricing or availability endpoints.
Kohl's and Gap close checkout to every bot, AI or not, which is standard. Kohl's also has a bot wall in front of the whole site. Its file is silent on AI, but the wall does the work.
Some of the "blocks" are narrow. The Journal cites Billions Network, a firm that verifies humans and AI agents, saying Chewy and Neiman Marcus block certain agents. In their robots.txt files, Chewy blocks only Amazonbot, Amazon's own crawler. Neiman Marcus blocks only Meta's crawler. The big AI assistants can read both.
eBay's policy and its file say different things. The Journal reports that eBay barred agents from acting on its platform without permission in January. Its robots.txt blocks 4 of the 5 training crawlers but still lets every search crawler and shopper fetch in.
QVC means it. Its file allows OpenAI's and Perplexity's search crawlers by name.
8 of the 18 don't mention AI at all. Walmart, Target, Costco, Sephora and the rest have no AI rules, so the default applies and the file says come in.
What it means
mapou's read: "locking the doors" is mostly about the checkout, and agents completing purchases is still small. The Journal cites a survey where only 3% of US adults would trust an agent to buy for them. The much bigger thing happening today is AI assistants reading product pages and recommending what to buy. On that, the doors are open everywhere except Amazon. If ChatGPT can't read Amazon, it builds the answer from everywhere else.
What this doesn't show
robots.txt is a request, not a wall. Some agents ignore it, and sites like Kohl's block bots at the firewall whatever the file says. Kohl's, Neiman Marcus and Best Buy blocked even a plain request for their file, so those three were read through a browser. Agents like Muse that act inside a shopper's browser may not identify as bots at all, so robots.txt can't speak to them. This is 18 retailers on one day, checked against one product URL pattern each. It shows what each retailer asks for, not what any assistant actually does.
Want to know which sites AI can read for your brand?
mapou can run this check across the retailers that carry you, your own site and your competitors, and show which pages AI assistants actually read when shoppers ask about your category.
Monthly research newsletter
Get the next article by email.
Short findings like this one, plus the monthly panel report. Unsubscribe in one click.
We never share email addresses. One unsubscribe link in every send.