Why Muse reads your site like a customer, not a crawler.

For twenty years, "can machines read my store?" meant one thing: can a crawler fetch the HTML and index it. Browser agents such as Meta's Muse have changed the question. They do not fetch your page. They visit it, the way a person would, and they leave the way a person would when it gets in their way.

What is the difference between a crawler and a browser agent?

A crawler downloads a page and files it; a browser agent uses the page to get something done. That one difference changes almost everything about what "readable" means.

A search crawler requests a URL, reads the markup, follows the links and moves on. It never clicks "add to cart". It does not care whether your size selector works, because it never touches it. A browser agent is given a goal by a shopper, such as "find me a waterproof jacket under $200 in a medium and tell me when it would arrive". It opens a real browser, loads the page with scripts running, works out what is on the screen, and takes actions toward that goal. Every step a customer takes, it takes.

The last row is the one to remember. When a crawler fails, you lose a listing. When a browser agent fails, it tells the shopper "I couldn't complete this on that site" and tries the next store.

What does a browser agent see when it opens your product page?

A browser agent sees roughly what a first-time visitor with a screen reader and a lot of patience would see: the rendered page, the names of the controls, and whatever is covering them. It does not see your intentions.

Browser agents generally work from some mix of the rendered DOM, the accessibility tree and screenshots of the page. In practice that means three things decide whether your page makes sense to one:

• Is the information on the page as text? Price, stock, delivery estimate, return window, materials, dimensions. If it is inside an image, a video or a tab that only loads on click, it may as well not exist.
• Do the controls say what they are? A size selector built from real buttons or a select element with labels is obvious. A row of unlabeled divs with click handlers is a guess. The accessibility work you may have put off is now commercial work.
• Is anything in the way? Cookie banners, newsletter modals, location pickers, chat widgets and app-install prompts all land on top of the page in the first three seconds. A person sighs and closes them. An agent has to work out that they are closable at all.

Where does a browser agent give up?

A browser agent gives up where a hurried customer would, only sooner. When we test stores with browser agents as part of an Audit, the same blockers come up again and again. In rough order of how often they end the session:

1. Bot walls and challenges. A CAPTCHA or an "are you human?" interstitial ends the visit before it starts. Rules written to stop scrapers rarely distinguish a scraper from an agent shopping on behalf of a paying customer.
2. Stacked pop-ups. One consent banner is manageable. A consent banner, then a discount modal, then a country selector is a maze. Close buttons that are tiny icons with no accessible name make it worse.
3. Prices that arrive late. If the price is injected by a script after the page has settled, or shows "from $..." until a variant is chosen, the agent may read the wrong number or none.
4. Variant selectors that are not real controls. Color swatches with no text, sizes that look available but are not, options that reset each other. The agent picks a medium, the page quietly switches to out of stock, and the add-to-cart button goes gray with no explanation.
5. Hidden costs and dates. Shipping cost and delivery time that only appear on the last step of checkout. The shopper asked "when would it arrive?" and the agent cannot answer without nearly buying.
6. Forced accounts. A checkout that demands registration before showing a total is a dead end. Guest checkout is the agent-friendly default.
7. Infinite scroll and lazy everything. Category pages that never end, and product details that load only when scrolled into view, make it hard to know when the page has been fully read.

None of this is exotic. The difference is the penalty. A human customer tolerates friction because they already want your product. An agent was asked for a waterproof jacket, not for yours.

If agents can use feeds, why does the live site still matter?

The live site matters because different agents read different versions of your product, and the browser agent's version is the one your customers actually see. ChatGPT Shopping and Gemini work largely from structured product feeds. Claude and Perplexity lean on what reviewers and forums say. Muse reads the live store. Those are three descriptions of the same product, and they often disagree on price, stock or delivery.

When they disagree, the safest thing for an agent to do is trust none of them and recommend someone else. So the live page is not just one channel among three. It is the reference copy the other two get checked against. We go through all three sources in llms.txt, schema and feeds: what AI shopping agents actually read.

How do you test your store the way a browser agent would?

You can find most browser-agent blockers in ten minutes by shopping your own store with your hands tied. No special tools are needed for a first pass.

1. Open a product page in a private window. Count everything that appears on top of the page in the first ten seconds. Each one is a hurdle.
2. Put the mouse away. Using only Tab, Enter, Space and the arrow keys, choose a size and color, add to cart and reach the payment step. Anywhere you get stuck, an agent may too.
3. Turn on a screen reader, or open the accessibility tree in your browser's developer tools. Are the variant options, the price and the add-to-cart button named sensibly? "Button" is not a name.
4. Throttle the connection and reload. Watch what the page says about price and stock before the scripts finish. That early state is sometimes what gets read.
5. Find the shipping cost, delivery estimate and return window without starting checkout. If you cannot, neither can the agent.
6. Check your bot rules. Ask whoever manages your CDN or firewall what happens to automated browsers. Many stores block them by default and have never reviewed the setting.
7. Ask an agent to do it. Give a browser agent a plain task on your own store, such as finding a product in a given size and reporting the delivered price. Watch where it hesitates.

What should you fix first?

Fix access first, then legibility, then the checkout. There is no point perfecting a size selector that an agent never reaches.

• Access: review bot rules so that agents acting for shoppers are not treated as scrapers. Cut the pop-up stack down to the legally required consent banner, with a clearly labeled button.
• Legibility: put price, stock, delivery and returns on the product page as text, present in the first render. Use real form controls with labels for variants. Make sure the visible price matches your feed and your Product schema.
• Checkout: offer guest checkout, show the delivered total early, and remove dead ends such as forced app downloads.

This is the core of our Fix service: we work in your codebase with your developers, and nothing ships without their review. If you want to know where you stand first, run the free check on one product URL, or see the full sequence in our checklist for getting recommended.

Written by the team at Recommended by Agents. Published August 21, 2026. Spotted something out of date? Tell us and we will fix it.