Fahad Azhar 開發網店檢查工具，評估 AI 智能體能否讀取網店
AI agents are your new customers. I built a checker to see which stores they can actually read
Fahad Azhar 開發了一款免費檢查工具，模擬 AI 智能體讀取網店的方式抓取商店頁面，並按 8 項檢查評分。檢查涵蓋 robots.txt 是否允許 AI 搜尋爬蟲、政策頁及商品 Product JSON-LD；工具會檢查 OAI-SearchBot、PerplexityBot 和 Googlebot。用戶輸入網店網址後毋須註冊，即可查看評分及各項未通過檢查的具體修正方法。
A shopper asks ChatGPT for "a waterproof hiking jacket under $150 that ships to Dubai". The answer gets built from whatever the agent could read in the few seconds it spent on your store. If it couldn't read you, you aren't in the answer. Nobody tells you that you lost the sale.
I kept getting the same question from store owners: "Are we visible to these things or not?" Guessing wasn't good enough, so I built a checker that fetches a store the way an agent does and scores what it finds.
What an agent needs from a store
Once you strip it down, it comes to 8 checks. They don't all count the same:
| Check | Weight |
|---|---|
| Store is open (no password page) | 15 |
| AI search crawlers allowed in robots.txt | 20 |
| Refund, privacy and terms policies published | 20 |
| Product pages carry Product JSON-LD (name, image, price, currency) | 20 |
| Brand, stock, barcode, shipping and returns in that data | 10 |
/.well-known/ucp answers (Universal Commerce Protocol) |
10 |
| Sitemap published | 5 |
| Descriptions answer real questions | 5 |
llms.txt / agents.md get reported but weigh 0. Nobody has shown agents rely on them yet, and I didn't want the score to reward hype.
The one that surprises people: robots.txt
Plenty of stores added blanket AI blocks in 2024 to stop training scrapers, and they took out the search bots in the same line. Training bots and shopping-answer bots are different user agents:
# Blocks training only. Fine.
User-agent: GPTBot
Disallow: /
# Blocks ChatGPT search answers. Probably not what you meant.
User-agent: OAI-SearchBot
Disallow: /
The checker looks at 3 agents: OAI-SearchBot, PerplexityBot and Googlebot. Training blocks get a note in the report but don't lower the score.
How it fetches without becoming a security hole
A public "enter any URL" tool turns into an SSRF tool unless you lock it down:
- https only, no custom ports, no raw IPs
- DNS is resolved and checked for a public address before every request
- redirects are followed by hand, and each hop is validated again
- 7 s timeout and a 1.5 MB cap per response
- 8 checks per IP per 10 minutes, results cached for 10 minutes
It samples up to 3 product pages, not one. A single product page lies too often: one product has rich data, the next has nothing.
The bug that taught me the most
On a first run, a big store scored badly on "product data". The JSON-LD was there. Shopify had answered my second request with a 429 (rate limited), and my code read the empty body as "no structured data".
Now a 429 gets polite retries with backoff. If the page still won't load, the report says "didn't load" as a warning instead of failing the store. A checker that confidently tells you wrong things is worse than no checker at all.
What I see most often
- Policies exist, but the refund page is an empty template
- Product JSON-LD without
priceCurrency, so agents can't compare prices - Theme apps injecting a second, conflicting Product block
- Robots rules copied from a blog post in 2024 that block
OAI-SearchBot
Each of these takes under an hour to fix. The problem is that nobody checks.
Try it
The checker is free and needs no signup: paste a store URL and you get a score with the exact fix for each failed check.
👉 Check if AI agents can read your store
If you build stores for clients, run it on one of yours and tell me in the comments what it caught. I'm still tuning the weights.
來源：DEV Community · dev.to