API 返回 200，Agent 仍可能收到已退化的數據

Here is a failure no dashboard will show you.

An API your agent calls every morning answers 200 OK. The JSON schema has not changed. Every field is still there. But 42 of the 50 products now have "price": null, and updated_at has not moved in months.

Your uptime check is green. Your schema diff is green. A person looking at this data would notice the empty column in two seconds. Your agent does not stop: it writes a confident report from what it received, and nobody finds out until a customer does.

Status codes say the server answered. Schemas say the shape is valid. Neither says the data your agent depends on is still there.

Test the data, from the consumer's side

So I built a small tool that treats this as a testing problem. A contract says what your agent needs from a source, and a check tells you when the source stops delivering it:

• Coverage: at least 90% of items carry a usable price. null, "", [] and {} do not count.
• Freshness: the newest updated_at is less than 3 days old.
• Volume: the list has at least 25 items, so an empty array is a failure and not a success.
• Distinct values: the column is not stuck on one value, like a price of 0 everywhere.

You do not have to write that by hand, and you do not need the provider's cooperation. You put a local proxy between your agent and the source, and it learns the contract from the real calls. It stores shapes, never values.

Try it in a minute

git clone https://github.com/driftprobe/driftprobe && cd driftprobe && npm install

# A demo API, and the proxy in front of it
node examples/demo-api.mjs &
node bin/driftprobe.mjs record --upstream http://127.0.0.1:4010 --port 8787 &

# Your agent calls the proxy instead of the API
curl -s http://127.0.0.1:8787/products > /dev/null
curl -s http://127.0.0.1:8787/products > /dev/null

# Learn a contract from those calls
node bin/driftprobe.mjs learn --observed .driftprobe/observed.jsonl --out-dir contracts

# The source degrades, and still answers 200 OK
curl -s -X POST http://127.0.0.1:4010/__break
curl -s -o /dev/null -w "HTTP %{http_code}\n" http://127.0.0.1:8787/products

Then the check:

$ node bin/driftprobe.mjs check contracts/*.yaml
GET /products
  x price: only 16% of 50 items carry a value (contract: at least 90%)
  x updated_at: newest value is 289 days old (contract: 3d)

1 contract(s): 0 ok, 1 broken, 0 could not be checked

Exit code 1, so it fails your CI. It also works on MCP tools, including over server-sent events.

How often does this really happen?

I do not know yet, and I could not find anyone who has measured it. So I am measuring it.

The official MCP registry listed 53,434 servers on 31 October 2026. I took a sample of 2,000 remote endpoints that do not declare a required credential, in a stable random order, at most 5 per domain:

• 655 asked for credentials anyway
• 1,115 answered with their tool list
• 770 declared at least one read only tool
• 415 returned data from a read only call with neutral arguments

From those, I picked 100 that return real lists, across 98 domains. I call one read only tool on each, twice a day, for 30 days, and record shapes only. I will publish the results whatever they show, including if almost nothing breaks.

To avoid crying wolf, the analysis only counts a break when the same rule fails on consecutive observations, and I tested the method on simulated servers where nothing changes: at most 2% get flagged.

What it does not do

It does not know if a value is true, only if it is there, fresh and varied. It learns what your agent receives: narrowing that to what your agent actually reads is manual for now (--fields name,price).

If you run LangGraph, quietfail watches behaviour from inside the agent, which is complementary. ToolBench grades how well MCP tools are defined, and FlareCanary watches tool schemas. This one sits outside the agent and looks at the data that comes back.

The code is MIT: github.com/driftprobe/driftprobe. If one of your sources has failed this way, I would like to hear how you found out.