AI 編碼工具聲稱修好問題，不能代替獨立驗證

"Fixed. The signup form now rejects bad emails."

You've probably read a line like that from an AI coding tool. Maybe it was true. Maybe the form still let bob@ straight through.

The problem usually isn't the model or the prompt. Nobody made it prove the claim.

In part 3 I gave Claude a way to see the code clearly. This post is about the next step: getting it to show its work before you believe it.

The best example I've found of this isn't mine. It's Cloudflare's.

What Cloudflare open-sourced

In June 2026, Cloudflare published the security audit skill that seeded their vulnerability discovery harness. That harness grew into a multi-stage system they run across their whole fleet. They describe it in their blog post.

A skill is a reusable set of instructions you give an AI coding tool, so it follows the same method every time.

It's MIT licensed. When I checked, it had about 53,000 stars and 3,400 forks on GitHub.

The six stages

Each stage has one job:

Map the code. Learn what the project is and where the risky parts live.
Hunt. Separate hunters look for bugs, each working alone.
Try to disprove. Every suspected bug goes to a fresh reviewer whose job is to knock it down.
Sort. Each one becomes confirmed, needs validation or rejected.
Check again. New agents verify the final list.
Report.

Stages 3 to 5 are the interesting part. Half the system exists to doubt the other half.

The rule worth stealing

From the repo's design principles:

Adversarial validation
"The agent that checks a finding is never the agent that found it."

That one line works far outside security.

Why not just ask "are you sure?"

That's the obvious move. Ask the same AI to double-check its own work.

It rarely works well, for a simple reason. The agent that made the claim already believes it. It wrote the reasoning, so the reasoning looks right to it.

Approach	What really happens
Ask the same agent "are you sure?"	It rereads its own logic and agrees with itself
Ask it to "test the fix"	It often writes tests that match what it built, not what you wanted
A fresh checker told to disprove it	It starts with no stake in the answer, and its job is to find the hole

Think of a student marking their own exam. They know what they meant, so they read every answer generously. A second marker only sees what's on the page.

Three verdicts, not two

Most of us treat AI output as pass or fail. Cloudflare's skill uses three outcomes:

Confirmed: the checker tried to break it and couldn't.
Needs validation: not proven yet. The skill writes down the exact fact that's still missing.
Rejected: the checker found the hole.

The middle one matters most. Without it, "I couldn't quite tell" quietly turns into "probably fine". The repo is strict here: "Only confirm established boundary failures."

Another principle stops claims from getting inflated: "Severity requires impact." A bug only counts as serious if you can say what actually goes wrong.

Run it more than once

This line from the repo is easy to skim past:

"In our test runs, a single run found roughly half of the vulnerabilities that repeated runs found in total."

AI runs don't come out the same twice. Each one notices different things. So one clean run doesn't mean there's nothing left to find.

Try it in your own AI coding work

You don't need a security system to use these ideas. Here's the same pattern for an everyday task.

Say your AI tool says it fixed a dark mode bug, where the settings page stayed white.

1. Start a fresh session. Not the one that wrote the fix. A new chat, with no memory of the reasoning.

2. Hand it the claim and ask it to disprove it:

Another agent claims this change fixes the bug where the settings
page stays white in dark mode. Your job is to prove that claim wrong.

Look at the change and the code around it. Find any case where the
page could still show the wrong colours.

End with one verdict:
- CONFIRMED: you tried and could not break it. Say what you checked.
- NEEDS VALIDATION: you can't tell. Name the exact fact you're missing.
- REJECTED: you found a case where it still fails. Show it.

3. Treat "needs validation" as open. Check the missing fact yourself, or give the checker what it needs and run it again.

4. For bigger changes, run the checker twice. If two runs come back with different holes, you've just learned something.

Keep the checker separate
If the checker can see the original agent's reasoning, it tends to adopt it. Give it the claim and the code, not the conversation.

Want the real thing?

If you do want a security audit, the repo has an install line:

npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit

Read the README first. It explains each stage and the format of the findings.

Check it worked

You'll know the pattern is working when your checker sometimes says REJECTED or NEEDS VALIDATION.

If it confirms everything, it isn't really trying. Make the prompt more hostile, or hand it a claim you already know is wrong and see if it catches it.

Credit where it's due

The design here is Cloudflare's: the security audit skill and their blog post on the harness. I'm pointing at it because the core idea travels well beyond security.

Next in Claude Maxxing

Part 1 gave Claude a desk. Part 2 gave it a memory. Part 3 gave it eyes. Part 4 is about not taking its word for it.

If an AI has ever told you a bug was gone and it wasn't, I'd like to hear how you caught it. Get in touch.