AI 智能體説「已完成」：它實際驗證了甚麼？

“Fixed the bug. All tests pass.”

Great.

Which tests? Did they run? Did they cover the bug—or did the agent change the test until it passed?

A confident completion message is easy to read as proof. But before you merge the changes, you need to understand what actually happened.

Here’s a simple workflow for checking an AI agent’s work without manually repeating every step.

1. Define “done” before it starts

“Fix the checkout bug” leaves too much open to interpretation.

Give the agent an observable outcome:

When a discount exceeds the order subtotal, the total should be zero. Add a regression test. Preserve valid discount behavior.

Now you have something concrete to verify.

For a UI task, describe the interaction. For an API task, specify the response and side effects.

Clear acceptance criteria make both implementation and verification easier.

2. Ask for evidence in the completion report

A useful report should include:

• What changed.
• Which commands ran.
• Whether they passed or failed.
• What could not be checked.

Compare these two reports:

“Everything works.”

And:

“Added a regression test for excessive discounts. The checkout test suite passed. Browser checkout was not tested because payment credentials were unavailable.”

The second report tells you what you can rely on and what still needs attention.

When your tool exposes execution logs, inspect them. A command mentioned in a summary is still a claim until you can confirm its execution.

3. Make sure the test catches the original bug

A passing test is useful only if it checks the behavior you care about.

Suppose the task is to prevent negative totals.

This assertion tells you almost nothing:

expect(result).toBeDefined();

This one checks the requirement:

expect(calculateTotal(50, 80)).toBe(0);

Where practical, confirm that the regression test fails against the original implementation and passes after the fix.

Then check nearby behavior:

expect(calculateTotal(50, 20)).toBe(30);

The goal is to demonstrate that the bug was fixed while preserving expected behavior.

4. Read changes to the tests themselves

Sometimes the implementation needs to change. Sometimes an outdated test needs to change too.

But a green result deserves closer inspection if the agent also:

• Skipped a failing test.
• Removed an important assertion.
• Replaced realistic behavior with a convenient mock.
• Changed the expected result to match the implementation.
• Disabled a check in configuration.

These changes can have valid reasons. Ask for the reason and compare it with the requirement.

“Tests pass” has limited meaning if the relevant checks were weakened.

5. Match verification to the change

Different changes need different evidence.

A successful build proves that the build completed. It doesn’t prove that checkout works.

A screenshot shows appearance. It doesn’t prove that the button saves anything.

Choose checks that exercise the promised behavior.

6. Look for fixes that moved the problem

A small bug can lead to a surprisingly large patch.

Before merging, ask:

• Did the agent change unrelated files?
• Did it introduce a dependency for a small task?
• Did it suppress an error instead of handling it?
• Did it remove validation to make the request succeed?
• Did it change a shared function and miss its other callers?

Start with the diff summary, then inspect the affected paths.

A large change isn’t automatically wrong. It should have an explanation proportional to its scope.

7. Treat blocked verification as unfinished verification

Sometimes the agent cannot run a check.

The database is unavailable. A dependency cannot be installed. Browser access is missing.

That is useful information.

The report should say:

“Implemented the change, but integration tests could not run because the test database was unavailable.”

Avoid turning “could not test” into “probably works.”

Record the missing check and complete it in the appropriate environment before relying on the result.

A prompt you can reuse

Add this to your next coding task:

Before editing, state the expected behavior and how you will verify it.

After editing, report:

1. What changed.
2. The exact checks you ran and their results.
3. Whether the regression test catches the original failure, if applicable.
4. Any tests or checks you changed, removed, or skipped—and why.
5. What remains unverified.

Only claim a check passed if you actually ran it and observed a passing result.

This won’t eliminate mistakes. It gives you a clearer starting point for assessing the work.

Before you click merge

Ask one question:

“What evidence shows that this change meets the requirement?”

Sometimes the answer is a focused regression test. Sometimes it is an integration run, a browser interaction, or several checks together.

The important part is being able to connect the evidence to the task.

An AI agent can help you implement and verify a change. Its final message should make that verification inspectable.

What’s the biggest gap you’ve found between an agent saying “done” and the work actually being finished?