智能體軟件工程缺少可機器檢查的判定機制

A recent software-engineering experiment reportedly consumed more than 500 billion tokens over three months.

That number attracted attention, but it was not the most important result.

The project used a fleet of AI coding agents to reconstruct a large application in C++. After the first month, the output looked promising: the code was readable, the application launched, and visible features worked.

However, much of the implementation was semantically wrong.

Functions used incorrect signatures. Data structures were misinterpreted. Logic was invented or removed. Some simple operations were replaced with unnecessarily expensive abstractions.

The agents were producing activity, but the team had no reliable definition of correctness.

Why a Reviewer Agent Was Not Enough

The initial workflow included worker agents and a reviewer agent.

That sounds reasonable, but both sides were operating from the same incomplete description of the goal. When a worker explained why it had made a change, the reviewer could accept that explanation instead of independently verifying the behavior.

This exposes a common weakness in multi-agent systems:

Adding another model does not automatically create independent evidence.

A reviewer can find obvious defects, but it cannot consistently enforce requirements that have never been translated into objective checks.

The System Changed When It Gained an Oracle

The team eventually introduced an automated verification harness.

Each reconstructed function was compared with the corresponding reference output. The result was deliberately simple: PASS or FAIL.

That feedback loop made the workflow much more reliable:

1. An agent produced an implementation.
2. The verifier checked it independently.
3. Failed work returned to the agent with concrete feedback.
4. Passing work was recorded and checked again in CI.
5. The verification mechanism itself was protected from modification.

The important component was not another prompt. It was an oracle: an external system capable of deciding whether an output satisfied the requirement.

What Other Agent Workflows Can Learn

Most projects cannot compare compiled functions byte for byte, but the same pattern applies elsewhere.

For coding agents

Use tests, static analysis, type checks, security scans, performance budgets, and reproducible builds.

For data agents

Validate schemas, row counts, invariants, allowed ranges, lineage, and reconciliation totals.

For customer-support agents

Check policy compliance, required fields, escalation conditions, tool permissions, and whether cited account data actually exists.

For research agents

Require source availability, citation support, date checks, contradiction detection, and explicit reporting of missing evidence.

For workflow agents

Validate state transitions and permissions before allowing irreversible actions.

A Minimal Agent Verification Loop

A production workflow can start with a simple structure:

define acceptance criteria
set a cost and retry budget

while budget remains:
    generate or revise the result
    run an independent verifier

    if verification passes:
        accept the result
        record the evidence
        stop

    return structured failure feedback

escalate to a human or fail safely

The verifier should ideally be:

• Independent from the generating agent
• Deterministic where possible
• Protected from modification
• Easy to audit
• Able to fail closed
• Connected to cost and retry limits

Measure Verified Outcomes, Not Agent Activity

Token usage, tool calls, generated files, and completed subtasks can all look like progress.

None of them proves that the result is correct.

A more useful operational metric is:

total workflow cost / number of verified successful outcomes

This captures an important tradeoff. A cheaper model is not cheaper if it generates more failures, consumes more retries, or creates additional human review work.

At the same time, a strong verification harness may make less expensive models viable because they receive precise feedback.

The Bigger Lesson

Model capability still matters, but production Agent systems need more than intelligence.

They need boundaries, evidence, protected evaluation, retry controls, and a clear path for human escalation.

Before scaling the number of agents, define the mechanism that tells them—and your team—when the work is actually correct.

What machine-checkable oracle could you add to your current AI workflow?

Sources:

• Maurice Heumann, “500+ Billion Tokens Later: Letting AI Agents Decompile a First-Person Shooter”
• Tom’s Hardware coverage published October 12, 2026

AI disclosure: This article was prepared with AI assistance. Before publication, I reviewed the cited material and take responsibility for the final text.