多智能體審查小組為何自信判錯？Erik Hill 以實測分析原因

I run a lot of code review through agent fan-outs. Dozens of agents, each with a different lens, structured output, adversarial verification passes. The kind of pipeline that looks rigorous in a diagram.

Last week it returned a confident, unanimous, wrong answer. I kept the transcripts, so I can tell you exactly what it cost and exactly what fixed it.

The numbers are from one week of runs: 158 agents across 18 workflows, 139 MB of transcript, roughly 28 million tokens. About a third of that was avoidable.

The failure

A panel of three agents reviewed a technical claim and had to decide whether it was real and worth acting on. All three said no. Each wrote a detailed, specific, well-argued rejection.

Their reasoning turned on a published standard that governed the decision. None of them could open it. The document sat behind a login, and the agents correctly refused to authenticate, which is the behavior I want. So they reasoned from the title, from search snippets, and from what a document with that title probably says.

It says close to the opposite.

I read it myself through an authenticated browser session. One tool call. The actual text answered, in its own worked example, the two specific objections all three agents had built their rejections on.

That read cost about 2,000 tokens. It overturned about 1.2 million.

A 600 to 1 asymmetry, and it is the single most useful thing I did that week.

The part that is actually uncomfortable

I re-ran the panel with the real document text. This time all three agents reversed and said yes.

Good result, except for one thing: I wrote both briefings. The first one told them to be hostile and assume the claim was wrong. The second one quoted the standard and noted the previous panel had misread it.

Same question. Same model. Same evidence. Opposite framings, opposite verdicts.

So the second run was not a correction. It was me getting the answer I had already decided on, and three agents handing it back with citations. I caught it only because I happened to reread my own prompt and noticed I had written "read that last paragraph twice" above the paragraph I wanted them to weight.

I killed that run and rewrote it: quote the source with no commentary, strip the helps and hurts labels off the evidence before passing it on, and add a required field asking each agent where the briefing is wrong.

That version also said yes. But this time two of the three told me my briefing had overstated one constraint and buried a counterargument I should have surfaced. One pointed out I had handed them a favorable argument while omitting an unfavorable clause from the same document.

That is what a panel is for. Agreement is cheap. The value is in the disagreement, and you only get it if you leave room for it.

The failure mode you are not watching for

Partway through the week I hit a usage limit. Four agents died mid-run.

That cost me nothing. The runtime printed a failures block. A one-line check over the journal confirmed it independently:

for d in */; do python3 -c "
import json;ev=[json.loads(l) for l in open('$d/journal.jsonl') if l.strip()]
print('$d',sum(1 for x in ev if x.get('type')=='started'),
            sum(1 for x in ev if x.get('type')=='result'))"; done

Started versus returned. If those numbers differ, something died. Loud failures are cheap because they are visible.

The expensive failure was the agent that succeeded. It completed normally, returned a well-structured result, and reported that it could not reach the document it needed. Then it reasoned from snippets anyway, and the agents downstream treated its output as settled. Nothing flagged it, because nothing had gone wrong in any way a harness can detect.

An agent that dies tells you it died. An agent that guesses tells you nothing.

The fix, which is nearly free

I added one required field to every judging agent's output schema:

provenance: for each load-bearing claim, state READ (and what),
            EXECUTED (and what you ran), INFERRED, or BLOCKED.
            A verdict whose key premise is INFERRED must say so.

Both models I tested comply with this well, and it costs almost nothing. It changes what you get back from confident to checkable.

The best example came from a run where an agent marked a login-gated page BLOCKED, listed precisely which figures it had taken on trust from an earlier agent without re-running them, and separated those from the facts it had verified at source itself. That agent's verdict was worth more than the three confident ones, because I could see where to push.

The clean negative that was the most thorough run

After the framing result I wanted a harder measurement, so I set up a blind test. This is the same method I use for grading eval suites: take material where the answer is already established, hide the answer, and see what the instrument actually detects. A detection rate you measure beats a detection rate you assert.

I locked six agents to a source tree, put every file containing prior analysis out of reach, and gave all six an identical prompt that never said what to look for.

Three of the six surfaced the underlying mechanism. None of the six assembled it into the thing that actually mattered, which was two separate observations that only become significant once you put them together.

The run I keep thinking about returned nothing at all.

That agent did the most careful work of the six. It built the project, ran the existing 391 tests, extracted every reference document in the repository, and checked each of its candidates against them. It correctly matched the pieces it found to issues that were already documented and accepted. Then it concluded, reasonably, that nothing new was there.

Its own notes include a sentence describing the exact lines the real problem lives on, and recording them as safe.

It was not careless. It was following my instruction to treat the list of already-known issues as a hard filter, and it followed that instruction more faithfully than the agents that got closer. Each documented issue described one mechanism. Each carried an exception for a case its own mechanism never reaches. The combination reaches it.

So my filter was wrong, not the agent. "Does this match something already known" discards the combination that escapes what is known. The question has to be "does this match, and does any combination of these candidates reach the exception that the known item itself states."

The general version: when you give agents a filter, you are also telling them what to throw away, and a careful agent will throw away more.

What I changed

Four rules, in the order they run:

Read the governing documents yourself, before any agent starts. Whatever the decision actually turns on: the spec, the standard, the policy, the config. Hand one facts file to every agent so none of them re-derive it, and none of them guess. If it is behind a login, that is a human's job, and it is one tool call.

Cheap checks before expensive ones. My pipeline built full reproductions and then discovered, late, that two candidates died on questions costing one lookup each. Now the cheap disqualifying questions run first, on everything, before anything expensive runs on anything.

Never brief a panel toward an answer. Quote sources without commentary. Strip verdict-shaped labels off evidence before passing it downstream. Require a disagreements field.

Cap exploratory work on elapsed time or error rate, not on completion. One probe ran 99 minutes and accumulated 465 errors before dying on rate limits, chasing something an earlier probe had already ruled out structurally. Nothing stopped it, because nothing was watching anything except whether it had finished.

Add a composition pass, and audit your filters. One agent, not a fan-out, asking a single question of the whole candidate list: does any combination here add up to something none of them is alone. And for every filter you hand an agent, write down what that filter throws away, then go and look at the discard pile once.

The honest limits

One week. One pipeline. One domain. I measured transcript volume and token counts, not a controlled experiment, and the framing result is n equals 3 on a single question.

What I am confident about is narrower than it sounds: when a panel of agents agrees, and one person wrote all the briefings, the agreement is evidence about the briefing. Everything else above is a consequence of that.

The deeper version of this is not new. A check that cannot fail is not a check. We know this about CI gates, where a required job satisfied by a skipped step passes review for years because nobody asked it to prove it could go red. We know it about test suites. It is just as true of a review panel, and it is harder to see, because the output is fluent, structured, and arrives in triplicate.

If you run agent fan-outs, go and read your own prompts. Look for the sentence where you told them what you wanted to hear.

I found mine.