誰來評估評估者？LLM 評審出錯時的檢查方法

You update a prompt, generate a new answer, and ask another LLM to evaluate it.
_
The score goes from 7/10 to 9/10._

That looks like progress. But what actually improved: the answer’s correctness, its presentation, or its ability to impress the evaluator?

For an AI engineer, the evaluation pipeline deserves the same attention as the application itself.

What is LLM-as-a-Judge?

LLM-as-a-Judge means using a language model to assess another model’s output.

You provide a question, candidate answers, and evaluation criteria. You might also include source documents or a reference answer.

The judge then assigns scores or selects the better answer.

This helps scale evaluation, but the evaluator also has limitations.

Where evaluation can go wrong

The paper Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena investigates limitations including position bias, verbosity bias, and reasoning errors. These are useful failure modes to test in an evaluation pipeline.

Position bias: The order of candidate answers can influence the judge’s preference.

Verbosity bias: A longer answer can receive more credit even when its extra detail adds little value.

Reasoning errors: The judge can accept a convincing explanation that contains a factual or logical mistake.

A detailed evaluation explanation can help with inspection, but it does not establish that the judgment is correct.

A concrete example

Consider this fictional policy:

Customers can cancel within 14 days of purchase. The policy provides no exceptions.

Question: Can I cancel my purchase after 20 days?

Candidate A:

“No. The policy allows cancellation within 14 days, so a cancellation after 20 days falls outside that window.”

Candidate B:

“Yes. Although the standard cancellation window is 14 days, customers can request an extension to 30 days by contacting support.”

Candidate B sounds helpful, but it invents an exception.

Now imagine these judge outputs:

This is a hypothetical illustration, not an experimental result.

The preferred answer changed even though the question, evidence, and candidate answers stayed the same. That flags order sensitivity.

Notice that we track the underlying candidate, not its display position. Otherwise, swapping labels could make a consistent judgment look inconsistent.

How to build a more reliable evaluation

1. Define specific criteria

“Rate this answer from 1 to 10” leaves too much open to interpretation.

Use separate dimensions:

Separate scores help expose trade-offs. An answer can be relevant and readable while still making unsupported claims.

2. Supply evidence and use direct checks

For document-based questions, give the judge the relevant source text.

For calculations, check results with code where possible. For structured outputs, validate the schema directly.

Use model judgment for aspects that require interpretation, alongside checks that can verify specific properties precisely.

3. Test answer-order sensitivity

Prepare 20 questions with one candidate pair per question.

Evaluate each pair twice: once in each order. That produces 40 judgments across 20 paired comparisons.

Map every verdict back to one of three outcomes:

• Candidate A wins.
• Candidate B wins.
• Tie.

Define the order-change rate as:

Pairs whose mapped outcome changes ÷ total pairs

Under this definition, A → B, A → Tie, and Tie → B all count as changes. Tie → Tie does not.

You can also report strict reversals separately: only A → B or B → A.

Keeping ties visible makes the metric easier to interpret.

4. Compare with human-reviewed examples

Build a small evaluation set containing correct, incorrect, incomplete, and unsupported answers.

Have reviewers use the same rubric and supporting evidence, then inspect disagreements with the model judge.

Human review also needs quality control. Ambiguous criteria or incorrect reference judgments can make agreement misleading. Resolve unclear cases before treating those labels as a reliable baseline.

What these safeguards cannot prove

A judge can consistently prefer the same answer and still be wrong.

Order consistency measures stability under one change. It does not measure factual accuracy by itself.

Likewise, agreement with human reviewers is useful only to the extent that their judgments are dependable.

These checks provide different pieces of evidence. No single score establishes that an evaluator is reliable across every task.

The engineering takeaway

Before treating a higher evaluation score as an improvement, inspect the rubric, supporting evidence, order sensitivity, and agreement with reliable human judgments.

When an LLM judges your application, the judge becomes another component you need to evaluate.

Do you use LLM-as-a-Judge in production, and how do you measure its agreement with human reviewers?