為何 LLM 評估不能只靠完全匹配

We run a service that reads research manuscripts and answers questions editors actually act on.

• Is there an ethical approval statement?
• Does the paper mention a clinical trial registration?
• Are figure references consistent?
• Is the self-citation rate too high?

Each question is a separate check, and an LLM does most of the reading. Because the system is already in production, mistakes matter in a very practical way: a false missing signal wastes someone’s time, and a false pass lets a real issue move further down the line.

At first, evaluating these checks looked simple. Most of them felt like yes/no questions, so exact match seemed like the obvious metric.

That worked for about two checks.

As soon as we started asking for supporting citations and structured extractions, exact match stopped measuring correctness and started measuring formatting noise.
A curly quote, a dash changed by PDF extraction, or one extra sentence around the right passage was enough to mark a correct answer as wrong.

This post is about what broke, what we changed, and why we ended up with a much simpler eval setup than we expected.

Why evals matter once the system is in production

Once a system is live, evals stop being a one-time validation exercise and become part of day-to-day engineering. Every new requirement, bug fix, or prompt update has to answer two questions: did it solve the target problem, and did it reduce accuracy anywhere else?

That second question matters more than it first appears. In LLM systems, small changes in prompts can have wide effects. A prompt tweak that helps one check may subtly change behavior on another set of manuscripts that was previously working. Without a repeatable eval set, those regressions are easy to miss until users hit them.

Our way of handling that is simple: for each check, we maintain a set of test documents in SharePoint and use them as a stable evaluation baseline. When something changes, we rerun the check on that same set and compare the outputs with the expected results. That gives us a practical way to protect existing behavior while still improving the system.

TL;DR

We started with exact match because most checks looked strict enough for it.

That fell apart as soon as checks started returning cited passages and extracted metadata. The model was often right, but exact match still said it was wrong.

We moved to a mix of exact match,F1 and fuzzy matching, with different thresholds depending on the field type.

The checks do not all fail in the same way

One of the first useful lessons was that “LLM checks” is too broad a category.

The checks we run fall into a few different shapes:

- Presence: is a required statement or section there at all?
- Position: is it where it should be in the paper?
- Extraction: pull out structured metadata such as authors, affiliations, or funding.
- Reference checks: do figures and tables cited in the text actually exist, and are they cited correctly?
- Inference: does the answer depend on combining evidence from several parts of the manuscript?

Most of the checks return structured JSON. That made the first version of the eval almost embarrassingly easy to build.

We started with exact match

The first setup was straightforward.

For every check, we keep a folder of expected outputs: one JSON file per test manuscript. We run the same prompt the service uses, save the output, and compare it against the expected JSON.
Same value, score 1. Different value, score 0.

For yes/no style checks, this worked surprisingly well. “Does the paper contain a consent statement?” has a concrete answer, and the model either gets it or it does not. We could change a prompt, rerun the suite, and see within minutes whether anything moved.

The harness itself is plain Python. One script runs every manuscript through the model and writes the outputs to a .jsonl file.

Each check has its own evaluator, which reads the actual outputs, compares them with the expected JSON, and writes a score. A small report script then combines those scores so runs can be compared over time.

It was simple, cheap, and easy to reason about.

Then we needed citations.

Then citations broke the metric

A lot of the useful checks do not just answer yes or no. They also point to the text that supports the answer: the sentence where the ethical approval is mentioned, the paragraph referencing Figure 3, the row of a table.

That is where exact match started lying to us.

The model would return the right passage, but with a curly quote where the expected file had a straight one. Or a hyphen had turned into a different dash. Or a Greek symbol, superscript, or special character had changed slightly during PDF conversion. Sometimes the model returned the relevant sentence plus one adjacent sentence, while the expected output contained only the shorter span chosen by the human annotator.

By any reasonable reading, those are correct answers.

By exact match, they are failures.

The clearest example was metadata extraction. In one run, that check scored 0.50 on exact match and 0.97 on F1. Looking at the failures made it obvious what was happening: the model was mostly correct, but the scoring method was punishing superficial differences.

That created a second problem. If the metric keeps reporting cosmetic failures, people stop trusting the report. Once that happens, a real regression can sit in the same output looking no different from all the false alarms.

We stopped scoring every field the same way

The fix was not to find one better universal metric.

The fix was to admit that different fields need different scoring rules.

We ended up with a simple split:

• Booleans and enumerations stay strict. A yes is not a maybe. These use a threshold of 1.0.
• Metadata fields get a high but not perfect threshold, around 0.95, because names and affiliations often vary in small ways without changing the answer.
• Reference text gets a looser threshold, around 0.85, because what matters is whether the model found the right passage, not whether it reproduced every character exactly.

For text fields, we normalize first. That includes lowercasing, collapsing whitespace, and stripping harmless punctuation differences and brackets. After that, we compare using fuzzy matching from rapidfuzz, taking the best score across token sort, token set, and partial ratio.

We also added a containment rule. If the model returns a meaningful chunk of the expected text, or if the expected text sits inside the returned text, that should count as a strong match. We scale that score by how much of the longer text is covered.

Stripped down, the logic looks like this:

def fuzzy_match_score(expected: str, actual: str, threshold: float = 0.85):
    if expected == actual:
        return {"exact_match": True, "fuzzy_score": 100.0, "passes_threshold": True}

    a, b = normalize_text(expected), normalize_text(actual)

    shorter, longer = sorted([a, b], key=len)
    if len(shorter) > 20 and shorter in longer:
        ratio = len(shorter) / len(longer)
        if ratio > 0.3:
            score = 85 + ratio * 15
            return {
                "exact_match": False,
                "fuzzy_score": score,
                "passes_threshold": score >= threshold * 100,
            }

    score = max(
        fuzz.token_sort_ratio(a, b),
        fuzz.token_set_ratio(a, b),
        fuzz.partial_ratio(a, b),
    )
    return {
        "exact_match": False,
        "fuzzy_score": score,
        "passes_threshold": score >= threshold * 100,
    }

The report now shows exact match, F1, and fuzzy match side by side.

Reading those together matters more than any single score. If exact match is low but fuzzy is high, the problem is usually formatting noise. If all three are low, something is actually broken.

For list-like outputs, such as figure references, we match greedily by item rather than by position. If the model finds the same five references in a different order, that should not count as five mistakes.

Why we kept the eval boring on purpose

We considered more sophisticated approaches.

LLM as a judge was the obvious one. Ask another model whether the output is correct. It is flexible, and it reduces the need for exact expected text. But it also adds a second non-deterministic component. When a score changes, you then have to work out whether the system under test moved or the judge moved. For checks with a clear right answer, that felt like adding noise to fix noise.

We also looked at evaluation frameworks such as promptfoo, DeepEval, and Ragas. They are useful tools, especially if you are starting from scratch. In our case, the checks return structured, domain-specific JSON with per-field scoring rules, so we would still have ended up writing most of the custom logic ourselves.

Embedding similarity was another option. It is attractive for free text, but it can be too forgiving. A sentence that mentions approval from the wrong committee can still look semantically very close to the correct answer. For our use case, “close in meaning” is sometimes still wrong in a way that matters.

And of course there is human review of every run, which is the most accurate option and also the least scalable.

What we ended up trusting most was much less exciting: ground-truth files, deterministic scoring, and simple string and set comparisons.

It is not fancy. But it is cheap, transparent, and fast. More importantly, when a score drops, we can open the actual and expected JSON files and see exactly why.

That turned out to matter more than any advanced metric.

Where the ground truth comes from

Our QE team prepared the expected outputs.

The manuscripts are a mix of real production papers and curated edge cases. Some were selected because they reflect the kinds of papers the service sees every day. Others were chosen or created specifically to stress the checks: unusual structure, buried statements, missing sections, or text that looks like the thing we want but is not actually the thing.

That mix is deliberate.

Real data tells you how the checks behave on typical manuscripts. Curated cases tell you how they behave on the papers that cause trouble. If you only use real data, the eval can look flatteringly easy. If you only use adversarial cases, it can look unrealistically harsh.

The dataset is small: roughly eight papers per check.

That is enough to catch obvious regressions after a prompt or pipeline change. It is not enough to claim a precise accuracy number, and we try not to pretend otherwise.

What I would tell someone starting out

Start with exact match. It is a good way to discover which fields are genuinely strict and which ones only look strict at first.

Choose metrics per field type, not per project. A boolean answer and a quoted passage should not share the same definition of correctness.

When a score drops, read the diffs before touching the prompt. More than once, the thing that was broken was the eval, not the model.

Keep a few ugly cases in the dataset on purpose. Production data is representative, which usually means it is mostly easy.

If part of a check is deterministic code, evaluate that part together with the model. The user experiences the whole pipeline, not just the prompt.

And version your prompts next to your eval results. Otherwise, a few weeks later, you will not remember which change moved which number.