文件問答 Demo 對知識庫外問題拒答率達 96%，對近似問題僅 39%；測試集揭示差異

Here are two numbers from my document Q&A demo. Ask it about something that is not in the knowledge base at all, and it refuses correctly 96.1% of the time. Ask it a question that sits close to a document but is not answered by it, and it refuses only 39% of the time.

If I had tried the demo by hand, I would have asked a few questions, seen it say "not in the docs" a couple of times, and called it good. The 39% only showed up because there was a labelled test set. That set is the subject of this post: a golden set, the cheapest acceptance test an AI feature can have.

What it is

A golden set is a list of real inputs with the answer a human accepts for each one. Nothing more. It is not training data and it is not a benchmark you publish. It is the thing you run after every change to ask one question: did it get better or worse?

Without it, every change is judged by whichever three examples someone happens to try. With it, "works pretty well" becomes a number you can argue about.

How I would build one in an afternoon

1. Take real inputs. 100 is enough to start. Pull them from the actual process: tickets, invoices, questions people really ask. Not examples you invented to look nice.
2. Cover the ugly cases on purpose. Roughly: most of it typical, a quarter hard or ambiguous, and a handful that must be refused or sent to a human. The ugly part is where the demo above found its 39%.
3. Label it with the person who owns the process. The owner says what "correct" means. If two people disagree on a label, that disagreement is a finding, write it down.
4. Freeze it. Do not tune the system on these examples. Tune on a separate split and touch the golden set only to report. In my ticket-triage demo the threshold is chosen on a validation split and the test set is used once.
5. Write the acceptance rule before you run it. For example: "at least 97% correct on what is handled automatically, everything else goes to a person." Then the run is a pass or a fail, not an opinion.

What it showed me in my own demos

These are public demos on public datasets (test data, not client results):

• Ticket triage. If everything were automated, accuracy would be 90.2%. Handle only the confident 75.9% and accuracy on those is 97.7%. The golden set is what made the trade-off visible.
• Document Q&A. Out-of-base questions are refused 96.1% of the time, but near-miss questions only 39.4%. The cited answers hit 93% precision against a 95% target. I reported the gap instead of hiding it, and it points to the next fix: a reranker or an LLM for exactly those near-miss cases.

In both cases the number told me where to work next. A hand-tried demo would have told me nothing.

Pitfalls

• Leakage. If the system has seen the golden examples while being tuned, the score is fiction.
• Too clean. A set made only of easy cases gives 99% and a false sense of safety.
• Never refreshed. Real inputs drift. Add new failures to the set whenever production surfaces one.
• One number for everything. Report per category. A fine average can hide a category that fails every time.

Takeaways

1. A golden set is real inputs plus answers a human accepts.
2. Build it before the pilot, with the process owner.
3. Include the ugly cases deliberately.
4. Freeze it, and write the pass rule first.
5. Report per category, and add every new production failure to it.

If you can only do one thing before your next AI pilot, label 100 examples. It is the cheapest hour of the project.

Portfolio and contacts: @Kotenev01_bot.

I write about practical, measurable AI adoption for business: demos on test data, no invented case studies. Telegram channel: @kotenev01_en