如何讓本地 RAG 模型在語料缺少答案時拒答
NOT IN CORPUS — teaching a local model to abstain instead of hallucinate
作者為本地 RAG 系統加入提示詞行為規則，要求模型只使用提供的上下文，缺少答案時回覆「NOT IN CORPUS」，並標註各項事實的來源標題。作者以已知不在語料中的查詢測試拒答，並指出案例中的檢索已找到相關內容，幻覺出現在小模型生成階段。
I asked my local RAG stack about my own files and it hallucinated. Confidently. The corpus was mine, the retrieval was correct, the small model just... invented an answer around thin context, in fluent English.
The fix wasn't a bigger model. It was a behavioral contract, enforced in the prompt and proven in testing:
text
You are a grounded assistant. Use ONLY the context below. Rules:
(1) If the context does not contain the answer, reply exactly: NOT IN CORPUS.
(2) Cite which source title each fact came from.
(3) Do not use outside knowledge.
The result: the same query that hallucinated now replies "NOT IN CORPUS." — and when the context is there, the answer arrives with source citations attached.
Two things I learned building this:
The retrieval was never the problem. Vector search found the right neighborhood every time. The mouth was the problem — a small model treating thin context as an invitation to improvise. People blame RAG for hallucinations that are actually generation-layer failures.
Abstention is a feature you have to test. I now run queries I know are absent from the corpus, specifically to check the model refuses. An abstention you haven't tested is an abstention you don't have.
The stack: Ollama + nomic-embed-text + ChromaDB + granite 3B, fully local on an 8GB MacBook Air. 300K+ chunks of my own archive, no cloud, no API, no accounts. The orchestrator dispatches each question to four workers and the EvidenceFlow guard verifies before anything reaches the screen.
My full archive answers questions in my own voice now — and it has permission to say "I don't know." That permission is the feature.
The tooling is published: ask-mirror — hybrid retrieval (vector + SQL) with the abstention guard built in, AGPL. The verification discipline comes from EvidenceFlow — fail-closed on Ship 5, now guarding the whole stack.
來源：DEV Community · dev.to