如何偵測 AI 幻覺：用完整程式碼建立 LLM 答案核查器

Here's an uncomfortable fact about AI answers: the lies are often the best-written part. I recently scored an AI-generated customer-support answer with my own evaluation tool — it earned 9/10 for clarity and 4/10 for faithfulness. Beautifully phrased, confidently wrong: it invented a 30-day refund window (the policy says 14), doubled a discount, and promised customers a free grinder that doesn't exist.

Fluent is not the same as true. If you're putting LLMs in front of customers — chatbots, support macros, document Q&A — you need a way to measure whether the output is grounded in your actual content. This guide builds one: an LLM Answer Checker that takes a source text and an AI answer and returns a trust report — faithfulness scores, a list of unsupported claims, and a suggested fix.

Try the live demo — it opens pre-loaded with the exact lying answer from the story above. One click and you'll watch all three fabrications get caught.

The pattern: LLM-as-judge

You can't regex your way to detecting hallucinations. The practical approach — used in serious eval frameworks — is LLM-as-judge: a second model call whose only job is auditing, with a rubric, against the source.

The key insight is in the framing. You don't ask "is this answer good?" You ask: "evaluate this answer against this source only — not against your own world knowledge." That one constraint turns a vague vibe-check into a grounded audit.

The judge prompt (the heart of the tool)

You are a rigorous AI-output auditor. You receive a SOURCE text and an
ANSWER that claims to be based on it. Evaluate the answer against the
source only — not against your own world knowledge.

Score each dimension as an integer 0-10:
- faithfulness: does every claim in the answer appear in or follow directly
  from the source? (10 = fully grounded; heavily penalize invented facts,
  numbers, names, or policies)
- relevance: does the answer address what the source covers?
- completeness: does it use the important relevant information available?
- clarity: is it well-organized and easy to understand?

Also produce:
- unsupported_claims: a list of strings, each quoting or closely
  paraphrasing a specific claim in the answer that the source does NOT
  support (empty list if none)
- verdict: one sentence, starting with "Trustworthy:", "Mostly reliable:",
  or "Unreliable:"
- suggested_fix: 1-2 sentences on how to correct the answer

Return JSON exactly:
{"faithfulness": n, "relevance": n, "completeness": n, "clarity": n,
 "unsupported_claims": [...], "verdict": "...", "suggested_fix": "..."}

Three design choices worth stealing:

1. "Against the source only." Without this, the judge "fixes" answers using its own training data — which is just hallucination with extra steps.
2. Quoted unsupported claims. A numeric score tells you that something's wrong; the quoted claims tell you what. That's the difference between a metric and a debugging tool.
3. Integer scales with anchored extremes. "10 = fully grounded; heavily penalize invented numbers" produces far more consistent scores than an unanchored 0–10.

The scoring endpoint (FastAPI)

The serving code is a small FastAPI app — the same free stack as my chatbot tutorial (Groq free tier + Vercel hosting). The essentials:

@app.post("/api/evaluate")
def evaluate(req: EvalRequest, request: Request) -> dict:
    context, answer = req.context.strip(), req.answer.strip()
    if not context or not answer:
        raise HTTPException(400, "Both the source text and the answer are required.")

    messages = [
        {"role": "system", "content": JUDGE_SYSTEM},
        {"role": "user", "content": f"SOURCE:\n{context}\n\nANSWER TO EVALUATE:\n{answer}"},
    ]

    def _call(strict_json: bool):
        kwargs = {"response_format": {"type": "json_object"}} if strict_json else {}
        return client.chat.completions.create(
            model=MODEL, messages=messages, max_tokens=3000,
            temperature=0.1, **kwargs,
        )

    try:
        try:
            resp = _call(strict_json=True)
        except Exception:
            resp = _call(strict_json=False)   # fallback: parse defensively
    except Exception as exc:
        raise HTTPException(502, f"LLM error: {exc}")

    return parse_report(resp.choices[0].message.content)

And one scoring decision in the report parser: the overall score weights faithfulness double —

overall = (faithfulness * 2 + relevance + completeness + clarity) / 5

— because a clear, relevant, complete answer that invents your refund policy is still a bad answer. Weight what you actually care about.

The three traps I hit (so you don't)

Trap 1 — reasoning models eat your token budget. My first deploy failed with a cryptic 400: Failed to generate JSON. The cause: modern "reasoning" models spend tokens thinking before they answer, and those tokens count against max_tokens. My 900-token budget was consumed by thinking; the JSON got truncated mid-object; strict JSON mode rejected it. Fix: a generous max_tokens=3000 for judge calls.

Trap 2 — strict JSON mode isn't guaranteed. Even with response_format={"type": "json_object"}, some model/provider combinations occasionally fail. The code above retries once without strict mode and parses defensively — stripping code fences, locating the outermost {…}.

Trap 3 — trust nothing in the parsed output. Clamp scores to 0–10 (max(0, min(10, int(x)))), tolerate missing keys, cap list lengths and string sizes. The judge is an LLM too; your parser is the adult in the room.

What to do with the scores

A checker is a diagnostic, not a cure. When faithfulness comes back low, the production fixes are:

1. Ground the system — give the model your real content and instruct it to answer only from it (that's RAG; full tutorial here)
2. Teach it to refuse — "if the answer is not in the excerpts, say you don't have that information" turns hallucinations into honest redirects
3. Judge continuously — run your test questions through the checker whenever you change prompts, models, or docs; it's unit testing for truthfulness

If you want the grounded-chatbot side ready-made, that's exactly what my AI Chatbot Starter Kit ($29) is — a RAG chatbot that answers only from your documents, with the refusal behavior built in. The free tutorial code is the lighter version.

I'm Sammishra — an AI engineer and published researcher. I build chatbots, RAG pipelines, and AI evaluation tooling for businesses. See my work (including the live demos) at *sammishra.hashnode.dev/work-with-me*, or email sammishrakh.freelancer@gmail.com.