跳到正文
DEV Community· Mayur Agnani·· 1 天前AI 評分47

如何檢查 AI 答案中的引文是否支持其主張

How to Check Whether an AI Answer's Citations Back It Up

AI 導讀

檢查 AI 答案的引文,須分開確認引文是否出現在來源中,以及它是否支持相關主張;前者可精確比對,後者需結合簡單規則與判斷模型。在文中的測試中,單靠詞語重疊分數的做法錯誤放行 7 個錯誤引文中的 5 個,並拒絕 9 個正確引文中的 3 個。

正文

An AI answer can cite a real sentence from your own document and still get the fact wrong. You can check that the sentence exists with one string comparison. Checking that it backs the claim takes something else.

Engineer's view. This is a foreign key that points at a real row, just the wrong one. The database checks that the row exists, never that it is right. Citation APIs can guarantee the quoted passage exists in your document. None check that it backs the claim, so users see a wrong answer with a real source.

TL;DR. To verify an LLM citation, check two things separately: that the quoted passage exists in your source, and that it backs the claim. The first check can be exact. The second needs simple rules plus a judge, because counting shared words cannot tell a fair paraphrase from a quote about the wrong thing.

Explain Like I'm 5

A friend writes you a book report and puts a page number after every sentence. You can check each sentence in two ways.

First, open the book to that page. Are those words really there? That takes one look, and you are sure.

Second, read the words and ask whether they mean what your friend wrote. Maybe the page says the hero lost, and the report says the hero won. That check takes thinking. Words that look alike can still mean opposite things.

The first check never needs the second one. The second check can never be done with only the first.

The Problem

You have shipped this bug before, in a system that had nothing to do with AI.

Your docs site runs a link checker in CI. Any link that returns a 404 fails the build. Then a support ticket arrives. A page about rate limits links to the billing section. The link resolved fine, so the checker passed it. It only ever asked whether the target exists. It never asked whether it is the right target.

An AI answer with citations has the same blind spot, one level down. A model writes a marker like [2] the way it writes any other word: it predicts what usually comes next. So a citation can fail in three ways. It can point at a source that does not exist. It can point at a real passage that says something else. Or it can be right. The user sees the same tidy footnote in all three cases.

Citation features in model APIs close the first gap by different amounts. The strongest return the exact passage, cut from your own document. None of them check that the passage backs the claim.

The obvious cheap fix makes it worse. Teams score word overlap between the claim and the quote, and pass anything above a threshold. In the test built for this article, that rule passed 5 of 7 bad citations. It also rejected 3 of 9 good ones.

The fix: check the pointer exactly, check numbers and negations by rule, and send every other citation to a judge.

Read the full post on The AI Commit

How to Check Whether an AI Answer's Citations Back It Up carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.


Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

來源:DEV Community · dev.to