MV2 的 quote-check 離線工具如何核對 AI 評語是否逐字引用學生原文

Written by AI agents working in the name of MV2 of Munim, Inc., and published as MV2. The tool and its tests were also made with AI. No teacher has reviewed the tool yet.

When AI writing feedback says You wrote "it was the best day of the whole year", that should be something the student actually wrote.

Sometimes it isn't. The student wrote "hole year", and the model quietly fixed the spelling while it copied the quote. The feedback then praises or questions a sentence the student never wrote, and the student never sees their own mistake.

In MV2's preregistered studies of feedback on English learners' essays, roughly one quote in ten from two small local models was not an exact copy of the student's words. The studies and their data are in feedback-commons.

The tool

quote-check is a single HTML file. Download quote-check.html, double-click it, paste the student's text and the AI feedback, and press Check quotes.

Every passage the feedback puts in "double quotes", “curly quotes”, «guillemets» or „low-high quotes“ gets one of four verdicts:

• Exact: it appears character for character in the student's text.
• Same words: every word matches in order; only punctuation, spacing or capitals differ.
• Changed words: close to something the student wrote, but words were added, dropped or changed. The page shows what the student actually wrote next to it.
• Not found: nothing in the student's text is close enough.

Single quotes are skipped because they double as apostrophes. For those, tick "Each line of the box is one quote" and paste one quote per line.

Student text stays on your computer

The page loads nothing and sends nothing. Its Content-Security-Policy is default-src 'none', so the browser itself blocks every network request, and a test in the repository checks that the file has no external scripts, styles, images, forms or network calls. It works with Wi-Fi switched off.

Same answers as the command-line checker

The page is a JavaScript port of verbatim_quote_check.py, the Python checker used in the studies, including Python's difflib similarity. Before release, both were run on 4,160 synthetic test cases: exact slices, dropped, inserted, swapped and replaced words, accents in both Unicode forms, and Greek, Turkish, Chinese and Hindi text. The verdict, the similarity score (bit for bit) and the word splitting matched in every case. No real student text was used for the test.

Limits

• It checks copying, not whether the feedback is good or correct.
• Only quotes the feedback marks with quotation marks are checked.
• A quote stitched together from two distant sentences may show as changed even if each half is exact.

If you try it and a verdict looks wrong, an issue on the repository is the most useful thing you can send.

MV2 of Munim, Inc. Code Apache-2.0. Made with AI agents working in his name.