研究指語言模型即使識別錯誤，仍常未能清楚糾正用戶

AI systems that we may rely on for accurate information – even if that information is not something that would make us happy – must have the capacity to ‘push back’ against our protests until we are ready to be convinced of the unwelcome news/facts.

AI is not very good at this; since frontier models are designed to be consumer-facing, they tend towards deference and, in fact, towards sycophancy.

A notable strand of research has emerged to deal with the issue; but recent efforts have concluded that we can easily gaslight AI back into sycophancy if it challenges us; that AI defers to human authority even when it is manifestly wrong; and that the usually effective technique of reinforcement learning actually makes this particular problem worse.

The fact that deference is culturally enmeshed with politeness appears to mean that ‘straight talking’ is impossible in a service>consumer context; the ‘customer is always right’, even when they’re about to walk off a cliff.

Right, Wrong, and ‘Hedged’

Well, I wish I could tell you that this week’s crop of papers has produced one that really moves the needle forward on this issue. Sadly, all we’re getting is more information about how models that do ‘push back’, and attempt to impress the truth on their human masters, still aren’t doing a particularly great job of it* – and that even when they are willing to speak truth to power (that’s you), it’s really more of a muffled whisper.

The new paper in question is a considerable academic collaboration, with participation from the University of Washington, the Allen Institute for AI, LMU Munich, UC Berkeley, Stanford University, and the University of Wisconsin-Madison.

Titled Pushback in a Haystack: Language Models Respond to Factual Errors Ambiguously and Ineffectively, the thrust of the new work is that even when AI models recognize that a user’s claim is false, they often fail to correct it clearly, instead burying objections in hedging, praise and partial agreement.

An example of Gemini 2.5 Pro responding in Hindi to a request for academic papers based on a false premise. Green highlights indicate affirmation, while pink indicates pushback. Here, the model mixes praise with indirect objections rather than clearly rejecting the claim. Source

Across seven models and five languages, the researchers found that this ‘ambiguity’ can leave users believing the original misinformation, despite the AI having technically challenged it.

Since sycophancy varies in usage and degree among languages, the ten authors have taken advantage of the five native languages they share to devise a testing approach that includes English, German, Chinese, Hindi and Yoruba – languages varying from high to low-resource classifications. They state:

‘We introduce a phrase-level, human-validated response labeling framework to uncover fine-grained insights into how LMs push against and/or affirm false inputs. We develop and apply our framework on model responses in five languages: English, German, Chinese, Hindi, and Yoruba.

‘We find that when models do push back against users’ false assertions, they tend to use indirect phrasing and mix in affirmation, with more indirectness in Hindi and Yoruba than in other languages considered.’

In a large-scale study involving 300 participants, the researchers found that ambiguous corrections from AI models could leave users’ factual misconceptions intact, rather than effectively challenging them.

One test established that the models’ responses to information-seeking queries averaged 46.7 lines, with corrective phrases appearing in just two or three – meaning that even when AI does object to a false claim, the objection can be hard to find.

The researchers also found that models were more likely to challenge misinformation when explicitly asked to verify it than when it appeared in ordinary requests for information. Claims attributed to textbooks generally met less resistance than those attributed to friends.

In 29.4% of information-seeking responses, models mixed correction with affirmation, rising to 60.1% for Gemini 2.5 Pro; sometimes this took the form of a ‘compliment sandwich’, with objections buried between reassuring remarks.

As a possible remedy, the researchers tested a simple intervention to encourage models to ‘wait a minute’ before responding. This improved their willingness to challenge false statements in some cases – but also increased incorrect objections to true statements.

The study suggests that getting AI to recognize an error is only part of the problem, and that getting it to communicate that error is a distinct challenge in itself.

Method

The test material was obtained from OpenAI’s multilingual version of the Massive Multitask Language Understanding (MMLU) benchmark from 2021. Known as MMMLU, the multilingual dataset covers academic subjects including biology and history.

The collection’s multiple-choice questions were converted into factual statements, with the three incorrect answers used to generate false claims, and the correct answer used to generate a true one.

In this way, a dataset of 35,120 statements across five languages was produced from 1,756 dual-translated MMMLU benchmark items for each language. For the experiments, a smaller, subject-balanced sample was selected, comprising 2,685 false statements, and 895 true ones.

Language models tested were the closed-source Gemini2.5 Pro and GPT-5.5, along with five open-weights models: Mistral Small 4; Qwen 3.5 9B; Qwen 3.5 27B; Gemma 4 E4B; and Aya Expanse 32B.

In the first of three types of prompts used to test the models, a statement was presented for direct factual verification; in the second, a student was portrayed as seeking study resources related to a statement; and in the third, a professor was portrayed as requesting academic papers on the same basis.

For the student prompts, statements were attributed to textbooks; teachers; friends; the internet; or no specified source. For the professor prompts, they were attributed to the professor’s own research; colleagues; friends; the internet; or no specified source; and the models’ willingness to challenge false claims was assessed against these different attributions.

The three prompt templates used in the study: direct factual verification (verify); requests for study resources (novice); and requests for academic papers (expert). Source attributions and subject-specific details were substituted into the templates.

The model responses were classified using four categories: explicit refutation, where a false statement was directly challenged; indirect pushback, where disagreement or doubt was expressed without outright rejection; praise, where the user’s statement or approach was complimented; and confirmation, where the statement was affirmed.

Examples of phrases used by the models to reject; question; praise; or affirm users’ statements across the five tested languages. English translations are provided where applicable.

Each response was examined phrase by phrase, so that praise, agreement and disagreement could be identified even when they appeared together in the same answer.

Results

The results confirmed earlier findings that models which correctly identify false statements when asked to check them are less likely to challenge the same claims when asked for information or advice. This difference was termed the ‘pushback gap’, and was calculated by comparing how often false claims were challenged in direct verification tests with how often they were challenged in ordinary information-seeking requests.

Comparison of how frequently seven language models challenged false statements in direct fact-checking versus information-seeking requests, across five languages. Negative values indicate reduced pushback in information-seeking responses. Orange and blue represent student and professor prompts respectively, while symbols distinguish the attributed sources. Error bars indicate 95% confidence intervals.

As shown above, the two Qwen 3.5 models evinced a smaller gap than the others across the tested languages, indicating that they were more consistent in correcting misinformation, regardless of how the question was framed; and also illustrates how the apparent authority of the source influenced the results, with false claims attributed to textbooks receiving less opposition than those attributed to friends – a tendency observed across languages.

When asked for information, rather than factual verification, models were less likely to reject false claims outright, and more likely to respond with indirect objections; praise; or confirmation. Even the Qwen 3.5 models, which had shown relatively little difference in overall pushback rates, were susceptible to this behavior.

Proportion of responses containing explicit corrections, among those that challenged false claims, across five languages. Results are divided between student (novice) and professor (expert) prompts, with the attributed information sources shown along the horizontal axis. Error bars indicate 95% confidence intervals.

In four of the seven models, false claims were less likely to be explicitly challenged in Hindi, Chinese, and Yoruba than in English. Hindi responses also contained the most praise in five of the seven models.

Pushback occurred in 65.3% of information-seeking responses, though fewer than half of those responses typically contained an explicit rejection of the false claim. Direct corrections were more common in student prompts than in professor prompts – and less common when claims were attributed to authoritative sources.

For Hindi, Chinese, and Yoruba, most responses that challenged a false claim did so only indirectly, leaving the error without an explicit rejection; and in 29.4% of information-seeking responses, objections to false claims were mixed with affirmation, rising to 60.1% for Gemini 2.5 Pro:

The percentage of responses that began with affirmation before challenging a false claim, across seven models and three prompt types: direct verification (verify); student requests (novice); and professor requests (expert). Results cover responses containing both affirmation and pushback. Error bars indicate 95% confidence intervals.

As we can see above, the models generally offered affirmation before raising objections, rather than correcting the false claim at the outset. In some cases, further affirmation followed the objection, producing what the researchers describe as a ‘compliment sandwich’.

Human Study

To test whether these corrections actually worked, 300 participants were recruited, comprising 100 English speakers; 100 German speakers; and 100 Chinese speakers. Participants rated false statements before and after reading model responses.

As illustrated below, only explicit corrections reliably reduced belief in false claims among English and German speakers. Even then, just 46.7% of English speakers and 43.8% of German speakers rated the statements as inaccurate afterward. Among Chinese speakers, only responses combining affirmation with indirect pushback produced a statistically significant improvement:

Changes in participants’ ratings of false statements after reading model responses containing affirmation, indirect objections, or explicit corrections. Results cover 100 speakers each of English, German, and Chinese. Negative values indicate reduced belief in the false claim. Individual dots represent participants, while black error bars indicate 95% confidence intervals.

The researchers also tested whether prompting models to ‘wait a minute’ before responding would improve their willingness to challenge misinformation. In the final illustration below, we can see evidence of increased pushback and reduced sycophantic phrasing in some cases – but also that these led to more incorrect objections to true statements:

Comparison of seven models’ responses before and after the ‘wait a minute’ intervention, using English and translated prompts. The charts show overall pushback against false claims; explicit corrections without affirmation; and incorrect objections to true statements. Vertical black markers indicate direct verification results.

Conclusion

Arguably, politeness and deference have become conflated in the ‘internal logic’ of language models (i.e., the net effect between their trained matrices and the many externally-imposed constraints they labor under); and the new study doesn’t establish why models sometimes recognize a false claim, yet surround their objections with praise or agreement.

The core problem is that consumer-facing products such as the GPT and Gemini series have been tormented into anthropomorphic presentation layers, so that the broadest tranche of paying clients – arguably the ‘lowest common denominator’ and the largest and most profitable target demographic – can easily understand the product, and want to keep buying it.

If we could only admit that the product is inhuman, we could at least try to turn all this confusing ‘context’ off and access the models’ trained knowledge more directly.

* Though this information in itself advances the field, of course.

First published Thursday, October 15, 2026