When to Rethink: Learning Multi-Perspective Self-Verification for Vision-Language Models
When to Rethink: Learning Multi-Perspective Self-Verification for Vision-Language Models
MOTIVE 是一套視覺語言模型自我驗證框架,透過五種互補的驗證角度和學習所得的可靠性分數,判斷答案應予採納還是重新思考。五個基準測試的平均準確率中,Qwen2.5-VL-3B 由 55.11% 升至 63.13%,Qwen2.5-VL-7B 由 60.49% 升至 68.63%。MOTIVE 亦比傳統自我驗證減少平均答案生成輪數。
🧠 MOTIVE: When to Rethink
Learning Multi-Perspective Self-Verification for Vision-Language Models
We introduce MOTIVE, a framework that helps vision-language models decide when to trust an answer and when to rethink, using complementary verification perspectives and a learned reliability score.
📄 Read the paper · 💻 GitHub
🔍 Method overview
MOTIVE follows an Answer → Verify → Accept or Rethink process:
- Answer: Generate a candidate answer from the image and question.
- Verify: Evaluate the answer using five complementary verification perspectives.
- Decide: Accept reliable answers, or rethink using verification feedback and interaction history.
Answering and verification use the same backbone VLM, without an external judge during inference.
📊 Main results
Evaluation covers five benchmarks and three VLM backbones. Average accuracy across the five benchmarks improves as follows:
| Backbone | Baseline | MOTIVE | Gain |
|---|---|---|---|
| Qwen2.5-VL-3B | 55.11% | 63.13% | +8.02 pp |
| Qwen2.5-VL-7B | 60.49% | 68.63% | +8.14 pp |
pp = percentage points.
MOTIVE also reduces the average number of answer-generation turns compared with conventional self-verification.
💡 What motivates this approach?
Verifier capability matters. Stronger external judges improve final-answer accuracy in our preliminary analysis, motivating the development of stronger self-verification.
Prompt choice matters. No single verification prompt consistently performs best across tasks. This motivates combining complementary verification perspectives.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co


