水下影像品質指標是否符合人類感知?一項涵蓋人類選擇、簡單編輯與生成漂移的基準測試
Do Underwater Image Quality Metrics Agree with Human Perception? A Benchmark with Human Choices, Simple Edits and Generative Drift
研究比較 73 種水下影像品質評估指標與人類偏好,發現沒有任何指標能持續符合人類判斷。五項參照中,Q-Align 的最差 Cohen's kappa 僅為 0.23,低於研究中兩名評分者的中位數 0.41;常用的 UCIQE 亦常偏好人們不接受的編輯。研究推出 TrustUIQA 基準;在 Nano Banana Pro 複製鏈中,只有 73 項分數中的 22 項隨影像噪聲增加而逐步下降。
Published on Nov 13
Authors:
,
Abstract
Underwater image enhancement is challenging to evaluate due to the lack of clean reference images, so researchers rely on no-reference image quality assessment (NR-IQA) metrics. However, it remains unclear whether these metrics agree with human judgments. We evaluate 73 handcrafted underwater, learned underwater, vision-language and general-purpose metrics against human preferences on five references: four published opinion-score sets and a new paired-comparison study. For each metric, we measure how often it selects the same image as people when comparing versions of the same scene. We further test whether simple image edits and small adversarial perturbations raise metric scores above the enhancement people preferred, and whether scores decrease as repeated generative edits add noise. Our results show that no metric consistently matches human preferences. The best worst-case Cohen's kappa over the five references is only 0.23 (Q-Align), while two raters in our study agree with a median kappa of 0.41. The widely used UCIQE often favors edits that people reject. On a Nano Banana Pro replica chain that should leave the image unchanged, only 22 of 73 scores decrease at every checkpoint, while noise grows by 2.5 times. We also introduce TrustUIQA, a benchmark that combines the human references, our study choices, the edited, attacked and generated images, and the scores of every metric on every image. The code is available at https://github.com/TrustUIQA/TrustUIQA and TrustUIQA at https://huggingface.co/datasets/TrustUIQA/TrustUIQA.
View arXiv page View PDF Project page GitHub 0 Add to collection
Get this paper in your agent:
hf papers read 2611.18429
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.18429 in a model README.md to link it from this page.
Datasets citing this paper 1
TrustUIQA/TrustUIQA
Viewer • Updated 29 minutes ago • 20.6k • 1.6k
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.18429 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co