跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 12 天前AI 評分39

對齊在哪裏出現,又意味着甚麼?深入探討柏拉圖表徵假説

Where Does Alignment Emerge and What Does It Mean? A Closer Look at the Platonic Representation Hypothesis

AI 導讀

研究分析單模態語言與視覺模型的跨模態表徵對齊,發現對齊程度會隨模型能力提升,但仍低於同模態模型。當圖説清晰且資訊充足時,這種對齊可超越明確對齊的視覺語言模型。

正文

Published on Nov 9

Authors:

,

,

Abstract

Cross-modal representational alignment between unimodal language and vision models increases with model capability when captions are clean and informative, but remains below same-modality alignment and can exceed explicitly aligned vision-language models.

The Platonic Representation Hypothesis (PRH) suggests that neural networks are converging toward similar representation spaces. As language models become more capable, their representations are aligning with those of vision models. However, recent work has questioned whether this trend continues to hold for newer language models. Moreover, it has been shown that representation similarity metrics can be confounded by model depth and width, requiring appropriate baselines for interpretation. In this work, we first investigate where alignment emerges between unimodal models across model layers and representation dimensions. Next, we find that the PRH capability-alignment trend persists for newer models as long as the captions are sufficiently clean and informative. Finally, we contextualize cross-modal alignment, introducing unimodal and multimodal baselines on various datasets. Our results show that the cross-modal representational alignment of unimodal models can surpass that of explicitly aligned vision-language models, but remains substantially behind the alignment observed for models of the same modality. Code is available at https://github.com/ExplainableML/prh-closer-look.

View arXiv page View PDF GitHub 5 Add to collection

Get this paper in your agent:

hf papers read 2611.12869

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.12869 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.12869 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.12869 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co