跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 13 天前AI 評分32

訓練 Vision Transformers 聚焦:空間注意力頭專門化的形成

Training Vision Transformers to Focus: Emergence of Spatial Attention Head Specialization

AI 導讀

研究提出兩種視覺聚焦損失函數,讓 Vision Transformer 的多頭注意力對應影像區域,並形成專門化的空間注意力頭。這些注意力頭無需標註即可展現多樣且聚焦於空間位置的感知能力,並提升零樣本影像分割等密集預測任務的表現。

正文

Published on Nov 9

Authors:

,

Abstract

Two visual focusing losses align a Vision Transformer's multi-head attention with image regions, producing specialized spatial attention heads and improving dense prediction such as zero-shot segmentation without labels.

Self-supervised learning in Vision Transformer models, such as DINO, enables robust visual perception without requiring labeled data by leveraging large-scale image datasets. However, these models lack a focusing mechanism during training, resulting in perceptual representations that are often ambiguous or overly similar. In this work, we introduce two visual focusing loss functions designed to establish correspondence between the model's attention and specific regions within images. Specifically, we leverage the multi-head attention mechanism within the model to selectively steer the model's focus, leading to the emergence of spatial attention head specialization, characterized by diverse and spatially focused perceptual capabilities without requiring annotations. Through both qualitative and quantitative evaluations, we demonstrate that our method substantially increases the spatial selectivity and diversity of attention heads, improving both interpretability and performance on dense prediction tasks such as zero-shot image segmentation.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2611.11890

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

yuji-nozawa/ReFoCo

Image Feature Extraction •

Updated about 3 hours ago

•

2

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.11890 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.11890 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co