ViT-HarDis:以困難樣本條件化的去相關注意力提升穩健人臉識別
Hard Sample-Conditioned Decorrelated Attention for Robust Face Recognition
ViT-HarDis按樣本難度調節注意力頭去相關,對身份較難區分的樣本施加更強去相關,並只在最後的 Transformer 層進行。以 WebFace4M 訓練時,該方法在 IJB-B、IJB-C 和 TinyFace Rank-1 的 ViT 方法中達到 SOTA;以 MS1MV3 訓練時,則在 IJB-B 和 TinyFace 取得最佳結果,且不需改動架構或增加推理成本。
Published on Oct 30
Authors:
,
,
,
Abstract
ViT-HarDis conditions attention head disentanglement on sample difficulty, applying stronger decorrelation to hard face-recognition samples and only in final transformer layers to improve recognition without architectural changes or extra inference cost.
Multi-head attention in Vision Transformers (ViTs) allows independent attention patterns across heads, but without explicit regularization, heads may learn redundant patterns rather than complementary roles. Correlation-based penalties encourage head disentanglement, yet apply uniform regularization across all training samples. In face recognition (FR), hard samples whose embeddings lie close to competing identities are critical for optimization and generalization. Motivated by this, we propose ViT-HarDis, which conditions attention head disentanglement on sample difficulty: samples with high cosine similarity to negative class centers receive stronger head decorrelation, enforcing specialization where identity ambiguity is highest. Experiments show improvements over both the baseline and uniform disentanglement on most benchmarks. Our analysis shows that ViT-HarDis reduces the similarity of hard samples to competing class centers while decreasing inter-head attention similarity, and that applying disentanglement only to the final transformer layers outperforms regularizing the entire network. Trained on WebFace4M, ViT-HarDis achieves SOTA results among ViT-based methods on IJB-B, IJB-C, and TinyFace Rank-1, while achieving the best IJB-B and TinyFace results when trained on MS1MV3, improving FR without architectural modifications or additional inference cost. Code: https://github.com/TaharChettaoui/Hard-Sample-Conditioned-Attention-for-FR.
View arXiv page View PDF Add to collection
Get this paper in your agent:
hf papers read 2610.41789
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.41789 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.41789 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.41789 in a Space README.md to link it from this page.
Collections including this paper 1
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co