跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-09-26AI 評分45

DoSA: Dots Sparse Attention

DoSA: Dots Sparse Attention

AI 導讀

DoSA 提出一種無損稀疏注意力方法,透過條件式計算減少索引器的完整評分,同時保留精確的 token 選擇。GPU 實作在不同工作負載下,完整 Indexer 相較 SGLang 最快提升 2.74 倍、相較 LiteTopK 提升 1.69 倍;在 GLM-5.2 和 DeepSeek-V3.2 上,首次 token 延遲最高分別提升 1.94 倍和 1.50 倍。

正文

Abstract

Sparse attention visits few keys, yet finding them can still require scoring every key with every indexer head. We introduce DoSA (Dots Sparse Attention), a lossless method for making this computation conditional while preserving exact token selection. Its central observation is that Top-K selection requires sufficient evidence to reject a candidate, not necessarily its full score. A seed set supplies a lower bound on the winning threshold. Nonnegative head projections and joint residual bounds upper-bound positive contributions, while activation-informed aggregation bounds the magnitude subtracted by negative heads. The certificate directs full scoring toward candidates that remain eligible; exact-score filtering further reduces writeback and selection work. We establish the certificate and its exact-selection conditions, and analyze how geometry and selection margins govern its effectiveness. Our GPU implementation accelerates the complete Indexer by up to 2.74 times over SGLang and 1.69 times over LiteTopK across English, Chinese, and code workloads. End-to-end evaluation on GLM-5.2 and DeepSeek-V3.2 achieves up to 1.94 times and 1.50 times time-to-first-token speedups over the two baselines, respectively.

View arXiv page View PDF GitHub 5 Add to collection

Get this paper in your agent:

hf papers read 2610.13990

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.13990 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.13990 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.13990 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co