跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 8 天前AI 評分44

LoopFill:面向 Looped Transformers 的流水線式多 token 推測解碼

LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers

AI 導讀

LoopFill 是一種免訓練的推測解碼演算法,將多 token 後綴草稿與淺層循環預測合併至同一草稿樹,並透過完整深度驗證保留自回歸輸出分佈。七項基準測試中,LoopFill 在 Ouro-2.6B 和 Huginn-3.5B 上較貪婪自回歸解碼分別加速 5.85 倍和 5.13 倍,較最強基線 LoopAhead 分別快 40% 和 25%。

正文

Published on Nov 14

Authors:

,

,

Abstract

LoopFill reduces decoding latency for looped language models by combining multi-token suffix drafts and shallow-loop predictions into a single draft tree with full-depth verification that preserves the autoregressive output distribution.

Looped Transformers improve parameter efficiency by repeatedly applying the same Transformer layers. During autoregressive decoding, however, generating each token requires several sequential passes through these layers, leading to high decoding latency. Pipelined self-speculative methods use shallow-loop predictions to process tokens at different loops in the same batch, but each token must reach the drafting depth before the next can enter the pipeline. We introduce LoopFill, a training-free speculative decoding algorithm for looped LLM that combines multi-token suffix drafts with shallow-loop predictions in a single draft tree. Both drafters contribute to one tree, so a suffix lookup can immediately extend a shallow prediction and several future tokens can enter the pipeline together. The shared block computes the highest-scoring draft candidates at different loop depths in one batch, and full-depth verification preserves the model's autoregressive output distribution. Across seven benchmarks, LoopFill's average speedup over greedy autoregressive decoding is 5.85times on Ouro-2.6B and 5.13times on Huginn-3.5B. The speedups exceed those of LoopAhead, the strongest baseline, by \(40\%\) on Ouro-2.6B and \(25\%\) on Huginn-3.5B.

View arXiv page View PDF Project page Add to collection

Get this paper in your agent:

hf papers read 2611.19649

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.19649 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.19649 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.19649 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co