跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 6 天前AI 評分41

DNeo:並行草稿模型需要更佳的目標特徵編碼

DNeo: Your Parallel Draft Model Needs Better Target Feature Encoding

AI 導讀

DNeo 改良並行草稿模型的目標特徵編碼,以提升大型語言模型推理速度。其架構結合 Mixture-of-Features Attention(MoFA)、拒絕記憶、2-tap 因果卷積、僅前綴交叉注意力及非均勻目標層選擇。相較 SoTA(DFlash-2),DNeo 的生成速度快 1.3x,且不影響草稿模型規模或生成質素。

正文

Published on Nov 16

Authors:

,

,

,

,

,

,

Abstract

A lightweight feature-encoding method with rejection memory and architectural enhancements enables faster large language model inference by improving speculative-diffusion draft verification.

Speculative-diffusion decoding uses a small parallel drafter model to generate b draft tokens in one forward pass, which then undergo parallel verification via a larger target model. Faster inference can be achieved when multiple draft tokens pass verification simultaneously. Recent parallel draft architectures (DFlash, DSpark) improve draft quality by conditioning on rich features computed by the target, through a mechanism known as feature encoding. However, standard feature encoding mechanisms have three limitations: (1) they are lossy, applying encoding projections that are destructive to the underlying target feature-space via a discovered effect known as residual mixing, (2) they are memory-less, with no record of previously processed proposals, and (3) they utilize features sampled uniformly from across the target layers, omitting important hidden states. To address these issues, we propose Mixture-of-Features Attention (MoFA), a lightweight feature-encoding approach equipped with a working memory that encodes previously rejected proposals. Leveraging MoFA, rejection memory, and a set of further enhancements, including 2-tap causal convolutions, prefix-only cross-attention, and non-uniform target layer selection, the final architecture, DNeo, achieves 1.3x faster generation speed than SoTA (DFlash-2) without impacting drafter footprint or generation quality. DNeo marks a new state of the art in high quality, fast LLM inference.

View arXiv page View PDF Project page GitHub 4 Add to collection

Get this paper in your agent:

hf papers read 2611.21925

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

J-sandler/Qwen3-4B-DNeo

Feature Extraction •

0.6B •

Updated about 15 hours ago

•

25

•

1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.21925 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.21925 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co