跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-08AI 評分37

投機解碼中平行草稿的上下文校正

Contextual Correction of Parallel Drafts for Speculative Decoding

AI 導讀

DCOCO(DFlash Contextual Corrector)透過平行評估相鄰草稿位置的候選 token,恢復平行草稿內的依賴關係,提升投機解碼的上下文選詞能力。在 4 種驗證器骨幹及 10 項基準測試的各項評估配置中,DCOCO 平均吞吐量均最高,較 vanilla decoding 加速最高達 6.6x。在 4-32 個同時處理請求下,DCOCO 的表現較最強競爭方法高 6-23%。

正文

Published on Oct 8

Authors:

,

,

,

,

,

,

Abstract

Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this paradigm particularly attractive: deep attention-based neural backbones provide strong predictions, while drafting multiple future positions in parallel avoids the sequential cost of autoregressive generation. This parallelism, however, comes at a cost. Tokens within a draft block are predicted without access to the realized identities of preceding draft tokens, removing intra-block dependencies that would naturally be available under autoregressive generation. We introduce DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores these dependencies while preserving parallel drafting. Rather than committing to each position's top prediction, DCOCO retains multiple candidate tokens and evaluates their contextual compatibility across adjacent draft positions in parallel, enabling contextually informed token selection at low additional cost. Across four verifier backbones and ten benchmarks, DCOCO achieves the highest average throughput in every evaluated configuration, spanning greedy and stochastic decoding with and without extended reasoning, with speedups of up to 6.6x over vanilla decoding. Its advantage persists under concurrent serving: DCOCO outperforms the strongest competing method by 6-23% across 4-32 concurrent requests, while other sequential correctors fall below uncorrected DFlash at these loads. These results demonstrate that recovering intra-block dependencies through parallel contextual scoring delivers consistent acceleration across decoding regimes and serving loads.

View arXiv page View PDF GitHub 2 Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.12570 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.12570 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.12570 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co