跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 18 天前AI 評分47

先解碼再聆聽:分塊交錯式全雙工語音語言模型的 Speculative Listening

Decode Before You Hear: Speculative Listening for Chunk-Interleaved Full-Duplex Speech Language Models

AI 導讀

Speculative Listening 是一種免訓練、無損的推測解碼框架,利用分塊交錯式全雙工語音語言模型的閒置運算,預先草擬並驗證後續音訊區塊。在八項任務的 TickTock-Bench 上,它令語音區塊加速 2.6-2.8x、工具呼叫區塊最高加速 23.4x,並將回應開始前的解碼延遲平均降低 73% 至 74-241 ms。

正文

Published on Nov 4

·

Submitted by

zly

on Nov 6

Authors:

,

,

,

,

,

,

,

Abstract

A training-free, lossless speculative decoding framework exploits idle compute in chunk-interleaved full-duplex speech language models to draft and verify upcoming audio chunks, substantially reducing response and tool-call delays.

Chunk-interleaved full-duplex speech language models listen and speak in fixed time chunks, and each chunk must be processed before the next one arrives. Their workload is asymmetric across chunks: listen chunks leave compute idle while awaiting audio, and speak and tool-call chunks require serial generation, which delays responses and tool calls. Using the idle compute for the next chunk requires decoding it before its audio arrives, under an assumed input. Taking silence as the assumed input and comparing the output with the output under real audio on four models, we observe two properties. First, the next audio chunk does not change the output in 95.3% of chunks. Second, 88% of changes begin at the first output token. Building on these properties, we propose Speculative Listening, a training-free, lossless framework for input-time self-speculative decoding. During idle time, the model drafts the next chunk under silence and, for changes at the first token, a second branch from the runner-up first token. It keeps all drafts in a cross-chunk hypothesis pool and verifies them when the audio arrives. On our eight-task benchmark TickTock-Bench, Speculative Listening speeds up speak chunks by 2.6-2.8x and tool-call chunks by up to 23.4x. On average, it reduces the decoding delay before a response starts by 73%, to 74-241 ms, close to the 200 ms gap between human turns, and the maximum lag behind real time by 94.4%.

View arXiv page View PDF Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.06982 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.06982 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.06982 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co