CoSTA:透過壓縮語音 token 適配提升語音對話模型效率
CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models
CoSTA 透過解耦語音對話模型的生成與合成 token 空間,在保留原有語音 tokenizer 和 decoder 的同時,提升生成效率。雙語語音對話基準測試顯示,CoSTA 的擴展語音 token 生成速度約提升 1.7 倍,完整回應延遲更低、吞吐量更高,並維持回應質素。
arxiv:2610.21143
Copy markdown
CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models
Published on Oct 15
·Submitted by Enjoyon Oct 30
·
Chinese Academic of Science Institute of Automation
Authors:
Chong Li,
Bingxin Liu,
Yi Shu,
Tianyu Peng,
Can Xie,
Siyao Song,
Shuo Ren,
Jiajun Zhang
Abstract
CoSTA decouples generation and synthesis token spaces to accelerate spoken dialogue models while preserving quality and compatibility.
Generated by thinkingmachines/Inkling-Small
Spoken Dialogue Models (SDMs) typically share a speech-token space for generation and synthesis, leading to long autoregressive sequences and costly inference. Lower-rate speech tokenizers shorten sequences but may introduce compression-quality trade-offs and costly replacement without breaking this coupling. To address this limitation, we propose CoSTA, a compatibility-preserving post-hoc framework that decouples generation and synthesis token spaces while retaining the original speech tokenizer and decoder. CoSTA extends the generation token space with compressed representations via mean-initialized vocabulary extension, pronunciation adaptation, and dialogue adaptation, followed by deterministic expansion back to the original speech-token space before synthesis. On bilingual speech-dialogue benchmarks, CoSTA achieves approximately 1.7times expanded speech-token generation speed, lower full-response latency, and higher throughput while preserving response quality. Analyses show more sustained interaction under limited budgets and a practical trade-off among compression, decoding efficiency, and quality. Further experiments demonstrate cross-architecture transfer and complementarity to low-rate tokenization.
View arXiv pageView PDFGitHubAdd to collection
Community
Paper author Paper submitter12 minutes ago
Excited to share CoSTA, a post-hoc adaptation framework for efficient spoken dialogue models!
Unlike conventional approaches that reduce speech token rates, CoSTA decouples generation and synthesis token spaces through reversible BPE-based compression, without replacing the original speech tokenizer or decoder.
🚀 Key highlights:
- ~1.7× faster expanded speech-token generation
- Lower response latency and higher inference throughput
- Preserved response quality across Chinese and English
- Compatible with existing speech tokenizers and complementary to low-rate tokenization
📄 Paper: https://arxiv.org/abs/2610.21143
💻 Code: https://github.com/yingzhuodeng/CoSTA
Feedback and discussions are welcome!
Reply
Edit Preview
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Comment ·Sign up or log in to comment
Get this paper in your agent:
hf papers read 2610.21143
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.21143 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.21143 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.21143 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
System theme
Company
Website
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co