AMBIT 以可執行計劃及自蒸餾 Transfusion 生成與編輯空間音訊
Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion
AMBIT 是一套統一框架,透過自回歸場景規劃與 rectified-flow 渲染,生成及編輯一階 Ambisonics(FOA)音訊。框架以可執行 ScenePlans 描述聲源內容、活動及 3D 軌跡,並結合 FOA 專用連續 VAE 和 on-policy 自蒸餾。評測顯示,AMBIT 在語音、音樂及音效生成的內容與空間品質,以及指令引導編輯的音質保真度和未編輯聲源保留方面,優於比較基線。
Published on Oct 25
Authors:
,
,
,
,
,
Abstract
AMBIT is a unified framework that generates and edits first-order ambisonics audio by coupling autoregressive scene planning with rectified-flow rendering, using a specialized continuous VAE and on-policy self-distillation to preserve spatial structure and acoustic fidelity.
Spatial audio authoring coordinates source decisions with soundfield synthesis. For editing, plans specify what to change and retain, while reference audio supplies further acoustic detail. We present AMBIT, a unified framework for native First-Order Ambisonics (FOA) generation and editing with three connected contributions. First, a shared Transfusion backbone couples autoregressive planning with rectified-flow rendering through executable ScenePlans that specify source content, activity, and 3D trajectories. For editing, the plan describes the full target scene, including retained sources; contrastive features ground the requested change, while full reference latents supply residual acoustic detail for preservation. Second, an FOA-aware continuous VAE combines shared-decoder omnidirectional recovery, asymmetric grouped KL, and spatial covariance matching to preserve content and inter-channel structure. Third, joint Transfusion on-policy self-distillation (OPSD) refines planning on student-generated prefixes and rendering at sampling states reached under student plans, with target-conditioned teachers supervising both modes of the shared student. Evaluations across speech, music, and sound show improved generation content and spatial quality, acoustic fidelity during instruction-guided editing, and unedited-source preservation over the compared baselines. Project page: https://wjm-george.github.io/AMBIT/
View arXiv page View PDF Project page GitHub 0 Add to collection
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.34415 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.34415 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.34415 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co