跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 20 天前AI 評分39

SeMo3D 以選擇性建模統一 3D 生成、編輯與補全

SeMo3D: Unifying 3D Generation, Editing, and Completion via Selective Modeling

AI 導讀

SeMo3D 以選擇性建模統一 3D 生成、編輯與補全,透過區域選擇性去噪和多條件擴散 Transformer 保留不變內容,並支援無配對資料訓練。實驗顯示,SeMo3D 在三個基準測試的所有任務中均超越現有頂尖方法,並可融合圖像、文字及點雲引導。

正文

Abstract

SeMo3D unifies 3D generation, editing, and completion through selective region modeling, using region-selective denoising and a multi-condition diffusion transformer to preserve unchanged content and enable training without paired data.

3D generation, editing, and completion are typically treated as regenerating the complete 3D asset under task-specific constraints, even when only a local region needs modification. This leads to task isolation, inconsistency in content that should remain unchanged, and dependence on task-specific paired data. We address these issues with a key observation: the three tasks all synthesize a designated region while keeping the rest intact---the full region for generation, the target region for editing, and the missing region for completion. We term this paradigm Selective Modeling and build SeMo3D upon it; it naturally unifies the three tasks, preserves unchanged content by construction, and enables training without paired editing or completion data, since any 3D asset can be partitioned into a selected region and its context. For training, we propose region-selective denoising, which randomly selects regions of varying sizes on standard 3D assets and applies noise and loss only to them, jointly learning all three tasks. For architecture, we propose Omni3DiT, a multi-condition 3D diffusion transformer that distinguishes regions to create from those to retain and fuses image, text, and point guidance for fine-grained control. Experiments across three benchmarks show SeMo3D surpasses state-of-the-art methods on all tasks, delivering multimodal controllability in generation and strong consistency and accuracy in editing and completion. SeMo3D further serves as a unified modeling tool for agents, enabling efficient 3D creation through natural-language interaction. Project page: https://xiaokunsun.github.io/SeMo3D.github.io/.

View arXiv page View PDF Project page GitHub Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.02568 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.02568 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.02568 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co