JoVox:用於輕量化體素生成的聯合階層式 Flow Matching
JoVox: Joint Hierarchical Flow Matching for Lightweight Voxel Generation
JoVox 提出離散 Flow Matching 框架,聯合學習體素階層各層的分佈,讓較細單元參與較粗單元的預測,提升生成品質。相較目前的 SOTA TRELLIS.2,JoVox 的訓練吞吐量高 8 倍,單次前向傳播的 FLOPs 為其 1/15,使用 4 張而非 32 張 GPU 訓練,Chamfer distance 亦低 11%。
Abstract
Generating high-fidelity voxels while keeping their lightweight nature remains an open problem in computer vision. Current generators are either high-fidelity but computationally heavy, or lightweight but of lower fidelity. Hierarchical methods keep the lightweight nature of voxels. However, their coarse levels, which decide where all finer cells can appear, are generated with the least information about the shape and are never revisited, so they are often plausible but wrong. Our key insight is to let finer cells take part in deciding coarse cells. To this end, we propose JoVox, a discrete flow matching framework that learns the joint distribution of all levels of the voxel hierarchy, instead of a separate distribution for each level, so that coarse cells can be predicted with finer cells already generated nearby. Within this framework, we further introduce a two-level prediction, which outputs the cells two levels finer at once instead of one level finer, halving the number of levels that the network processes and yielding voxels of higher fidelity. We also introduce an unmasking strategy that revisits early decisions once finer cells are generated and corrects inconsistencies in masked diffusion models. Our framework outperforms the baselines on both unconditional and image-conditioned voxel generation. Compared with the current state-of-the-art TRELLIS.2, we achieve 8x higher training throughput, 1/15 of the FLOPs in a forward pass, train on 4 vs. 32 GPUs, and obtain higher quality with an 11% lower Chamfer distance.
View arXiv page View PDF Project page GitHub 1 Add to collection
Get this paper in your agent:
hf papers read 2611.10690
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 1
Image-to-3D • 2B • Updated 33 minutes ago • 1xiangjuntang/JoVox-64
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.10690 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.10690 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co