GaussianRenderFormer:以 Transformer 架構渲染 3D Gaussian splats
GaussianRenderFormer: Transformer-based Rendering of 3D Gaussian Splats
GaussianRenderFormer 是一種 Transformer,可接收一組 Gaussians 和一部相機,無需光柵化器便直接輸出圖像。模型以 26,820 個 Objaverse 物件訓練,每個物件最多保留 20,000 個 Gaussians;在 300 個未見過的物件上,PSNR 為 38.08 dB,而將相同的精簡輸入光柵化則為 44.02 dB。
Published on Nov 3
·
on Nov 5
Authors:
Abstract
Can a neural network learn to render 3D Gaussian splats on its own, with no rasterizer in the renderer? We present GaussianRenderFormer, a transformer that takes a set of Gaussians and a camera and outputs the image directly. Each Gaussian becomes one token in a view-independent encoder, and a view-dependent decoder turns small patches of camera rays into pixels by attending to those tokens. Trained on 26,820 Objaverse objects pruned to 20,000 Gaussians each, GaussianRenderFormer renders 300 unseen objects at 38.08 dB PSNR against the full, unpruned splats, while rasterizing the same pruned input reaches 44.02 dB. The gap is not one of generalization, and our diagnostics are consistent with a bottleneck in the view decoder: telling it the projected footprint of each Gaussian, which it could in principle infer, adds 1.4 dB. Under-training is not ruled out. Code, trained models and videos: https://svlwoof.github.io/gaussianrenderformer
View arXiv page View PDF Project page GitHub 0 Add to collection
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co