VibePrevisBench:評測智能體編碼生成 Blender 4D 場景的基準
VibePrevisBench: Evaluating Agentic Coding for 4D Scene Generation in Blender
VibePrevisBench 是評測智能體把簡短描述轉化為可執行 4D 場景並渲染代理影片的基準,涵蓋 242 項任務及四種輸入類型。研究以規則式 Scene Check 和 VLM 式 Screen Check 評估場景與影片,並分析 11 個前沿編碼智能體:它們在空間和鏡頭運動理解方面表現有別,亦未能有效修正渲染錯誤。
Published on Nov 11
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Controllable real-world video generation models can condition on a rendered graybox proxy video, which fixes layout, motion and camera. Coding agents can build such videos by reconstructing a 4D scene in Blender, yet no benchmark checks both these videos and the long, multi-shot scenes behind them. We introduce VibePrevisBench, a benchmark for 4D scene reconstruction: an agent turns a brief into an executable 4D scene and renders its proxy video. Its 242 tasks span four input kinds, including scripts, photos, continuations, and reference videos. Each delivery is scored along two evaluation dimensions. Rule-based Scene Check re-runs the code and compares the exported 4D scene with hidden ground truth facts in nine dimensions. VLM-based Screen Check asks a VLM judge whether the facts that pass can be recognized in the proxy video. We analyze the evaluation results and trajectories of 11 frontier coding agents and identify three key observations. First, multimodal understanding separates the agents, especially in understanding space and camera motion. Second, all agents seem to already master basic camera work and staging, and fail at challenging requests. Third, agents inspect their own renders but fail to correct their errors. Agents thus need better visual understanding of space and camera, explicit checks for challenging requests, and item-by-item verification of renders against the brief.
View arXiv page View PDF Project page GitHub 10 Add to collection
Get this paper in your agent:
hf papers read 2611.15789
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.15789 in a model README.md to link it from this page.
Datasets citing this paper 1
Kling-Keye/vibeprevisbench
Viewer • Updated about 5 hours ago • 242 • 2.04k • 3
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.15789 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co