跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 15 天前AI 評分46

MotionGraphicsBench:以程式化驗證評估動態圖形生成

MotionGraphicsBench: Evaluating Motion Graphics Generation with Programmatic Verification

AI 導讀

MotionGraphicsBench以100個專家策劃案例,評估大型語言模型根據文字提示為靜態SVG生成動態圖形的能力。其確定性驗證程式與人類判斷的一致率為93.3%,高於GPT-6 Astra VLM基準的83.3%;涵蓋32個LLM、12家供應商的47種配置,pass@1介乎6.0%至67.7%。

正文

Published on Nov 7

Authors:

,

,

Abstract

Large language models can generate motion graphics from static images and natural language requests, but it remains difficult to evaluate how well models can follow requests that combine spatial, appearance, and temporal requirements. We introduce MotionGraphicsBench, an expert-curated benchmark of 100 instances for animating an input static SVG with a motion edit prompt. By supplying all required visual elements, our editing task reflects a designer-like workflow and isolates motion generation from graphic generation. Derived from real-world Lottie motion designs, our benchmark contains diverse combinations of spatio-temporal animation behaviors. Each benchmark instance pairs its model input with a deterministic, human-readable verification program that accepts different valid implementations and reports which prompt requirements pass or fail, rather than comparing against a reference animation or using a VLM judge. We construct these programs with a multi-agent pipeline using human-labeled animations and synthesized test cases. On a balanced test set of 180 animations, our programs achieve 93.3% raw agreement with human judgments, compared with 83.3% for a GPT-6 Astra VLM baseline, and outperform the baseline in both approval and rejection recall. Across 47 configurations from 32 LLMs and 12 providers, pass@1 ranges from 6.0% to 67.7%. Across 15 models evaluated at their highest available reasoning effort with 32 attempts per instance, additional attempts improve every model, but pass@32 still ranges from 38.0% to 95.0%. Our analysis of verification programs reveals that generated animations often fail to match reference poses, follow requested motion trajectories, or coordinate related events. We release our benchmark at https://huggingface.co/datasets/figma-ai/motion-graphics-bench.

View arXiv page View PDF Project page Add to collection

Get this paper in your agent:

hf papers read 2611.10592

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.10592 in a model README.md to link it from this page.

Datasets citing this paper 1

figma-ai/motion-graphics-bench

Viewer •

Updated about 17 hours ago

•

53.3k

•

54

•

5

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.10592 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co