Lab2Sim:將真實世界實驗影片轉化為互動式虛擬實驗
Lab2Sim: Turning Real-World Experiment Videos into Interactive Virtual Experiments
Lab2Sim 提出一項把科學實驗影片轉化為互動式虛擬實驗的基準,涵蓋 218 條影片及 11,881 個評分項目。EduVideo2Code 將實驗程序編碼為明確狀態機,並透過執行驗證改善生成頁面;在兩個骨幹模型上,四個評估維度均有提升,器材保真度增加 24.15 分。
Published on Nov 2
Authors:
,
,
Abstract
Lab2Sim introduces a benchmark for converting science videos into interactive virtual experiments, and EduVideo2Code improves apparatus fidelity by encoding procedures as state machines for verified code generation.
Science experiment videos demonstrate how experiments are performed, but offer no direct interaction for learners. Turning them into virtual experiments requires not only reproducing the observed apparatus and state changes, but also enforcing the preconditions governing each operation. We introduce Lab2Sim, a benchmark of 218 science experiment videos with evidence-grounded specifications and 11,881 rubric items. To address this task, we propose EduVideo2Code, which extracts experimental procedures hierarchically, encodes them as explicit state machines to guide code generation, and refines the resulting pages through execution-based verification. Experiments show that existing multimodal models readily generate runnable pages but struggle to faithfully reconstruct experimental apparatus. EduVideo2Code consistently improves all four evaluation dimensions across two backbone models, with apparatus fidelity gains of 24.15 points.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2611.02928
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.02928 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.02928 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.02928 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co