跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-19AI 評分41

RoboValue:用於統一評估機械人價值模型的細粒度模擬與真實世界基準

RoboValue: A Fine-Grained Sim-and-Real Benchmark for Unified Evaluation of Robotic Value Models

AI 導讀

RoboValue 提出細粒度模擬與真實世界基準,統一評估機械人價值模型的四項能力。RoboValue-Dataset 涵蓋 15 項模擬及 20 項真實世界操作任務,並提供專家示範和標註軌跡。測試發現,模型在指令細節理解、依賴歷史的進度追蹤及恢復評估方面仍有錯誤。

正文

Published on Oct 19

Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

RoboValue introduces a benchmark to evaluate robotic value models across task-state understanding, progress tracking, failure recovery, and consistency in both simulated and real-world settings.

Robotic value models assess task execution from visual observations and instructions, providing feedback for policy learning and execution monitoring. However, existing evaluations cover only selected aspects of execution understanding, leaving the reliability of value feedback insufficiently tested. We introduce RoboValue, a fine-grained sim-and-real benchmark for unified evaluation of robotic value models. Its framework evaluates four complementary capability dimensions through shared interfaces: task-state understanding, temporal progress monitoring, failure and recovery reasoning, and value consistency. Diagnostic trajectories and instruction variations test whether value judgments accurately reflect task requirements and changes throughout execution. RoboValue-Dataset covers 15 simulation and 20 real-world manipulation tasks, providing expert demonstrations for training and annotated trajectories for fine-grained evaluation. We evaluate representative robotic value models in zero-shot and one-shot settings under standard conditions and separate shifts in robot embodiment and environment. Our results reveal errors in fine-grained instruction grounding, history-dependent progress tracking, and recovery assessment, even in models with strong outcome and forward-progress scores. The same semantic subtask can also receive substantially different value gains across valid execution orders. These findings identify concrete targets for value supervision, while the dataset and public leaderboard support continued assessment of feedback reliability. The website is available at https://robovalue-benchmark.github.io/.

View arXiv page View PDF Project page GitHub 13 Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.26663 in a model README.md to link it from this page.

Datasets citing this paper 1

wzlllong/value-model-benchmark-training-data

Viewer •

Updated about 3 hours ago

•

10.6k

•

571

•

2

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.26663 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co