跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-10AI 評分41

從文字到現實:互動式實體環境中的大語言模型評測

Words to World: Benchmarking LLMs in Interactive Physical Environments

AI 導讀

研究提出一套基準,評測大語言模型在真實環境中的閉環實體互動能力。系統連接機械臂、相機和感測器;模型須發出指令、接收環境回饋、判讀狀態並調整後續行動。實驗分析模型將語言推理轉化為實體行動的能力,並指出模型幻覺出狀態變化、應對意外結果時復原不佳,以及長期計劃不穩定等常見失誤。

正文

Published on Oct 10

Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

Words are increasingly becoming an interface for action: large language models (LLMs) can now invoke tools, operate devices, and make decisions that affect the physical world. However, current benchmarks largely evaluate LLMs through static text responses, offline tool calls, or simulated tasks, leaving open the question of how well they can act in real physical environments. We present a benchmark for measuring LLMs in closed-loop physical interaction, where models control a harness system connected to real-world components including robotic arms, cameras, and sensors. In each task, the model must issue instructions, receive physical feedback from the environment, interpret the resulting state, and adapt its next action accordingly. This setting evaluates capabilities beyond language understanding, including grounded perception, action planning, tool selection, spatial reasoning, feedback-driven correction, and robustness to execution uncertainty. Through experiments with representative LLMs, we characterize their ability to translate linguistic reasoning into physical actions and identify common failure modes such as hallucinated state changes, poor recovery from unexpected outcomes, and unstable long-horizon plans. Our benchmark aims to provide a systematic foundation for evaluating LLMs as interactive agents that connect words, tools, and the physical world.

View arXiv page View PDF Add to collection

Get this paper in your agent:

hf papers read 2610.15228

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.15228 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.15228 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.15228 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co