跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 28 天前AI 評分47

太空中的人類:MLLMs 能否透過主動外視角探索建立以人為中心的空間理解?

Humans in Space: Can MLLMs Construct Human-Centric Spatial Understanding via Active Exocentric Exploration?

AI 導讀

研究提出 SATE-Bench,評估 MLLMs 在以人為中心的場景中能否主動探索視角,以建立空間理解。基準包含 316 個真實場景及 2,525 組多輪問答;評測發現,模型常在證據不足時提早停止探索,且難以處理精細的 3D 幾何任務。以 GRPO 微調雖增加主動探索,卻降低整體準確度。

正文

Published on Oct 25

Authors:

,

,

,

Abstract

While Multimodal Large Language Models (MLLMs) excel at zero-shot reasoning, their spatial capabilities are primarily evaluated through static, passive observation. In complex, human-centric scenes, occlusion often hides crucial geometric evidence, and passive benchmarks force models to guess rather than recognize their own visual uncertainty. To bridge this gap, we introduce SATE-Bench, a novel interactive benchmark comprising 316 real-world scenes and 2,525 multi-turn QA pairs to evaluate visual self-awareness and active exocentric view-exploration. It combines a Passive multi-view control with an Active exploration setting, where models start from a severely occluded view and must iteratively query adjacent viewpoints until they gather enough evidence to fully ``sate'' the question, scored by a step-penalized efficiency reward. Evaluating state-of-the-art MLLMs reveals that they terminate exploration prematurely while reporting high confidence, and struggle with fine-grained 3D geometric tasks; matched-view controls further attribute the Active--Passive gap primarily to insufficient evidence acquisition. Moreover, fine-tuning with Group Relative Policy Optimization (GRPO) shows that rewarding view-seeking increases active exploration but lowers overall accuracy, as models over-explore and fail to fuse expanding multi-view contexts across turns. These findings indicate that scaling passive multi-view perception does not intrinsically yield active spatial reasoning, highlighting the need for models that know where to look, how to integrate what they see, and when to stop. Code and data are available at https://gtc-gh.github.io/SATE-Bench/

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.34149

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.34149 in a model README.md to link it from this page.

Datasets citing this paper 1

gtc-hf/SATE-Bench

Viewer •

Updated about 1 hour ago

•

2.06k

•

35

•

1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.34149 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co