跳到正文
上海人工智能實驗室 InternLM:原創項目·· 2026-10-06AI 評分29

InternLM/SpatialASM

InternLM/SpatialASM

AI 導讀

SpatialASM 是一套可擴展框架,透過兩階段訓練將主動空間心智化融入 LVLM,提升空間推理表現。框架把空間問答資料轉化為規劃、想像及推理軌跡,並以新視角生成器合成訓練觀察。團隊以 SpatialASM 訓練了四個 Qwen3-VL 和 Qwen3.5 的 LVLM 骨幹;最終模型推理時不依賴世界模型、渲染器或額外視覺觀察。

正文

📢 News

  • 🔥 SpatialASM is coming soon. Please stay tuned for the paper, models, data, and code.

🌈 Overview

Humans rarely solve difficult spatial problems by passively inspecting the views in front of them. Instead, we revisit a scene from task-relevant viewpoints and mentally reconstruct what should be visible there. We call this capability Active Spatial Mentalization (ASM).

We are thrilled to introduce SpatialASM, a scalable framework that internalizes ASM into LVLMs. Through the two-stage training on LVLMs, SpatialASM elicits active spatial exploration and imagination and enhances the overall spatial reasoning performance.

Teaser

SpatialASM consists of three complementary components:

  1. Allocentric Novel-View Generator. SpatialASM-NVG renders the scene from a target viewpoint specified by an anchor image, a pixel location, and a viewing direction. This allocentric formulation avoids lengthy sequences of camera-centric movements.
  2. ASM Trajectory Synthesis. Given a spatial QA sample, an LVLM plans informative viewpoints and the NVG renders the corresponding observations. After GeoP and answer-consistency filtering, rendered views are replaced by textual captions to form explicit plan-imagine-reason trajectories for SFT.
  3. View-Imagination Reinforcement Learning. Geometry-filtered candidate views are used to construct automatically verifiable perspective-taking tasks. GRPO further strengthens the association between a viewpoint description and its expected observation.

Pipeline

💡 Highlights

  • 🔥 Active spatial reasoning: SpatialASM jointly internalizes task-conditioned view planning and viewpoint-conditioned view imagination, enabling LVLMs to spontaneously seek and mentally construct missing spatial evidence.
  • 🔥 Scalable spatial CoT supervision: SpatialASM converts existing answer-only spatial QA data into explicit plan-imagine-reason trajectories and automatically verifiable perspective-taking tasks, alleviating the scarcity of spatial Chain-of-Thought supervision.
  • 🔥 Training-time spatial simulation: A novel-view generator acts as temporary training scaffolding. The final model does not depend on a world model, renderer, or additional visual observations at inference time.
  • 🔥 Grounded and reliable supervision: An allocentric viewpoint interface reaches a target perspective in one query, while the Geometry Perturbation (GeoP) metric and answer-consistency checks filter unreliable synthesized views and trajectories.

Comparison

📊 Results

We train four LVLM backbones of Qwen3-VL and Qwen3.5 with our SpatialASM framework, and the experimental results on both spatial reasoning and general-VQA tasks are shown below.

Exp

✒️Citation

If you find this project useful, please kindly cite:

@article{liu2026spatialasm,
  title={SpatialASM: Internalizing Active Mentalization for Spatial Reasoning},
  author={Yuhong Liu, Shenxi Wu, Junyi Chi, Beichen Zhang, Xuanlang Dai, Jiaqi Wang, Yuhang Zang, Dahua Lin},
  journal={arXiv preprint arXiv:2610.13832},
  year={2026}
}

📄 License

Code License Data License

Use of SpatialASM-NVG: The model SpatialASM-NVG is trained on FLUX.2-klein-9B, so please follow FLUX-NON-COMMERCIAL LICENSE.

Usage and License Notices: The data and code are intended and licensed for research use only.

❤️ Acknowledgement

The evaluation part is built on VLMEvalkit, a powerful toolkit to evaluate a vast range of LMMs!

The NVG training part is built on DiffSynth-Studio, an excellent codebase for Diffusion model training!

來源:上海人工智能實驗室 InternLM:原創項目 · github.com