PAIR:銜接視覺-語言-動作模型中的感知與動作
PAIR: Bridging Perception and Action in Vision-Language-Action Models
PAIR 為連續動作的視覺-語言-動作模型提出明確中間表示,銜接視覺語言理解與可執行的機械人動作。該表示結合任務語義與專家動作結構,透過對齊動作潛在表示學習 Bridge Tokens,並提早注入 Action Expert;在 LIBERO、LIBERO-Plus、CALVIN 及真實世界操作任務中均提升表現。
Excited to share PAIR: Bridging Perception and Action in Vision-Language-Action Models ! 🚀
A key challenge in continuous-action VLAs is bridging the gap between visual-language understanding and executable robot actions. Instead of leaving this transition implicit, PAIR learns an explicit intermediate representation that combines task semantics with expert action structure.
Our key idea is to learn Bridge Tokens through alignment with action latents and inject them early into the Action Expert, providing an action-aware starting point for action generation.
PAIR consistently improves performance across LIBERO, LIBERO-Plus, CALVIN, and real-world manipulation tasks.
We hope PAIR inspires further exploration of explicit perception-action representations for VLA models. Feedback and discussions are very welcome!
📄 Paper: https://arxiv.org/abs/2610.09016
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co