機械人的記憶只是時鐘嗎?記憶增強型 VLA 策略中的時間捷徑
Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies
研究在 RoboMME 計數任務中發現,記憶增強型視覺語言動作(VLA)策略按時間索引記憶,可能把無效等待當作進度,令機械人提早停止。加入短暫停頓或手腕抖動後,兩項任務的成功數分別由 47/50 降至 16/50、由 44/50 降至 10/50;把這些畫面從記憶中移除後,成功表現恢復。以事件索引記憶可在不重新訓練下挽回至少一半損失,而加入暫停訓練未能消除問題並損害計數表現。
Published on Nov 10
Authors:
Abstract
Memory in vision-language-action policies can mistake pauses for progress because it is indexed by time, causing early stopping, while event-based indexing mitigates this without retraining.
Memory-augmented vision-language-action (VLA) policies are meant to remember what happened. Many of them, however, fill memory by time: they sample frames uniformly across the episode and stamp each frame with its time step. Because time passes even when nothing happens, such a memory may not tell waiting from working. We show that it does not. On the counting tasks of RoboMME, we make the robot briefly pause or fidget its wrist. Such a null event changes nothing the task depends on, yet a released memory VLA counts the interval as work done and stops early. We call this failure phantom progress. It cuts success from 47 to 16 of 50 on one task and from 44 to 10 on the other. When the same frames are withheld from memory, success returns, so the cause lies in memory rather than in the interruption itself. We trace the failure to a temporal shortcut: because demonstrations proceed at a steady pace, elapsed time predicts progress. Indeed, rescaling the time stamps alone makes the policy stop early when the clock runs fast and repeat when it runs slow. Surprisingly, removing the time encoding does not cure the failure, because the frames of a null event still occupy slots of the time-sampled memory. Across RoboMME's released designs, vulnerability follows how memory is indexed. Of the fixes the diagnosis suggests, training with injected pauses does not remove the failure and impairs counting, whereas indexing memory by events recovers half or more of the loss without retraining. A memory meant to remember what happened should count events rather than clock ticks.
View arXiv page View PDF GitHub Add to collection
Get this paper in your agent:
hf papers read 2611.14488
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 1
Robotics • Updated 8 minutes agobenni23/memory-clock
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.14488 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.14488 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co