按策略蒸餾只需少量額外數據
On‑policy distillation works with limited additional data
AutoDAgger 在線恢復教師模型的操作軌跡，讓學生策略在執行過程中吸收教師的糾正行為，無需耗費人力監督，也省去獨立收集數據的階段。其數據收集速度比 HG-DAgger 快逾 2 倍；學生策略使用 30 個回合後，分數由 27.8 升至 63.9，90 個回合後達 77.8。
AutoDAgger recovers teacher trajectories online, letting compact policies inherit large‑model skill while requiring only modest extra data. By embedding the teacher’s corrective behavior directly into the student’s rollout loop, the method sidesteps the costly separate data‑collection stage that has dominated prior distillation pipelines [1].
Before this work, practitioners relied on pure behavior cloning from static human demonstrations or offline distillation that frozen the teacher’s expertise before any interaction with the environment. Human‑in‑the‑loop DAgger variants such as HG‑DAgger still needed a person to monitor progress and intervene whenever the learner deviated, inflating both labor and dataset size. These approaches made scaling down high‑capacity policies to edge devices an engineering bottleneck.
AutoDAgger collects recovery data more than 2 × faster than HG‑DAgger, eliminating the human monitoring loop entirely [1]. The autonomous reward model watches task progress and triggers teacher takeovers only when necessary, turning what used to be a manual handoff into a rapid self‑service pipeline.
With just 30 AutoDAgger episodes the student jumps from a baseline score of 27.8 to 63.9, and after 90 episodes it reaches 77.8, showing that a handful of on‑policy recovery rollouts suffice for dramatic improvement [1]. This efficiency suggests that a substantially smaller dataset can achieve performance levels comparable to those obtained with larger offline collections, as demonstrated by the modest additional data used in the experiments.
TILDE achieves scores of 80.6 on the Pick Pen task (and 63.9 and 69.2 on Fold Dishcloth and Pick Beverage), outperforming all evaluated baselines on each benchmark [1]. Those numbers sit squarely in the range of the original embodied foundation models, confirming that on‑policy distillation can transfer near‑teacher capability without exhaustive pretraining.
The study evaluates only three manipulation tasks and assumes a reliable reward model to signal failure, leaving open whether the same speedup and performance gains hold for more complex or less well‑instrumented domains. One unanswered question is how AutoDAgger scales when teacher interventions become sparse or when the reward signal is noisy; future work will need to probe robustness in those regimes.
If autonomous recovery truly replaces human‑in‑the‑loop data collection, benchmark pipelines that currently amass tens of thousands of trajectories should be re‑run with AutoDAgger, expecting comparable scores at a fraction of the data cost. This shift could make policy distillation practical for on‑device deployment across a wide array of robotics applications.
References
來源：DEV Community · dev.to