跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 12 天前AI 評分37

ExRL:透過執行長度控制,以高樣本效率在線適應長動作區塊策略

ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control

AI 導讀

ExRL 以輕量級適配器預測每個動作區塊的執行長度,並透過離策略強化學習訓練,提升長動作區塊機械人策略在線適應的樣本效率。模擬基準測試及真實機械人任務結果顯示,ExRL 可適應多種長動作區塊策略,且只需少量任務專屬超參數調整。

正文

Published on Nov 10

·

Submitted by

Xu Luo

on Nov 11

Authors:

,

,

,

,

,

,

Abstract

Execution at the Right Length (ExRL) enables sample-efficient adaptation of long-chunk robot policies by training a lightweight adapter to predict each chunk’s execution length using off-policy reinforcement learning.

Predicting long action chunks, often spanning dozens of timesteps, has become a key ingredient in increasingly capable robot policies. In this paper, we find that adapting these long-chunk policies with existing reinforcement learning (RL) methods can require many online samples due to the high dimensionality of the action chunks being optimized, making such adaptation impractical in real-world scenarios where interaction data are costly to collect. Our key insight is to shift adaptation from optimizing actions to optimizing how long to execute each predicted action chunk. This keeps the adaptation space low-dimensional while its expressivity grows with the chunk size. We thus introduce Execution at the Right Length (ExRL), a simple approach which augments the base policy with a lightweight adapter that predicts the execution length of each chunk and is trained using off-policy RL algorithms. Across simulation benchmarks and real-world robotic tasks, ExRL enables sample-efficient adaptation of a diverse range of long-chunk policies with minimal task-specific hyperparameter tuning.

View arXiv page View PDF Project page GitHub 1 Add to collection

Get this paper in your agent:

hf papers read 2611.13482

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.13482 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.13482 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.13482 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co