跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-15AI 評分41

Lean4AgenticRL:以可驗證的多層級過程獎勵推進智能體強化學習

Lean4AgenticRL: Advancing Agentic RL with Verifiable Multi-level Process Rewards

AI 導讀

Lean4AgenticRL 將可驗證的多層級過程獎勵納入 Lean4 形式化框架,提升軟件工程及終端任務的智能體強化學習。該框架用於訓練 LAR-9B 和 LAR-27B,LAR-27B 在 TerminalBench-2.1 得分 77.3%。兩款模型平均較基礎模型高 9.89%,並較已知同等規模的第二佳模型高 5.07%。

正文

Published on Oct 15

Authors:

,

,

Abstract

Lean4AgenticRL integrates multi-level process rewards into a Lean4 formalization framework to improve agentic reinforcement learning for software engineering and terminal tasks, yielding stronger model performance.

Enhancing the agentic capabilities of Large Language Models (LLMs) to tackle complex software engineering (SWE) and general terminal-usage problems is a pressing challenge. Current state-of-the-art LLMs often go through large-scale agentic Reinforcement Learning (RL). However, they mostly rely on high-level outcome rewards, with minimal consideration of low-level toolcall and mid-level workflow-stage rewards, which makes the training less effective. To overcome this limitation, we propose **Lean4AgenticRL**, which, to the best of our knowledge, is the first framework that grounds multi-level process rewards in the unified Lean4 formalization schema for open-ended agentic RL. **Lean4AgenticRL** first builds a *problem-specific workflow* system that balances the agent's own capability and controllability during execution, laying the foundation for further verification. Subsequently, our workflow formalization and multi-level reward annotation methods enable agent behavior verification within a general harness system. It provides multi-level rewards to support more effective agentic RL with the workflow-stage-normalized GRPO algorithm. Using **Lean4AgenticRL**, we train **LAR-9B/27B**, a family of models with enhanced agentic capability. Extensive experiments indicate that our model outperforms the leading model of similar size. **LAR-27B** achieves **77.3%** on TerminalBench-2.1. On average, our models outperform the base model by **9.89%** and surpass the second-best model of comparable size we know by **5.07%**. These results indicate the potential of integrating formal methods in enhancing open-ended agentic RL.

View arXiv page View PDF Project page GitHub 1 Add to collection

Get this paper in your agent:

hf papers read 2610.22448

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 2

amazon/LAR-9B

9B •

Updated 19 minutes ago

•

6

•

1

amazon/LAR-27B

27B •

Updated 18 minutes ago

•

6

•

1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.22448 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.22448 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co