透過 Harness-Policy Optimization 實現自我演進的編碼智能體
Self-Evolving Coding Agents via Harness-Policy Optimization
Self-HPO 提出交替優化 harness 與策略的編碼智能體自我改進框架,並以強化學習針對不斷演變的失敗模式提升表現。在三項編碼基準上,Qwen3.5-35B-A3B 的平均解決率由 57.8% 升至 64.8%,增益亦可泛化至分佈外設定。使用外部模型 Opus 4.8 作為 harness 提案者,可進一步提升表現。
Published on Oct 16
Authors:
,
,
,
,
,
,
Abstract
Self-HPO is a recursive framework that alternates harness and policy optimization for coding agents, improving performance by targeting evolving failure modes through reinforcement learning.
Large language model (LLM) agents have shown promising potential in coding tasks, and their performance is jointly determined by the policy model and the harness that mediates their interaction with the environment. Indeed, harness and policy are mutually reinforcing and evolving only one component is suboptimal---a harness tuned for one checkpoint may not target the failure modes of an updated policy, while policy learning remains limited to the trajectories elicited by a fixed harness. In light of this, we propose Self-HPO, a recursive self-improving framework for coding agents that alternates harness and policy optimization, using the evolving policy as its own harness proposer. Specifically, in each round, the current policy first analyzes its own failures and proposes new harness candidates (H-step). The policy is then updated through reinforcement learning (RL) under the selected harness (P-step), and hard tasks identified during training form the next harness search set, keeping harness optimization targeted to the policy's evolving failure modes. Across three coding benchmarks, Self-HPO consistently outperforms harness-only and policy-only baselines, improving Qwen3.5-35B-A3B's average resolve rate from 57.8% to 64.8%, with gains generalizing to out-of-distribution settings. Further analysis shows that both optimized components retain gains even when paired with their base counterparts, while their joint gain exceeds the sum of their individual gains. Self-HPO also improves reliability (Pass@1) and expands the capability frontier (Pass@k) over the base agent. Additionally, we show that with a stronger external model (e.g., Opus 4.8) as the harness proposer, the same framework yields a further performance boost.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.22791
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 2
Reinforcement Learning • 36B • Updated 1 day ago • 30self-hpo/Self-HPO-Qwen3.5-35B-A3B
Reinforcement Learning • 36B • Updated 1 day ago • 28self-hpo/HPO-Qwen3.5-35B-A3B
Datasets citing this paper 1
self-hpo/Self-HPO-RL-Data
Preview • Updated 1 day ago • 45
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.22791 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co