跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 19 天前AI 評分38

Harness-S:從同系 Harness 學習,訓練具泛化能力的智能體

Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training

AI 導讀

Harness-S 以同系 Harness 挽救強化學習中失敗的 rollout,並將所得能力轉移至單一策略,使其在原有 Harness 下維持效能。自適應 Harness 路由器會在基礎 Harness 的所有初始嘗試失敗時,選出 Top-k 變體產生替代軌跡。於 AppWorld、SWE-Gym 和 DBBench,訓練所得策略在原有 Harness 下優於僅訓練策略的方法。

正文

Image 1: Hugging Face's logoHugging Face


Papers

arxiv:2611.04277

Copy markdown

Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training

Published on Nov 3

·Submitted byImage 2 张康宁on Nov 6

·Image 3: SJUT1Shanghai JiaoTong University

- [x] Upvote 6

  • Image 4
  • Image 5
  • Image 6
  • Image 7
  • Image 8
  • Image 9

Authors:

Kangning Zhang ,

Wenxiang Jiao ,

Bowen Fang ,

Shuai Shao ,

Jiachen Zhu ,

Rong Shan ,

Shijian Wang ,

Hao Wang ,

Yuan Lu ,

Weiwen Liu ,

Weinan Zhang

Abstract

A selective harness-rescue training method uses specialized sibling harnesses to replace failed reinforcement learning rollouts and transfer their benefits into a single policy that performs better under the original base harness.

Generated by deepseek-ai/DeepSeek-V4.1-Flash

Harness evolution from a common base exhibits Locality, Complementarity, and Nonstationarity: variants repair different failures, and their utility changes as the policy learns. Rather than seeking a universally superior harness, we introduce Harness-S to selectively rescue failures of a base execution system through sibling harnesses and transfer the resulting capabilities into a single policy that remains effective under the original harness. Directional Harness Evolution develops specialized sibling harnesses along distinct repair directions, forming a reusable Sibling Harness Pool that remains fixed during policy training. Complementary Harness Rescue Training invokes this pool only when all initial attempts under the base harness fail. The same policy generates fresh rollouts under selected auxiliary harnesses, and these trajectories replace the failed group for reinforcement learning. An Adaptive Harness Router selects the Top-k variants using Harness Behavior Profiles and voting across multiple LLMs. Policy-aware profile re-anchoring refreshes this evidence as the policy learns. Across AppWorld, SWE-Gym, and DBBench, the trained policy substantially outperforms policy-only training when returned to the original base harness. Behavioral analyses and held-out cases support the transfer of harness-elicited behaviors into the policy itself. Reusing the router for Rank-1 inference provides additional gains at the same executor rollout count.

View arXiv pageView PDFGitHubAdd to collection

Community

Image 10

zhangkangning

Paper submitter22 minutes ago

Harness evolution from a common base exhibits Locality, Complementarity, and Nonstationarity: variants repair different failures, and their utility changes as the policy learns. Rather than seeking a universally superior harness, we introduce Harness-S to selectively rescue failures of a base execution system through sibling harnesses and transfer the resulting capabilities into a single policy that remains effective under the original harness. Directional Harness Evolution develops specialized sibling harnesses along distinct repair directions, forming a reusable Sibling Harness Pool that remains fixed during policy training. Complementary Harness Rescue Training invokes this pool only when all initial attempts under the base harness fail. The same policy generates fresh rollouts under selected auxiliary harnesses, and these trajectories replace the failed group for reinforcement learning. An Adaptive Harness Router selects the Top-k variants using Harness Behavior Profiles and voting across multiple LLMs. Policy-aware profile re-anchoring refreshes this evidence as the policy learns. Across AppWorld, SWE-Gym, and DBBench, the trained policy substantially outperforms policy-only training when returned to the original base harness. Behavioral analyses and held-out cases support the transfer of harness-elicited behaviors into the policy itself. Reusing the router for Rank-1 inference provides additional gains at the same executor rollout count.

Reply

Edit Preview

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Comment ·Sign up or log in to comment

- [x] Upvote 6

  • Image 11
  • Image 12
  • Image 13
  • Image 14
  • Image 15
  • Image 16

Get this paper in your agent:

hf papers read 2611.04277

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.04277 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.04277 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.04277 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

System theme

Company

TOSPrivacyAboutCareers

Website

ModelsDatasetsSpacesPricingDocs

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co