OpenHands 與 Snowflake：如何為自訂 Agent harness 後訓練模型

OpenHands × Snowflake AI Research

AI coding agents are increasingly being customized for specific development workflows. Developers configure system prompts, tools, skills, execution environments, and turn budgets to help agents work effectively within their repositories and engineering processes.

But there’s a disconnect: while the agent harness is customized for your workflows, the model powering it typically isn’t.

Most models are post-trained using general-purpose agent configurations that differ from the environments where they’re ultimately deployed. The result can be familiar problems: incorrect tool calls, ignored skills, unnecessary steps, or agents that exhaust their turn budgets before completing a task.

Developers can continue refining prompts and tools to address these issues, but there’s another option: post-train the model to perform better within the agent harness itself.

Reinforcement learning (RL) provides a way to do this. The model attempts tasks within an agent environment, receives feedback based on the results, and updates its weights to improve performance.

The challenge is that traditional RL frameworks often introduce their own agent loops, tool formats, and task structures. Developers must recreate their existing harness inside the training framework, introducing additional engineering work and potential differences between training and deployment.

OpenHands and Snowflake are addressing this gap by enabling developers to post-train models using their existing OpenHands agent harnesses.

With the OpenHands Agent SDK, developers can configure agents with their own prompts, tools, and skills. Tasks are defined using the open Harbor format, while Snowflake Cortex Training provides the GPU infrastructure for reinforcement learning.

The result is a workflow where developers can optimize open-weight models for their specific agent configurations without rebuilding the harness in a separate training framework.

Your harness is the training harness

OpenHands provides an open, extensible framework for building AI agents that can interact with development environments, execute commands, use tools, and complete multi-step tasks.

Rather than prescribing a single model or agent configuration, the OpenHands Agent SDK gives developers control over how their agents operate.

That flexibility extends to post-training.

During training, every rollout is a real execution of your OpenHands agent. The system prompt, tools, skills, and execution limits are passed as configuration rather than recreated inside a separate RL framework.

The agent doesn’t need to know it’s being trained. It sends ordinary chat-completion requests, and a proxy between the agent and the model records the exact tokens the model produces on each turn. Those tokens become the training data: the model learns from its own outputs, while the prompts and tool results are excluded from the loss.

This allows developers to retain their existing agent configuration and execution logic rather than maintaining a separate training implementation.

Tasks use the open Harbor format, which developers may already use for agent evaluation with harbor run.

A Harbor task is a directory containing:

• instruction.md, which describes the task.

• environment/, which defines the sandbox.

• tests/test.sh, which evaluates the result.

The test script’s score becomes the reward signal used during training.

Each rollout runs in its own Snowflake Cortex Sandbox. The agent’s terminal and the grader execute within the same sandbox, ensuring the reward reflects the outcome of the agent’s actions.

This is the key advantage of combining OpenHands and Snowflake: developers can reuse their existing agent harness and task definitions while Snowflake provides the isolated execution environments and GPU infrastructure needed for RL training.

Instead of maintaining separate systems for agent development and model post-training, developers can use the same underlying agent implementation across both workflows.

Let OpenHands run the training

Even with an existing agent harness, configuring and managing an RL training run involves several operational decisions.

Developers need to allocate GPUs, configure batch sizes and sequence limits, launch training jobs, monitor progress, troubleshoot failed rollouts, and release compute resources when training finishes.

The new cortex-training-rl skill in OpenHands/extensions automates much of this process.

Using OpenHands, developers can describe the training run they want, and the coding agent handles the setup and execution.

For example, in OpenHands:

Use /cortex-training-rl to post-train Qwen/Qwen3.5-4B on my Harbor tasks in ~/my-harbor-tasks/train and ~/my-harbor-tasks/eval with my agent's skill at ~/my-skill/SKILL.md, a 15-turn budget, for 50 steps.

Or from a terminal, with the OpenHands CLI:

openhands --headless -t "Use /cortex-training-rl to post-train Qwen/Qwen3.5-4B on my Harbor tasks in ~/my-harbor-tasks/train and ~/my-harbor-tasks/eval with my agent's skill at ~/my-skill/SKILL.md, a 15-turn budget, for 50 steps."

The prompt names the training and evaluation task directories, the harness, and the run length; the agent turns them into the training configuration.

The agent then:

1. Configures the training launcher for your harness and tasks.

2. Checks GPU availability on Snowflake Cortex Training.

3. Starts the training run in the background.

4. Reports reward metrics and failed rollouts at every step.

5. Plots the reward curve.

6. Releases GPU resources when the run completes.

The skill works with OpenHands and can also be used from Claude Code and Cursor.

Your local machine only needs to run the agent rollouts on CPU. Model training and sampling run on Snowflake Cortex Training GPUs, using four H200 GPUs for each by default.

This separation allows developers to work with their existing agent environments without provisioning or managing a dedicated GPU cluster.

It also demonstrates how OpenHands’ extensible agent architecture can support workflows beyond code generation, including the configuration and management of model training jobs.

Example: Post-training a custom code localization agent

To test the approach end to end, we built a custom agent harness for code localization, a task from OpenHands’ CodeScout work.

The agent receives a GitHub issue from a real Python repository in the SWE-smith dataset. Its objective is to identify the files, modules, and functions that need to change to resolve the issue.

This is a useful test case for harness-specific post-training because successful execution depends on more than the model’s general coding knowledge.

The agent must navigate the repository, use its available tools, follow the configured workflow, and produce a structured result within a limited number of turns.

Agent configuration

The harness uses an OpenHands agent with:

• A system prompt designed specifically for code localization.

• A terminal tool for exploring the repository.

• A custom localization_finish tool for submitting the answer.

• A maximum budget of 10 turns.

The reward is calculated as the sum of the agent’s F1 scores at the file, module, and function levels.

The maximum possible reward is 3.0.

We trained Qwen3.5-4B within this harness for 25 steps, using 64 rollouts per step.

Training results

Figure 1. Mean reward (file + module + function F1) across training steps for Qwen3.5-4B trained in a custom OpenHands code localization harness.

The results show that the model learned to perform the specialized code localization task more effectively within the configured harness.

This is an important result: in this experiment, training through the existing OpenHands harness and standard Harbor task format achieved learning outcomes comparable to a purpose-built RL implementation.

The approach preserved the ability to optimize the model for the task without requiring the agent harness to be rebuilt for training.

The complete run, from Harbor tasks to a model trained for the harness, finished in a couple of hours. Post-training no longer has to be a separate research project with its own framework and GPU cluster. It becomes one more step in the harness development loop, alongside tuning prompts, tools, and skills.

Try it with your own agent harness

Open OpenHands and describe the training run, as in the example above. The cortex-training-rl skill comes from OpenHands/extensions, and all a run needs is a Snowflake account with Cortex Training and a set of Harbor tasks.

For full control over the training configuration, the Arctic Platform Harbor recipe that the skill uses can also be run directly. Its README documents every setting and shows how to reproduce the code localization experiment described above.

Good to know

There are a few implementation details to consider when configuring training.

• Sandboxed by default: Training involves executing thousands of model-generated commands. Each rollout receives its own Snowflake Cortex Sandbox, and the training machine’s credentials are not exposed to the sandbox. This provides isolation between training infrastructure and agent execution.

• Terminal-based file operations: The agent’s terminal executes each command in a fresh shell. The working directory persists between commands, but exported environment variables and background processes do not. The OpenHands file_editor tool is not currently available during training, so agents must edit files through the terminal.

• GPU configuration: The launcher targets Hopper GPUs by default and uses FlashAttention 3. Other GPU configurations can be selected through the available settings.

What’s next

OpenHands gives developers the flexibility to build AI coding agents around their preferred models, tools, skills, and execution environments.

With Snowflake Cortex Training, that flexibility now extends to post-training models for the specific workflows those agents execute.

Rather than adapting an existing agent to a separate RL framework, developers can train models using their configured OpenHands harness, evaluate performance against real tasks, and use managed infrastructure to run the training process.

The code localization experiment demonstrates that this approach can improve task-specific performance while producing results comparable to a purpose-built RL implementation.

If you’ve built an agent harness with OpenHands, you can now experiment with training a model for it.

Try the cortex-training-rl skill, explore the Arctic Platform Harbor recipe, and let us know what you train.

If your harness requires functionality that isn’t supported yet, open an issue in OpenHands/extensions or Arctic Platform.

Resources

• cortex-training-rl skill: OpenHands/extensions, skills/cortex-training-rl

• Arctic Platform Harbor recipe: Snowflake-AI-Research/Arctic-Platform, recipes/rl/harbor

• OpenHands Agent SDK: OpenHands/software-agent-sdk

• Harbor: harbor-framework/harbor

• SWE-smith code search dataset: OpenHands/SWE-smith-py-code-search on Hugging Face