跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 24 天前AI 評分50

論文提出面向實際 iOS 應用程式導航的規劃與 UI 定位智能體在線密集強化學習框架

Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS Apps

AI 導讀

這篇論文提出在線強化學習框架,直接在實際運行中的 iOS 應用程式訓練 UI 導航智能體。框架把智能體拆分為規劃智能體與 UI 定位模型,以逐步及任務完成獎勵配合 PPO 訓練,並提出 GiGPPO,利用並行軌跡中視覺相似狀態的獎勵穩定價值估算器。

正文

Image 1: Hugging Face's logoHugging Face


Papers

arxiv:2610.39264

Copy markdown

Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS Apps

Published on Oct 29

·Submitted byImage 2 Peitong Duanon Oct 30

·Image 3: appleApple

- [x] Upvote 1

  • Image 4

Authors:

Peitong Duan,

Khashayar Gatmiry,

Jeffrey Nichols,

Samy Bengio,

Emmanuel Abbe

Abstract

An online reinforcement learning framework trains UI navigation agents on live mobile apps using a planning agent, UI grounding model, and a novel neighbor-conditioned value estimation method to improve robustness and reduce inference costs.

Generated by thinkingmachines/Inkling-Small

While frontier vision-language models demonstrate strong capabilities in user interface (UI) navigation, their high inference costs prohibit everyday use. Conversely, smaller models trained via supervised fine-tuning (SFT) or offline reinforcement learning often memorize static trajectories, leading to brittle execution and an inability to recover from errors. To address these limitations, we propose a novel online reinforcement learning framework that trains UI navigation agents directly on live iOS applications. Our framework decouples the agent into two specialized components: a Planning agent and a UI Grounding model. We rely on an LLM-as-a-Judge to provide comprehensive reward signals, combining dense, step-level feedback to evaluate intermediate progress with a terminal reward for overall task completion. To effectively leverage this dual reward structure, we optimize the agent using Proximal Policy Optimization (PPO) with step-level advantage estimation. Furthermore, we introduce GiGPPO, a novel neighbor-conditioned value estimation method that improves critic stability by leveraging reward signals from visually similar states across concurrent trajectories. To support this online training paradigm at scale, we introduce an efficient infrastructure that orchestrates hundreds of concurrent virtual iOS devices. Evaluations on rigorous benchmarks, including UINavBench, iOSWorld, and AndroidWorld, demonstrate that our online RL trained agents substantially outperform both SFT and terminal-only RL baselines while accelerating training convergence. Notably, our 8B RL agents achieve nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments, while outperforming Gemini-3.5-Flash and tying with Gemini-3.1-Pro on real iOS applications.

View arXiv pageView PDFAdd to collection

Community

Image 5

peitongd

Paper submitter27 minutes ago

While frontier vision-language models demonstrate strong capabilities in user interface (UI) navigation, their high inference costs prohibit everyday use. Conversely, smaller models trained via supervised fine-tuning (SFT) or offline reinforcement learning often memorize static trajectories, leading to brittle execution and an inability to recover from errors. To address these limitations, we propose a novel online reinforcement learning framework that trains UI navigation agents directly on live iOS applications. Our framework decouples the agent into two specialized components: a Planning agent and a UI Grounding model. We rely on an LLM-as-a-Judge to provide comprehensive reward signals, combining dense, step-level feedback to evaluate intermediate progress with a terminal reward for overall task completion. To effectively leverage this dual reward structure, we optimize the agent using Proximal Policy Optimization (PPO) with step-level advantage estimation. Furthermore, we introduce GiGPPO, a novel neighbor-conditioned value estimation method that improves critic stability by leveraging reward signals from visually similar states across concurrent trajectories. To support this online training paradigm at scale, we introduce an efficient infrastructure that orchestrates hundreds of concurrent virtual iOS devices. Evaluations on rigorous benchmarks, including UINavBench, iOSWorld, and AndroidWorld, demonstrate that our online RL trained agents substantially outperform both SFT and terminal-only RL baselines while accelerating training convergence. Notably, our 8B RL agents achieve nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments, while outperforming Gemini-3.5-Flash and tying with Gemini-3.1-Pro on real iOS applications.

Reply

Edit Preview

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Comment ·Sign up or log in to comment

- [x] Upvote 1

  • Image 6

Get this paper in your agent:

hf papers read 2610.39264

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.39264 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.39264 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.39264 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

System theme

Company

TOSPrivacyAboutCareers

Website

ModelsDatasetsSpacesPricingDocs

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co