Google DeepMind 推出 Gemini 4 Argon 機械人模型

2026年10月13日

Gemini 4 Argon 在實體理解、程式碼生成及視覺語義推理方面帶來突破，為機械人技術堆疊解鎖新的工作流程，並加快邁向實體通用人工智能的步伐。

這項能力飛躍可在具身推理（ERQA）基準中見到。該基準測試模型理解及解決 3D 環境中實體和空間問題的能力。Argon 僅憑推理便取得 87% 的分數，超越所有其他前沿模型。

Argon 是通用實體推理模型，可應用於機械人生命週期的各個階段。在這篇文章中，我們分享 Argon 在四個主要範疇的代表性提示詞：

• 機械人控制 → 透過 MuJoCo 練習進行移動操作、零樣本機械人控制，以及對影片世界模型進行閉環操控。
• Real2Sim → 根據機械人攝影機畫面、人類第一身影片或手繪紙上草圖，重建經物理校準的模擬孿生模型。
• 實體理解 → 透過具代理能力的多模態事後標註，詳細理解靈巧操作、密集價值函數及進度估算。
• 研究與自我改進 → 透過模擬改進動作，並在硬件上反覆自我改進。

透過 Argon 解鎖機械人控制

1. Spot 移動操作：透過 MuJoCo 練習把物品放進垃圾桶

Argon 控制 Boston Dynamics Spot® 四足機械人（12 自由度腿部 + 6 自由度手臂），學習一項新技能：踩下腳踏式垃圾桶的踏板，丟棄汽水罐。Argon 建立腳踏式垃圾桶的 MuJoCo 數碼孿生模型，並讓模型具備關節結構，在踩下實體踏板、把汽水罐丟進打開的垃圾桶前，先練習相關動作。

抱歉，你的瀏覽器不支援播放這段影片

Spot 移動操作——由真實環境到模擬再回到真實環境的腳踏式垃圾桶棄置任務：Spot 拾起汽水罐、查看手腕攝影機、在 MuJoCo 練習三足平衡、踩下踏板、確認手部對準打開的垃圾桶中央，然後把汽水罐丟進桶內。

提示詞：

There's a Diet Coke can behind you and another on the black table on your left.
Use the spot-mobile-manipulation skill to pick them up and trash them in the black foot-pedal bin.
Before stepping down on the pedal, verify in the front cameras that your front-right foot target is centered on the pedal, and use a 2-agent verification swarm (foot-verifier and hand-verifier) to verify both the foot-pedal step and the 3D hand-in-bin alignment before releasing.

純文字

已複製

2. 零樣本控制

Argon 控制一隻簡單的機械手臂，扶正倒下的杯子，並把揉皺的餐巾放進杯內。Argon 運用視覺座標架構及動作原語進行推理，並透過多輪操作完成任務。

抱歉，你的瀏覽器不支援播放這段影片

左：第三身真實機械人畫面；右：經校準的手腕攝影機畫面，顯示投影的 2 cm 世界座標及夾爪閉合瞄準標記。

提示詞：

[Attached: wrist_cam.png, obs.json]

Task: Get the robot arm to reorient the mug upright and put the napkin inside the mug.

## Execution Discipline
- Simply output 1 sentence observation followed immediately by your `curl` command (with `# Plan:` on Turn 1 and `# Thought:` on every turn). Everything needed is in `obs.json` and the calibrated reticle/grid drawn on `wrist_cam.png`.

## Visual Coordinate Reference (Annotated on wrist_cam.png)
- `+X` (UP in image): Away from robot base | `-X` (DOWN): Towards robot base
- `+Y` (LEFT in image): World left         | `-Y` (RIGHT): World right
- `+Z`: Height above table                | `Yellow numbers`: World (X, Y) in centimetres (2 cm grid at `grid_z`)
- `Cyan + crosshair`: Current 3D pinch center
- `Cyan [ + ] brackets`: Current jaw opening and closing line
- `Green line`: Current object long-axis orientation (`yaw_deg`: 0° = vertical along +X, 90° = horizontal along +Y)
- `pitch_deg = 90°`: Straight down (-Z) top-down grasp/place; `pitch_deg < 60°`: Tilted coaxial approach along finger axis

## Closed-Loop Turn Workflow
1. Inspect current state & view `wrist_cam.png`:
   `curl -s http://127.0.0.1:8081/obs` (returns `ee_pos`, `holding`, `status`, `workspace`, `wrist_cam_path`).
2. Select VLM call granularity upfront on Turn 1:
   - Coarse (2–4 calls): `look_at` -> `grasp` -> `place` -> `done`
   - Pre-Grasp Hover Verify (4–7 calls): `look_at` -> `goto(z=0.04)` (verify cyan `[ + ]` brackets straddle target) -> `grasp` -> `place_until_contact` -> `done`
   - Fine Visual Servoing (8–20 calls): `look_at` -> `grasp` -> pre-pose `goto` -> closed-loop `move_by` / `move_until_contact` -> `done`
3. Send motion primitives (`POST /plan`) and verify visually after every turn in the updated `wrist_cam.png`. If `status == "grasp_failed"`, call `robot.look_at(...)` to re-observe and recover closed-loop.

## Code-as-Policies Primitives (`POST http://127.0.0.1:8081/plan`)
robot.look_at(x=0.35, y=0.13, z=0.14, grid_z=0.015)
x, y = robot.deproject(u=0.52, v=0.48, z_plane=0.015)
robot.grasp(x=x, y=y, z=0.015, yaw_deg=-35, pitch_deg=90)
robot.place(x=0.24, y=0.06, z=0.075)
robot.place_until_contact(x=0.24, y=0.06, z=0.015, max_lag=0.012, relief=0.002)
robot.goto(x=0.28, y=0.05, z=0.04, arc_z=0.14, pitch_deg=25, azimuth_deg=90, roll_deg=0, speed=0.10)
robot.move_by(forward=0.025, dz=-0.002, dpitch_deg=-3, droll_deg=5, speed=0.04)
robot.move_until_contact(dz=-0.06, speed=0.03, max_lag=0.012, relief=0.002)
robot.open_gripper() / robot.close_gripper() / robot.home() / robot.done()

純文字

已複製

3. Omni 控制

Argon 在 Gemini Omni World Model 中操控機械人，把五塊散落的 Scrabble 字母牌（G, R, A, N, O）排成一行，拼出 A - R - G - O - N。Argon 並非只使用一個固定提示詞，而是在閉環視覺迴路中進行操控。它會撰寫附有時間戳記的動作提示詞，檢查中間生成過程的關鍵影格，並逐步改進提示詞。

抱歉，你的瀏覽器不支援播放這段影片

反覆操控影片世界模型中的靈巧手部：經 33 步操控生成的影片，拼出 A-R-G-O-N。

提示詞：

Goal: {task_goal}

You are steering a video world model (Gemini Omni) in an iterative visual feedback loop. Inspect the attached frames from the latest rollout and write the next timestamped action prompt for the video world model:
- Break complex motions into one simple, single-direction action at a time.
- Keep inactive hands retracted and avoid blocking the camera's view of objects in the workspace.
- Explicitly state which areas of the workspace become empty after an object moves.
- Use [MM:SS.mmm] timestamps for each action beat and wrap your output in <omni_prompt> tags.

Example format (sample timestamps only, not the full prompt):
<omni_prompt>
[00:00.000] Character A's left hand rests at the side while the right hand reaches toward the target object.
[00:02.500] Character A's right fingertips push the object smoothly into the center of the table, leaving its starting area empty.
[00:05.000] Character A's right hand opens and retracts to the side so all placed objects remain in clear view of the camera.
</omni_prompt>

純文字

已複製

透過 Argon 解鎖 Real2Sim

1. 從機械人視角轉換為模擬

Argon 觀看一段真實機械人遙距操作片段，並建立數碼孿生體；它會取得或建立模擬資產，令這些資產的外觀及與靈巧手接觸時的行為都與真實物件相符。Argon 透過反覆視覺推理循環調整資產和場景，使模擬畫面及接觸事件與真實錄製數據中的情況一致。

抱歉，你的瀏覽器不支援播放這段影片

左方（真實視角）：同步的真實機械人鏡頭畫面；右方（模擬）：由 Argon 重建及校準的 3D MuJoCo 場景。

提示詞：

[Attached: robot_rollout_episode]

Build a MuJoCo scene of the robot and reconstructed task objects. When each episode is replayed frame by frame, by driving the robot joints toward the logged observation.state with joint-space PD.

Iterate using two verification objectives:
(1) Visual match: MuJoCo renders from the head_left, left_wrist, and right_wrist cameras line up with, and look like, those real observations.
(2) Physical match: the simulated contact force/torque on each Sharpa fingertip, in the sensor frame, matches the logged tactile_force in timing, magnitude and direction.

What you can tune happens before simulation starts:
(1) Static object assets: realistic geometry, collision shapes, mass, inertia, friction and joints, with appearance close to the real objects, plus the initial object poses at t = 0.
(2) Offline-planned robot commands, changed only in frames where the real fingertips are near an object. Each command must stay within 0.55 rad (arm joints) and 0.65 rad (hand joints) of the logged teleop trajectory. Everywhere else, replay the logged trajectory unchanged.

No sim cheating: task objects may move only through real MuJoCo contact with the robot hands and the environment (mj_step).
Not allowed after t = 0: changing MjModel parameters; applying external forces or torques to objects; welds or equality constraints; feedforward torques beyond joint-space PD; setting or rolling back object qpos/qvel; disabling or excluding collisions; reporting contact forces that don't come from the MuJoCo solver (for example, copying the logged tactile readings).

Immediately present the reconstructed scene and rollout results as an interactive HTML session viewer with one page per episode, showing:
- At the bottom: A synced timeline (play, scrub, speed control) that drives every panel.
- On the left panel: An interactive 3D Blender scene showing the simulated robot and objects, showing contact points and force arrows as replay progresses.
- On the right panel: Real vs. sim camera views side by side for all 3 cameras. Real vs. sim finger force visualizations.

純文字

已複製

2. 人類視角轉模擬

面對一段呈現操作任務的第一身視角人類影片，Argon 會識別與任務相關的物件，以由凸分解構成的複合網格取代粗略邊界框，呈現凹形容器；並在 MuJoCo 接觸物理模擬中，將人手軌跡重新映射至一個 76-DoF 雙手人形機械人，其五指手有 22-DoF。

抱歉，你的瀏覽器不支援播放這段影片

人類視角轉模擬——第一身視角人類影片轉為靈巧人形機械人孿生體：左方顯示輸入的第一身視角人類 RGB 影片；右方顯示同步的 500 Hz MuJoCo + Blender Cycles 多視角模擬推演。

提示詞：

[Attached: egocentric_human_video.mp4, candidate_sim_views.png, hand_3d_keypoints.json]

You are building a physically solvable 500 Hz MuJoCo manipulation environment (`scene.xml`) and studio Blender render that faithfully reproduces the tabletop task shown in `egocentric_human_video.mp4` on a bimanual humanoid robot with 22-DoF dexterous five-fingered hands:

1. Scene Reconstruction & Camera Calibration:
   - Identify only the manipulation-relevant objects (the grasped target object and the receiving container or landmark) and triangulate their 3D tabletop coordinates from the egocentric camera intrinsics/extrinsics.
   - Replace crude bounding boxes with compound convex-decomposed 3D meshes so concave containers (nested bowls, recessed wooden trays, open bins) support physical interior placement.

2. 500 Hz Dexterous Motion-Fitting Stress Test:
   - Retarget the 3D human wrist and finger keypoints to the humanoid's 7-DoF arm and 22-DoF hand joints, and step the trajectory in MuJoCo at 500 Hz (dt = 0.002s) with zero kinematic weld constraints.
   - Verify opposing thumb-and-finger force closure, clean object lift (> 2 cm), and stable final placement inside or beside the target landmark.

純文字

已複製

3. 草圖轉模擬

使用 Argon，開發者只需拍攝一張粗略的紙筆草圖——圖中 Apollo 2 人形機械人沿拋物線投球，擊倒桌上按 3-2-1 排列的箱子——Argon 便會自動建立 GPU 加速的 mujoco_warp 場景，合成平衡的 76-DoF 全身及五指投擲軌跡，在閉環物理模擬中驗證 6/6 個目標均被擊倒，並輸出 Blender 4.5 Cycles 渲染結果。

抱歉，你的瀏覽器不支援播放這段影片

草圖轉模擬——手繪草圖轉為靈巧人形機械人軌跡及 Cycles 渲染。

提示詞：

[Attached: hand_drawn_throw_sketch.jpg]

Translate this hand-drawn sketch into a physically verified humanoid simulation in `mujoco_warp` and a raytraced Blender Cycles video:

1. Scene & Kinematic Setup from Sketch:
   - Parse the spatial layout from `hand_drawn_throw_sketch.jpg`: place the 76-DoF humanoid (32-DoF body equipped with two 22-DoF five-finger dexterous hands) at the origin facing a table holding a 3-2-1 pyramid of 6 rigid target boxes, with a projectile sphere grasped in the right five-finger hand.

2. Whole-Body Balance & Dexterous Throw Synthesis:
   - Synthesize a 50 Hz whole-body throwing trajectory (140 steps, 2.80 s) across four phases: (1) stable stance & 5-finger power grasp closure, (2) torso/shoulder wind-up with ankle-hip Center-of-Mass compensation, (3) forward whipping acceleration, and (4) synchronized 22-DoF five-finger opening at peak hand velocity so the ball launches on a ballistic arc into the pyramid's second-tier center of mass.
   - Verify in `mujoco_warp` contact physics that the humanoid maintains upright balance (< 3 cm pelvis drift) and knocks down all 6 target boxes.

純文字

已複製

透過 Argon 解鎖物理理解能力

1. 事後標註與數據品質

Argon 對靈巧操作具備精確的物理理解能力，能從影片提供詳細的分段及文字標註。Argon 能識別操作欠佳的片段，例如停頓、抓握不穩和操作失誤。給予 Argon 一段俯視影片，內容是一個雙手 ALOHA 2 機械人在 136 秒內把一張正方形紙摺成摺紙狐狸，Argon 便會產生時間對齊的四層事後標註架構：片段目標（G1）、6 個連續的語義子任務（G2）、17 個接觸層級的原子動作基元（G3），以及自動品質控制（QC）重新加權機制，在操作失誤和重試期間令任務進度維持平坦。

抱歉，你的瀏覽器不支援播放這段影片

Argon 多粒度事後標註與遙測 HUD。

提示詞：

[Attached: Aloha2_Origami_20250309.mp4]

Produce a time-aligned multi-granularity hindsight annotation JSON with four levels:
1. G1_episode_instruction: Concise imperative sentence describing the overall manipulation goal.
2. G2_subtasks: Contiguous semantic subtask phases specifying which arm(s) anchor vs. fold/rotate the object.
3. G3_atomic_primitives: Fine-grained contact-level primitives (3–15s) aligned with grasp closure, crease sweeps, release, or retry attempts.
4. quality_control_and_reweighting: Hold task_progress_pct flat during non-progressing slip/delamination retry loops and downweight them (0.15x–0.25x) for Behavior Cloning.

純文字

已複製

2. 找出異球

為了用機械人重現經典的 12 球天平謎題，Argon 建立一個雙手 Franka FR3 MuJoCo 環境，內有 12 顆外觀完全相同的美式桌球（其中一顆暗中較重或較輕）及一座可活動的 Roberval 雙盤天平。接着，Argon 編寫一個純視覺閉環解題器，利用立體 RGB 相機對球進行三角定位，在每次稱量後透過影像判讀天平傾斜情況，並在 3 次稱量內將異球分辨出來，放進重球（紅色）或輕球（藍色）托盤。

抱歉，你的瀏覽器不支援播放這段影片

找出異常球（純視覺閉環求解器）：雙臂 Franka FR3 將候選球放在金色／青綠色秤盤上稱重，讀取立體視覺磅秤的傾斜角度，並在 3 次稱重內把確認較重的異常球單獨放入紅色托盤。

提示詞：

# Prompt 1 — Physics-Accurate MuJoCo Environment & Benchmark Generation
Make a MuJoCo environment where a dual-arm robot (two 7-DoF Franka FR3 arms with parallel-jaw grippers on a shared tabletop) is tasked to solve the classic 12-ball balance scale weight puzzle from https://www.mathsisfun.com/pool_balls_solution.html.
1. Create 12 visually indistinguishable pool balls in a 2x6 starting rack (11 standard 150 g, 1 odd ball either 270 g or 30 g).
2. Build a functional two-pan Roberval balance scale at the center of the table with parallelogram equality constraints (`mjEQ_JOINT`) so the Left (Gold) and Right (Teal) pans stay horizontal, plus Red (Heavy) and Blue (Light) submission trays.
3. Track distinct settled weighings (`weighing_count`) and reward `1.00` for isolating the true odd ball in the correct Heavy/Light tray within <= 3 weighings.

# Prompt 2 — Closed-Loop Vision-Only Solver
Implement an independent closed-loop policy relying strictly on stereo RGB camera inputs (`head_camera_left_rgb_image`, `head_camera_right_rgb_image`) and robot proprioception (`add_privileged_task_observations=False`):
1. Segment the 12 balls, Gold/Teal pans, and Red/Blue trays in both stereo views and triangulate their 3D coordinates.
2. Visually read the balance scale tilt after each weighing from stereo vertical pixel disparity (+1 = left heavier, -1 = right heavier, 0 = balanced).
3. Rotate the gripper wrist yaw by 90 degrees when grasping/placing inside the narrow scale pans to clear the raised rims, and isolate the odd ball into the Red or Blue tray within <= 3 weighings.

純文字

已複製

3. 價值函數

Argon 是一個適用於機械人任務的通用價值函數。在一段取自人類 UMI 數據集、長 60 秒的雙手環形分類任務中，Argon 會在接觸狀態轉換前後，反覆檢視同步的頭部、左夾爪及右夾爪攝影機畫面，生成密集的價值函數／任務進度預測，並將軌跡分解為 28 個片段，分別呈現正向、中性及負向的進度斜率。

抱歉，你的瀏覽器不支援播放這段影片

價值函數：同步播放 UMI 的 3 路攝影機畫面（頭部、左夾爪、右夾爪），並與動畫任務進度曲線及雙手操作片段標註同步顯示。

提示詞：

[Attached: umi_head.mp4, umi_left_gripper.mp4, umi_right_gripper.mp4]

Look at 3 videos of human wearing a gripper performing activities from head, left, right gripper views. Look at the videos thoroughly and give a detailed plot of progress at the task (with positive, neutral, negative slope), identifying important segments in the video and labeling the activities. Look at individual frames or subsegments of the video if necessary to get a better understanding.

純文字

已複製

透過 Argon 推動研究與自我改進

1. 透過模擬改進動作

Argon 可在模擬環境中生成新數據，以提升通用視覺－語言－動作（VLA）模型的效能。只需從真實桌面的單張 RGB 相片開始，Argon 便能自主建立經校準的 MuJoCo 模擬孿生環境、定義新任務，並合成閉環動作生成器，在模擬環境中收集 4,500+ 個經域隨機化的回合，將實體機械人在真實世界的成功率由 56% 提升至 75%，並將完成時間由 35s 縮短至 24s。

抱歉，你的瀏覽器不支援播放這段影片

透過模擬改進動作：提升真實世界評估基準（成功率由 56% 升至 75%，完成時間由 35s 縮短至 24s）。

提示詞：

[Attached: real_tabletop_photo.jpg]

You are given a single overhead RGB camera photo (`real_tabletop_photo.jpg`) of a bimanual robot workspace (two 7-DoF Franka FR3 arms with parallel-jaw grippers and 4 RGB cameras) facing a table arranged with photobooth party props.
1. Reconstruct procedural 3D meshes and compound MuJoCo collision primitives for all 9 party props matching their real-world dimensions, colors, and resting poses, including spring-hinged deformable crown panels for the soft plush cap.
2. Define a 77-task single-arm and bimanual curriculum (pick-and-place, uprighting, directional pushing, mid-air handovers, bimanual scoop-and-push, and pipelined table clearing).
3. Synthesize a closed-loop IK trajectory generator across ±10 cm pose randomization, verify `reward == 1.0` in MuJoCo physics, and export 4,500+ episodes for VLA fine-tuning.

純文字

已複製

2. 自我改進視覺抓取與放置

在一個簡單的 6-DoF 機械臂上，Argon 透過試行和自我反思，反覆改進抓取與放置任務的執行速度及放置準確度。

抱歉，你的瀏覽器不支援播放這段影片

SO-101 視覺引導抓取與放置——跨越第 1 → 2 → 3 次迭代的硬件自我改進：第 1 次迭代（58.9s）、第 2 次迭代（22.9s，放下時擦過盒邊）及第 3 次迭代（17.4s，放在正中央）。

提示詞：

# Stage 1 — Measure Objects & Dry-Run Plan (No Hardware Motion)
Stage 1: measure objects — find the objects and plan the whole motion. The arm does not move in this stage.

A. I put a LEGO brick and a box on the table in front of the arm, and the wooden box to one side. Find the brick and the wooden box in the image. And also in the real world. Show this in an annotated image.

B. A dry-run plan (no hardware). Write the planner to pick and place the lego block. Print the plan table, draw all poses onto the overhead image, and stop. Do not move the robot.
Report with evidence.

# Stage 2 — Pre-Pick Hover & Wrist-Camera Verification
Stage 2: Put the gripper above the brick without touching it, and verify with the wrist camera that the arm and the vision agree. Please record executions with both cameras (compressed video is OK).

# Stage 3 (Autonomous Pick-and-Place → Self-Improvement → Optimization):
1. Execute autonomously: Turn the Stage 1 dry run into a real grasp, lift, and place into the wooden box. Run it yourself subject to safety rules from AGENTS.md.

2. Self-improve: When an attempt fails, diagnose it yourself. Review your own logs and wrist/overhead footage, find the root cause, check the fix in a dry run, and retry.

3. Optimize on feedback: Once it succeeds, make it at least 2× faster within safety limits (rule R5) unless I authorize more, or give feedback.

純文字

已複製

3. 在 MuJoCo WebAssembly 中以單一提示詞重現機械人論文

每篇論文只需一個提示詞，Argon 便會閲讀機械人控制論文及任何已發布的檢查點，將物理模型和閉環控制器（從 DeepLoco 的 7 個網絡階層式運動控制、HUSKY 的 50 Hz PPO 策略網絡，到 Whole-Body MPC 的 Web Worker iLQR）移植至獨立運作的 MuJoCo WebAssembly 瀏覽器實驗室，並在無頭 Chromium 中驗證。

抱歉，你的瀏覽器不支援播放這段影片

五篇具代表性的腿式與人形機械人論文在瀏覽器中即時運行（MuJoCo WASM）：Robotic Soccer OP3 1v1、DeepLoco、Massively Parallel Deep RL ANYmal C、Whole-Body MPC Go2，以及 HUSKY G1 Skateboarding（由 Argon 重現的 68 個互動式瀏覽器沙盒中所展示）。

提示詞：

[Attached: paper.pdf, optional author_checkpoints/ or urdf_assets/]

Reproduce the paper "<Paper Title & arXiv ID>" from scratch in MuJoCo WebAssembly and deploy a self-contained browser lab + visual essay:
1. Match the paper's exact physics timestep, actuator gains (kp/kv), torque limits, and kinematic constraints in MuJoCo XML, and run real neural inference or Web Worker iLQR in JavaScript at the paper's exact control rates.
2. Build `index.html` (Spatial UI simulator with camera presets, live telemetry plots, and disturbance injection) and `scrollytelling.html` (interactive visual essay).
3. Run a headless Chromium (Playwright) verification loop asserting 0 console errors and non-zero task progress over 300+ frames.

純文字

已複製

以 Argon 推進具身智能

Argon 結合程式編寫、視覺感知、物理模擬及具身機械人控制。透過原生地對多視角影片、3D 幾何、接觸力學及本體感覺遙測數據進行推理，Argon 不但能理解場景中的物件是甚麼，還能理解物件如何與智能體進行實體互動，以完成全身靈巧操作任務。Gemini 4 Argon 標誌著加快邁向實體 AGI 的重要里程碑。透過建立這項核心智能，我們的目標是推動機械人與人類並肩工作，解決複雜挑戰。

開始使用

你現在可以透過 Google 的開發者平台，開始使用 Gemini 4 Argon 進行開發：

• Google AI Studio：供 Google AI Ultra 訂閲用戶使用。
• Gemini API：可透過 AI Studio 中 Gemini API 的付費方案使用。在 Interactions API 中使用 gemini-4-argon，google-genai SDK 均從推出首日起支援。
• Google Antigravity：供 Google AI Ultra 訂閲用戶使用。如要開始開發，請在 Google Antigravity 中選擇 Gemini 4 Argon（gemini-4-argon）作為目前使用的模型，以進行複雜規劃、/goal 執行作業及多智能體協調。

上一頁

下一頁