Gemini 4 Neon 基準測試比較編碼、智能體表現與每項任務成本
Gemini 4 Neon Benchmarks: Coding, Agents and Cost per Task
Gemini 4 Neon 在 Artificial Analysis Coding Agent Index 以 58.6 分排名第 1,Google 稱其每項任務成本較 Gemini 4 Carbon 低 32%。
Gemini 4 Neon benchmarks put it #1 on the Artificial Analysis Coding Agent Index (58.6), at 65.2% on Terminal-Bench 4.1 and #2 on the Vals Index (70.9%). Google also says it costs 32% less per task than Gemini 4 Carbon at its default high thinking level. Most rows were run at xhigh, though, and some on Google’s own harness.
That last detail is the point of this post. Our Gemini 4 Neon overview covers specs and pricing. Here you get every published score, who measured it, at which thinking level, and how to turn “cost per task” into a number you can check against your own bill.
Gemini 4 Neon benchmarks at a glance
Gemini 4 Neon is Google’s middle Gemini 4 model, released November 18, 2026, priced at $3 per million input tokens and $15 per million output tokens. Its strongest published results are in agentic coding and knowledge work, where it ranks #1 or #2 on four independent leaderboards.
These are the headline rows from Google’s launch post:
Google also reports #1 on Vals AI’s Legal Research Bench and Tax Agent Bench.
Gemini 4 Neon coding benchmarks
Coding is where Neon’s case is strongest, and also where the harness matters most. Google’s evaluation methodology PDF explains how each coding row was produced.
Artificial Analysis Coding Agent Index: 58.6, #1. This is an independent composite, so it’s the row least shaped by Google’s setup. You can check the current standings on Artificial Analysis.
Terminal-Bench 4.1: 65.2%. Artificial Analysis ran Neon using the public Antigravity-CLI harness, version 1.5.9. Other models’ scores come from the official leaderboard, and every result uses cheating-adjusted accuracy. Fair, but Neon ran inside Google’s own agent tooling.
FrontierCode 2.0: 46.2%. This one is from the public leaderboard and was run at high, the default you get in production. If you only trust one coding number for everyday traffic, trust this one.
SWE-sweep: 39.0% in Antigravity, 12.8% on mini-SWE. Both are self-computed by Google. The 3x jump comes from the harness: Google notes that Antigravity-CLI includes a default tool that fans sub-agents out across the codebase, which suits bug-finding tasks. Opus 5.5 ran in Claude Code and GPT-6.2 Sol in Codex for the same comparison. Read 12.8% as the model and 39.0% as the model plus its best tooling.
Agent and knowledge-work benchmarks
These rows matter if you plan to run Neon unattended across many steps.
- Agent Arena Index: 1,336, #2. Built from 4,200+ real-world agent sessions. Google also says Neon ranks #2 there on steerability, meaning it adapts cleanly when a user corrects it mid-task.
- AutomationBench: 56.3%, #1. Zapier’s benchmark for business automation, from the public leaderboard.
- Vending-Bench 2: $16,843, #1. Andon Labs’ long-horizon simulation of running a business. The score is money earned, not a percentage.
- Agents’ Last Exam v1.1: 75.8%. Self-computed on the 122-task set inside Antigravity-CLI 1.4.8 with a 5-hour window per run. Opus 5.5’s score comes from the public leaderboard and GPT-6.2 Sol’s from its system card, so the setups differ.
- Chartography: 73.5%. Reading dense professional charts, self-computed without tools.
For science and math, Neon posts 70.5% on Terminal-Bench Science (self-computed, averaged over 5 attempts), 84.0% on RiemannBench and 71.3% on HLE-Diamond (from the Artificial Analysis leaderboard).
How Google ran the numbers
Four rules from the methodology change how you should read the table:
-
Highest thinking level by default. “All of the results are run with the Gemini API with the highest thinking settings unless indicated otherwise.” That’s
xhigh. Only FrontierCode 2.0 is explicitly markedhigh. Production calls default tohigh, so expect fewer thinking tokens and possibly lower scores than the table shows. - Pass@1, single attempt. No majority voting or parallel test-time compute, except where noted (Terminal-Bench Science averages 5 attempts).
- Competitor numbers are self-reported. Non-Gemini scores come from each provider unless noted. Opus 5.5 and GPT-6.2 Sol use their maximum reasoning setting where available.
- Safety filters on, except cyber. The cybersecurity rows ran with filters off.
Our thinking levels guide covers when xhigh is worth its extra tokens.
Cost per task: what the 32% claim means
Cost per task is the total spend to finish one benchmark task, not the price per token. A model with a higher token price can still cost less per task if it reaches the answer in fewer steps and fewer thinking tokens.
Google makes three cost claims:
| Claim | Thinking level | Compared with |
|---|---|---|
| 32% lower cost per task, with a high AA Intelligence Index score | high |
Gemini 4 Carbon |
| Roughly 50% to 80% lower cost per task on Vals Index and Agent Arena | high |
Top competing frontier models |
| 36%+ lower cost per task | xhigh |
Competing models at their highest setting |
Note the mismatch: the accuracy rows mostly use xhigh, while the headline 32% saving is at high. Both are true, but not at the same time. You get the cost saving or the top score, and the gap between them is yours to measure.
Here’s how to estimate your own cost per task. Thinking tokens bill as output.
$$\text{cost} = \frac{\text{input} \times 3 + \text{cached input} \times 0.15 + \text{output} \times 15}{1{,}000{,}000}$$
A coding task with 20,000 input tokens and 4,000 output tokens costs $0.06 + $0.06 = $0.12. If 15,000 of those input tokens hit the cache, it drops to $0.015 + $0.00225 + $0.06 = about $0.077. Run the same task at xhigh and the output count is what moves, so log tokens per level, not just pass or fail. Our Gemini 4 Neon pricing breakdown covers Batch, Flex and Priority rates.
Cyber and safety rows
The cyber scores come from “Gemini 4 Neon for cyber,” which goes to trusted defenders through the Fairwind Program rather than the public API.
| Benchmark | Score | Note |
|---|---|---|
| SEC-bench Pro | 90.5% | State of the art, roughly half the cost per challenge |
| WebExploitBench v1.2 | 51.26% | State of the art, half the cost of large models |
| AutoBaxBench (hard tier) | 76% | State of the art, more secure code than Carbon on every tier |
| CWE-bench v2 | 70.7% | 3.4% uplift over Carbon |
| ExploitGym | Not quoted | Frontier-level using half the tokens of Carbon |
Safety numbers from the same post: Neon refused 1.98% of Terminal-Bench 4.0 tasks, scored 0.0% cheating on Terminal-Bench 4.0, took shortcuts on 2.45% of Google’s coding cheating evaluation, and passed 99.2% of its agent honesty evaluation. With real-time safeguards at its highest thinking level, it resisted every attack in Gray Swan’s indirect prompt injection benchmark.
Where Neon doesn’t lead
Neon isn’t first everywhere, and Google says so:
- Artificial Analysis Intelligence Index: #3 overall, “just behind larger, slower models.”
- Vals Index: #2, not #1.
- Agent Arena Index: #2.
Google also still points you to Carbon for its hardest long-horizon projects. If you’re choosing between the three Gemini 4 models, our Neon vs Carbon vs Hydrogen comparison maps the trade-offs, and the Gemini 4 Argon benchmarks post shows how the earlier model’s table read.
Benchmark Gemini 4 Neon yourself with Apidog CLI
Leaderboards describe Google’s harness, not your product. The fastest way to know whether Neon earns its price is to let your coding agent run the same prompts at high and xhigh and log tokens and latency for each.
There’s no free API tier for Neon, so you need a paid Gemini API key. This request follows Google’s documented Interactions API format; we haven’t live-run it for this post:
curl -s "https://generativelanguage.googleapis.com/v1beta/interactions" \
-H "x-goog-api-key: $GEMINI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "gemini-4-neon", "input": "Find the race condition in this retry queue.", "generation_config": {"thinking_level": "high"}}'
Apidog CLI lets Claude Code, Codex, Cursor or OpenCode create and run API tests from the terminal. Set it up once:
npm install -g apidog-cli@latest
apidog login --with-token <YOUR_TOKEN>
npx -y skills add https://apidog.com # pick your agent when prompted
Then give your agent one prompt:
In Apidog project <id>, create a Gemini API environment with GEMINI_API_KEY
and THINKING variables. Add a test case for POST /v1beta/interactions on
gemini-4-neon that sends my 10 eval prompts from prompts.csv, asserts HTTP 200
and a non-empty output, and records token usage and response time. Run it at
THINKING=high and THINKING=xhigh and compare cost per task at $3/$15 per 1M.
The agent creates the environment, checks the test case payload against its schema, then runs it once per thinking level:
apidog environment create gemini-api --project <id> --base-url https://generativelanguage.googleapis.com
apidog cli-schema get test-case-create
apidog cli-schema validate test-case-create --file case.json
apidog test-case create --project <id> --file case.json
apidog run --test-case <caseId> --project <id> -e <envId> -d prompts.csv --env-var "THINKING=high" -r cli,json
apidog run --test-case <caseId> --project <id> -e <envId> -d prompts.csv --env-var "THINKING=xhigh" -r cli,json
Add --timeout-request 900000 if your prompts are long, since hard xhigh runs can take minutes. To rerun the comparison on every prompt change, put the same apidog run ... -r junit into CI with the Apidog CLI CI/CD guide, or start from Apidog CLI in Claude Code. Apidog’s Free plan covers up to 4 users.
FAQ
What is Gemini 4 Neon’s best benchmark result? #1 on the Artificial Analysis Coding Agent Index at 58.6, plus #1 on AutomationBench (56.3%) and Vending-Bench 2 ($16,843).
What thinking level were the benchmarks run at? Mostly xhigh, the highest setting. Google’s methodology says results use the highest thinking settings unless noted; FrontierCode 2.0 (46.2%) is marked as the default high.
Is Gemini 4 Neon cheaper than Carbon per task? Google says 32% lower cost per task at high. That’s a per-task figure from benchmark runs, so confirm it on your own prompts before moving traffic.
Why does Neon score 39.0% and 12.8% on SWE-sweep? Different harnesses. 39.0% is inside Google Antigravity, which fans sub-agents out across the codebase; 12.8% is the simpler mini-SWE agent.
Are Gemini 4 Neon’s benchmarks independent? Partly. The Artificial Analysis, Vals, Arena, AutomationBench and FrontierCode rows come from third parties. SWE-sweep, Agents’ Last Exam and Terminal-Bench Science are self-computed by Google.
來源:DEV Community · dev.to


