Claude Code 與 Codex 同題對戰,作者以評分表比較表現
We Made Claude Code and Codex Fight Over the Same Task — With a Score Sheet
作者以相同任務、提示詞和輸入,讓 Claude Code 與 Codex 對戰,並用事先編寫的標準答案及 100 分評分表比較結果。評分涵蓋準確度 40 分、返工 20 分、速度 15 分、品質 15 分及遵循指令 10 分,人工修正也計入評分。
Every week someone posts "I switched from Claude Code to Codex" (or the reverse), and the evidence is always the same: vibes.
We run a small AI team — a personal AI secretary system with several agents — and we got tired of vibes. So we started running same-task battles: both agents get the identical task, the identical prompt, and the identical input. Then we score them against an answer key we wrote before the battle started.
Here is exactly how it works, with a worked example.
The four fairness rules
- One prompt. Both agents receive the same instruction text, character for character. No tuning the prompt to each model's strengths.
- Golden answer first. Before either agent runs, we write the ideal output (or the checklist it must satisfy). If you write the answer key afterwards, you will unconsciously grade toward whoever went first.
- Mechanical scoring. A 100-point rubric (below). Anything that can't be scored mechanically gets cut from the rubric.
- Rework counts. If an agent's output needs fixing, the fix is part of its score — not a footnote. "It got there eventually" is a cost, and costs belong in the result.
The rubric
| Category | Points | What we check |
|---|---|---|
| Accuracy | 40 | Matches the golden answer: facts, figures, required items |
| Rework | 20 | Deductions for every correction a human had to make |
| Speed | 15 | Wall-clock time, scored relative to the faster agent |
| Quality | 15 | Structure, readability, formatting as specified |
| Instruction-following | 10 | Did exactly what the prompt said — no more, no less |
A worked example (sample data)
Task: "Turn these meeting notes into a decision summary: decisions, owners, deadlines. Nothing else."
The (fictional) notes mention a launch date of March 3, two owners, one deadline — plus a long digression about office snacks. The golden answer lists exactly the three decisions, both owners, and the deadline.
What typically happens:
- Agent A returns a clean summary, correct date, both owners — but adds a "Risks" section nobody asked for. Accuracy 40, instruction-following −4.
- Agent B returns exactly the requested format, but writes the deadline as March 13. Accuracy −10 for the wrong date, no matter how pretty the formatting is.
Final scores might land at 96 vs 89 — and crucially, you can point at why. The score sheet survives the argument.
Our series so far
Real battles, same rules:
- Battle 1 — meeting-notes cleanup (writing task): Claude Code 99, Codex 92. Nearly perfect on accuracy; the gap came from rework.
- Battle 2 — CSV to monthly aggregation (implementation task): Codex 96, Claude Code 93. The rematch flipped the winner. Task type matters more than brand loyalty.
Series record: 1–1. The honest conclusion after two battles is that "which agent is better" is the wrong question. "Which agent is better at this task type, under this rubric" is one you can actually answer — and route work with.
Try it yourself
We packaged the whole method — the shared prompt template, the 100-point rubric with the scoring rules for speed and rework, and a how-to for building golden answer keys — as a small kit:
Same-Task Battle Kit on Gumroad ($5)
It's the exact files we use for the series, not a summary of them. Run one battle with your own task, and you'll know more about your agents than a month of vibes will tell you.
We write about running a personal AI secretary and its agent team — the illustrated series is on note (English edition: kajuiwa_en) and Medium.
來源:DEV Community · dev.to