跳到正文
DEV Community· Space App·· 2026-10-20AI 評分44

Six AI models played chess through their CLIs. None of them delivered checkmate.

Six AI models played chess through their CLIs. None of them delivered checkmate.

AI 導讀

六個 AI 模型透過各自的 CLI 進行 12 局棋賽,沒有一局以將死結束。11 局因第三次犯規告負,唯一打滿 60 ply 的對局由 Gemini 4 Argon 與 Opus 5.5 下成和局。Opus 5.5 和 Gemini 4 Argon 未用過後備走法;Opus 5.5 平均每步 13 秒,是最快的模型。

正文

Cover image for Six AI models played chess through their CLIs. None of them delivered checkmate.

Space App

I build SpaceApp, a self-hosted workspace for running AI coding CLIs side by side. On 18 October I let six models play chess against each other there, each through its own CLI. Nobody won by checkmate. The way they lost is the interesting part.

The setup

  • Players: Opus 5.5 and Fable 5.1 (through Claude Code), GPT-6-Astra and GPT-6.1-Sol (through Codex), all at xhigh reasoning; Gemini 4 Argon (High) and Gemini 3.8 Flash (High) (through the Gemini CLI).
  • Format: a 4-round Swiss tournament, 12 games.
  • Clock: 120 seconds per move. Each move is one one-shot turn of the model's CLI: it gets the position and answers with a move.
  • Strikes: a timeout, a provider error or a second illegal answer plays a random legal move for that side and counts a strike. The third strike forfeits the game.
  • Length: after 60 plies (30 moves each) the game is adjudicated on material.

The results

# Model Points W / D / L Elo Avg per move Forfeits
1 Opus 5.5 (xhigh) 3.5 3 / 1 / 0 1535 13 s 0
2 Gemini 4 Argon (High) 3.5 3 / 1 / 0 1535 23 s 0
3 GPT-6-Astra (xhigh) 2 2 / 0 / 2 1500 37 s 2
4 Fable 5.1 (xhigh) 1 1 / 0 / 3 1478 26 s 3
5 Gemini 3.8 Flash (High) 1 1 / 0 / 3 1477 40 s 3
6 GPT-6.1-Sol (xhigh) 1 1 / 0 / 3 1475 34 s 3
  • Zero checkmates. 11 of the 12 games ended on a third strike.
  • The 12th game, Gemini 4 Argon against Opus 5.5, ran the full 60 plies and was drawn on material.
  • Opus 5.5 and Gemini 4 Argon were the only models that never needed a fallback move. Opus 5.5 was also the fastest, at 13 seconds per move on average.

What it measures, and what it does not

Twelve games are a small sample, and a tournament decided by forfeits says very little about chess strength. What it does show is how reliably a model answers through its CLI when every answer has a deadline: no timeouts, no provider errors, no illegal moves, game after game. That is the property I care about most when agents run unattended for hours, and it is not the same ranking you get from a single-prompt leaderboard.

Try SpaceApp

SpaceApp runs on Linux, macOS and Windows 11 with Node.js 20.11 or newer:

npx --yes run-spaceapp@latest install

It is source-available under the Business Source License 1.1. Code, docs and known limitations: https://github.com/oll4com/spaceapp

If you have run something similar, I would like to know which models held up for you, and which rules you would change for a fairer second run.

來源:DEV Community · dev.to