如何設定 model-radar，找出真正會呼叫工具的免費 LLM
Set up model-radar: catch free LLMs that really make tool calls
model-radar 提供 model-radar 和 model-canary 兩個命令：前者測試 OpenRouter 免費模型能否實際呼叫工具，後者監察付費模型及信用額，並在故障時切換至較便宜或本地模型。限流或逾時會標示為 BUSY，不作失敗判定；model-canary 需連續兩次硬失敗才切換，逾時和 429 不計入失敗。
model-radar is two small commands I run for my AI agents:
-
model-radarlists the free models on OpenRouter and actually asks each new one to make a tool call. Loads of free models say they support tools and then answer in prose. -
model-canarywatches the paid model my agents use, and the credit balance, and fails over to a cheaper model or a local one when either runs out.
Neither asks a model for an opinion. They measure and act. Setup is about ten minutes, and the guide is run on a fresh Ubuntu machine every week.
Install
Python 3.11 or newer, nothing to pip install.
git clone https://github.com/casareanderson/model-radar.git && cd model-radar
cp config.example.toml config.toml
./bin/model-radar --help
Check it fails safely first
Before adding a key, run both with no credentials:
./bin/model-radar --dry-run # both sources "no credential, skipped", empty baseline
./bin/model-canary --dry-run # "no OPENROUTER_API_KEY -- nothing to check", exit 1
That exit 1 matters. If the canary can't check anything, a timer should notice, not shrug.
The real run
export OPENROUTER_API_KEY=sk-or-...
./bin/model-radar --dry-run
One line per model: PASS, FAIL or BUSY. BUSY means rate-limited or timed out, and it's never treated as a verdict, it just gets tried next run. On 8 October that gave me 4 passes out of 8 probed, two 403s and two busy. One free model I'd tried earlier answered every request and never once called the tool.
Run it once without --dry-run to save a baseline. From then on it only probes models it hasn't seen.
The canary
Point it at whatever your agents use (a JSON file, or hermes-agent profiles), add a paid fallback ladder and a local model in config.toml, and run it every few hours from cron. Two hard failures in a row and it walks the ladder. Timeouts and 429s never count.
What CI can't prove
Anything with a key. The weekly run has none, so it proves install, config, help and the no-key behaviour. The probes and the canary's failover need your OpenRouter account.
Full guide: github.com/casareanderson/model-radar/blob/main/docs/SETUP.md
How do you pick models for your agents, and how do you find out when one quietly stops working?
🤖 Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.
來源：DEV Community · dev.to