OpenNVR 在 4 核 CPU 上測試攝影機智能體模型，較大模型反而落敗
We tested every small model our camera agent uses on a 4-core CPU. The bigger model lost.
OpenNVR 維護者在無 GPU 的 4 vCPU 電腦上測試攝影機智能體依賴的模型，測試後有四項預設設定改變。語言模型測試中，granite4:micro 得分 44 / 60，舊預設 qwen3:1.7b 得分 21 / 60，而 qwen3:4b 得分 9 / 60；作者指出，模型在決策前是否調用工具，比參數量更關鍵。
I maintain OpenNVR, an open source video recorder with a camera agent built in. You can ask it what's happening on a camera, arm an alarm, search old footage, and soon it will look into events on its own.
Most people who run it don't have a GPU. They have a small PC with 4 cores. So for this round we tested every model the agent depends on, using a 4 vCPU machine with no GPU. Until now we had picked models by reputation, like most projects do. We wanted numbers that anyone in open source can check, so every test script is in the repo.
Four of our defaults changed. Some results were not what we expected.
How we tested
One small test per job:
- Language model: can it use tools and finish a multi-step investigation?
- Vision model: questions with known answers on labelled images.
- Speech to text: word error rate on the phrases people actually say to the agent.
- Text to speech: is it faster than real time, and is it understandable?
- Search: find the right image from a text description.
All timings are on 4 CPU threads.
Language model
We gave it 20 investigation cases, for example "person at the loading bay at 01:14". It has to check the camera, the history and the site rules, then decide. Each case ran 3 times.
| Model | Passed | Checked before deciding |
|---|---|---|
| granite4:micro | 44 / 60 | 56 / 60 |
| granite4:tiny-h (7B MoE, 1B active) | 35 / 60 | 49 / 60 |
| qwen3:1.7b (our old default) | 21 / 60 | 36 / 60 |
| LFM2-1.2B-Tool | 19 / 60 | 37 / 60 |
| qwen3:4b | 9 / 60 | 9 / 60 |
qwen3:4b scored less than half of qwen3:1.7b. When we read the traces, the reason was simple. It thought for a long time and almost never called a tool, so it decided without looking. Granite 4 micro calls about two tools before it answers, and that alone more than doubled the pass rate. It is about twice as slow per turn. For an investigation that runs in the background, that's fine, so it is our default now.
We also found a bug in our own agent. We had told the model to always look before describing, so when a user said "thanks, that's all", Granite went and checked a camera. Now small talk goes to the model without any tools. That helped every model, not just Granite.
Vision model
128 labelled images, 3 questions each: one about something in the picture, one about something that isn't (a "yes" here is a hallucination), and a people count.
| Model | Wrong "yes", of 126 | Count within one, of 61 | CPU time per new frame |
|---|---|---|---|
| qwen3-vl:2b-instruct | 0 | 50 | 18 s (follow-up questions are free) |
| gemma3:4b | 8 | 48 | 17 s |
| moondream 0.5b | 22 | 42 | 7.6 s |
Qwen3-VL never said yes to something that wasn't there. On a GPU it answers in about 3 seconds, so it's the default when a GPU is present. But on 4 cores it needs 18 seconds just to read the frame. If someone is at the gate, an answer after 18 seconds is too late to be useful. So on CPU we kept Moondream.
One tip if you use Ollama: the plain qwen3-vl:2b tag is the thinking version and ignores think: false. Use the -instruct tag.
Speech to text
| Model | Word error rate | Time per sentence |
|---|---|---|
| Parakeet TDT 0.6B v3 (onnx-asr, int8) | 1.4% | 0.31 s |
| faster-whisper base.en (int8) | 3.3% | 1.42 s |
| faster-whisper small.en | 3.7% | 2.67 s |
This was the easy decision. Half the errors in about a fifth of the time. Speech recognition used to be the slowest step in a voice turn and now it's a third of a second. Parakeet can't translate and doesn't do Hindi, so Whisper is still there for sites that need that.
Text to speech
Kokoro-82M sounds nicer than Piper. But on 4 cores it runs at 1.9x real time, so it takes longer to make the audio than to play it, and replies would pause mid-sentence. Piper runs at 0.23x. We kept Piper.
Search and detection
For searching footage by description, like "red truck at the dock", SigLIP 2 B/16 scored 0.672 mAP against CLIP ViT-B/32's 0.571. It's slower per image (57 ms against 26 ms), but search quality matters more here. It's the default now, and OpenNVR re-embeds old footage automatically after the upgrade.
The always-on detector moved from YOLOv8n to YOLO26n: 62 ms per frame instead of 75, slightly more accurate, same output format.
What we learned
- For agents, size wasn't what mattered. Whether the model checks with its tools before deciding mattered more than parameter count.
- On a camera, a slow answer is a bad answer. A good answer at 8 seconds beat the best answer at 18.
- Test on the hardware your users have. Most of these rankings flip on a GPU, and most of our users don't have one.
All the scripts and raw numbers are in the repo: MODELS_AND_LATENCY.md. If you run them on other hardware, please send a PR with your numbers.
One gap we couldn't close: a vision model under 2B parameters that reads a frame in under 5 seconds on CPU and still doesn't hallucinate. If you know one, tell me in the comments.
Originally published on opennvr.org. Drafted with AI assistance; all measurements are from the OpenNVR repository.
來源：DEV Community · dev.to