Llama-3 8B 在無獨立 GPU 的四核心手提電腦 CPU 上生成速度達 6.1–6.5 tokens/sec
An 8B LLM on a 4-core laptop CPU, no GPU: 6 tokens/sec
作者在配備 i7-1165G7、12 GB RAM、沒有獨立 GPU 的 HP 手提電腦上運行 8B Llama-3（Q4_K_M），新對話生成速度達 6.1–6.5 tokens/sec。
I ran an 8-billion-parameter Llama-3 model (Q4_K_M) on an HP laptop with an i7-1165G7, 12 GB of RAM and no discrete GPU.
| Measurement | Result |
|---|---|
| Writing speed (fresh chat) | 6.1–6.5 tokens/sec |
| Memory, full 8,192-token context | About 4.6 GB |
| CPU power cap | 20 W long-term, 22 W short-term |
At 6 tokens/sec, output runs at roughly reading speed.
Three results surprised me.
The integrated GPU was slower. Iris Xe read prompts nearly 4× faster than the CPU but wrote answers 2.5× slower. Writing is limited by memory bandwidth, and an integrated GPU shares the CPU's memory.
The 8-bit KV cache cost more than it saved. It looked free in short benchmarks. On a real 1,700-token summarization task, the 16-bit cache finished in 173 seconds instead of 234 and ran 7 °C cooler.
Capping CPU power cost nothing. Cutting the short-term limit from 51 W to 22 W eliminated the 98 °C startup spikes, and writing speed was unchanged.
If you're interested in more information, the full write-up has hardware details, every benchmark table, the exact llama.cpp commands, and a systemd service for the power cap:
來源：DEV Community · dev.to