跳到正文
DEV Community· Savindu Thathsara·· 3 小時前AI 評分35

Hearthlen 回覆緩慢,原因是 CPU 運算時間

The Slow Reply Was CPU Time

AI 導讀

Hearthlen 加入 Sinhala 後,回覆緩慢並非因為記憶體不足,而是模型運算耗時。雙語語音模型處理短片段用了 2407 ms,Qwen2.5-3B 生成 Sinhala 回覆用了 9564 ms;改回較小的英文語音模型後,仍由 3B 模型生成回覆,因此回覆仍需數秒。

正文

The Slow Reply Was CPU Time

Thirteen gigabytes of memory were still free, and the answer was still slow.

Savindu Thathsara, savithathsara.me. This is a Hearthlen note from the laptop.

Sinhala had just been added beside English, and a question took long enough to feel stuck. The check was /proc/meminfo: 13.0 GB available out of 15.5 GB. The listener was 1507 MB. The language model was not loaded until someone spoke. So this was not a machine that had run out of memory.

The time was the models doing work. The bilingual speech model spent 2407 ms on a short clip, because it also had to guess the language. A Sinhala reply from Qwen2.5-3B took 9564 ms. English is the only speech path again, on the smaller English model. After that restart the listener was 1331 MB, and 13.0 GB was still free. The 3B model is still what thinks, so a reply is still a few seconds. It is no longer also paying for a second language. The bilingual note is English and Sinhala.


Savindu Thathsara builds practical AI and automation in London. This series is the public log of an offline AI operating system he is writing and learning at the same time.

來源:DEV Community · dev.to