LFM2.5-24B-A2B:為快速裝置端智能體而設
LFM2.5-24B-A2B: Built for fast on-device agents
LFM2.5-24B-A2B 面向裝置端智能體,在裝置端比同類 MoE 快至 1.7×,於 Apple M5 Max 達 184 tokens/s,單張 NVIDIA B300 GPU 的輸出速度則達 21K tokens/s。它在 AutomationBench 表現突出,但評論者指出,該基準測試的評測內容未獲説明,模型在其他基準測試的得分則與其他本地模型相近。
> it runs up to 1.7× faster than comparable MoEs on-device, with 184 tokens/s on an Apple M5 Max and up to 21K output tokens/s on a single NVIDIA B300 GPU
Wow, those are seriously impressive numbers for how intelligent this model seems to be!
One thing that this article doesn't clarify, and that I guess I really should look into, is what exactly AutomationBench tests for as compared to other benchmarks. The standout performance on AutomationBench is interesting to see, but I don't know what it really means because the model seems to score relatively close to other local models on every other benchmark.
來源:Hacker News:AI 熱帖 · news.ycombinator.com