Orin AI（AE-Assistant）：我如何從零打造小型語言模型，以及它如何運作
** I built a small language model from scratch. Here's how it works.**
Orin AI（AE-Assistant）是作者從零打造、仍在開發中的小型語言模型項目，目標是在沒有專用 GPU 的一般手提電腦上本地運行。項目探索 decoder-only Transformer、自訂 BPE tokenization，並試驗循環深度架構及基於檢索的 grounding；作者亦製作影片介紹項目並展示模型運行。
I've been building Orin AI (AE-Assistant), a small language-model project designed to run locally on ordinary laptop hardware without a dedicated GPU.
The project explores a decoder-only Transformer, custom BPE tokenization, and experiments with recurrent-depth architectures and retrieval-based grounding.
I made a video walking through the project and showing the model running.
I'm still developing it, and I'd appreciate constructive feedback from people interested in small language models and training models from scratch.
What would you improve first?
來源：DEV Community · dev.to