N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models
N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models
Conditional Program(CP)以共享重建程序、短碼和局部尺度,按輸入需求重建語言模型嵌入表的資料列,將儲存量減少 75%。CP 的兩位元編碼相較兩位元最接近值取整,令輸出 KL 降低 41--63%。應用於 DeepSeek 時,查找表儲存量由 203GB 減至 51GB。
Published on Oct 16
·
on Oct 20
Authors:
,
,
,
Abstract
Conditional Program compresses language-model embedding tables by reconstructing only requested rows via shared codes and local scales, reducing storage by 75% while improving fidelity.
Conditional memory expands language-model capacity through selective lookup, but the full embedding table must remain available for every possible input. We introduce Conditional Program (CP), a compressed representation that reconstructs only the rows an input requests. CP represents each row by a short code and a local scale, with a reconstruction program shared across the table. We instantiate this representation with a Gaussian trellis, norm restoration and independently decodable row records. CP's two-bit code lowers output KL by 41--63\% relative to 2-bit round-to-nearest. Analysis across three trained n-gram memory families shows that Gaussian-like coordinate statistics coexist with learned relationships among rows; at matched storage budgets, the shared trellis outperforms the fitted codebooks and low-rank residual variants we test. For DeepSeek, it reduces table storage from 203GB to 51GB, leaving room for larger workloads when the table is kept on GPU.
View arXiv page View PDF Add to collection
Get this paper in your agent:
hf papers read 2610.21810
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.21810 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.21810 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.21810 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co