跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-20AI 評分46

Lafa:大型流語言模型

Large Flow Language Models

AI 導讀

Lafa 是一款大型連續流語言模型,透過區塊式流設計、自我投機解碼及多 token 預測,實現高效訓練、高品質生成與更快推理。模型以少於 30B tokens 訓練,涵蓋 0.5B、7B 和 8B 規模;在品質相若的情況下,自我投機解碼的吞吐量達自回歸模型的 2.54x。

正文

Published on Oct 20

Authors:

,

,

,

,

,

Abstract

A large-scale continuous flow language model achieves efficient training, high-quality generation, and faster inference through blockwise flow design, self-speculative decoding, and multi-token prediction.

Continuous flow language models offer a path to parallel generation with tools for control and acceleration unavailable to discrete diffusion, yet effective techniques for building them at the billion-parameter scale remain less established. We present Lafa (Large flow language model), a fast, high-performing flow language model scaled to 8B parameters in a data-efficient manner. Lafa is a blockwise flow model with causal attention on the prefix and bidirectional, asynchronous flow within each block. This design enables efficient training by adapting pretrained autoregressive (AR) models and fast inference via prefix caching and blockwise parallel decoding. We train Lafa at 0.5B, 7B, and 8B scales with fewer than 30B tokens, more efficient than previous diffusion models, and evaluate it on instruction following, reasoning, and coding, where the learned flow matches or surpasses the quality of AR and can be distilled into flow maps for one- and few-step generation. Notably, flow training preserves or even improves pretrained AR capabilities via a multi-token prediction effect, which we leverage in a self-speculative decoding scheme that verifies flow-based drafts autoregressively within a single model. This yields 2.54x the throughput of an AR model at comparable quality, outperforming multi-token prediction and discrete diffusion methods. Our findings show that continuous flows can serve as a scalable foundation for language modeling.

View arXiv page View PDF GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.28559

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 3

lafa-hf/lafa-0.5B

Text Generation •

0.7B •

Updated 10 minutes ago

•

7

lafa-hf/lafa-7B

Text Generation •

8B •

Updated 10 minutes ago

•

6

lafa-hf/lafa-8B

Text Generation •

8B •

Updated 10 minutes ago

•

6

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.28559 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.28559 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co