ZAYA1-8B-Diffusion-Preview 在 AMD 上實現高效並行解碼

ModelZAYA1-8B-Diffusion-Preview: Efficient Parallel Decoding on AMDZyphra presents ZAYA1-8B-Diffusion-Preview, a preview of our early work in diffusion-language models. We demonstrate that we can convert the existing ZAYA1-8B language model, which was trained autoregressively, into a discrete diffusion model with no systematic loss of evaluation performance. ZAYA1-8B-Diffusion-Preview diffuses blocks of 16 tokens simultaneously resulting in a 4.6x speedup with a lossless sampler and 7.7x speedup with our new logit-mixing sampler. ZAYA1-8B-Diffusion-Preview is the first MoE diffusion model converted from an Autoregressive LLM and the first diffusion-language model trained on AMD.May 14, 2026

來源：Zyphra Research · zyphra.com