跳到正文
DEV Community· Papers Mache·· 57 分鐘前AI 評分38

P²-MOPD 以 on-policy 蒸餾加快 MoE 恢復

On‑policy distillation speeds up MoE recovery

AI 導讀

P²-MOPD 在分佈轉移後以約一半的訓練步數,讓多教師學生模型達到相同的恢復表現。1.7 B 參數模型完全收斂時的平均恢復率為 94.7%,高於 MOPD 的 80.9%。評估涵蓋四個領域及 1.7 B、4 B 兩種模型規模。

正文

Papers Mache

Treating expert weights as priors cuts the time needed to recover a multi‑teacher student after a distribution shift by roughly half. The shortcut comes from jointly learning mixing coefficients and student parameters while keeping the experts fixed as priors, allowing cheap updates that remain on‑policy.

Before P²‑MOPD, practitioners relied on vanilla multi‑teacher on‑policy distillation (MOPD), which repeatedly samples trajectories from a frozen ensemble of experts and trains a fresh student until it matches their performance. A competing line of work—task arithmetic—combines parameters without any further training but forces the result to stay inside the linear span of the originals, limiting flexibility.

P²‑MOPD “achieves faster and more balanced recovery than MOPD from the same initialization, surpassing its final average recovery in half the training steps” [1]. By treating each expert as a parameter prior while still allowing the student to move beyond the frozen subspace, the method reaches the same performance ceiling with only fifty percent of the usual distillation budget.

At full convergence the 1.7 B‑parameter model attains “94.7% average recovery, versus 80.9% for MOPD” [1], showing that speed does not come at the expense of final quality. The gap widens the practical appeal of on‑policy distillation for production pipelines where both latency and accuracy matter.

The evaluation is confined to four domains and two model scales (1.7 B and 4 B) [1], leaving open how well the prior‑based approach generalizes to larger experts, more heterogeneous data shifts, or continual learning settings. Moreover, while inference cost stays unchanged, each distillation step still incurs the same compute as vanilla MOPD, so total wall‑clock savings depend on the exact training schedule.

If rapid model refreshes are a requirement, replace the standard MOPD loop with P²‑MOPD in any update benchmark and report both steps‑to‑recovery and final recovery. The expected half‑step speedup makes it feasible to run daily or even hourly updates without inflating infrastructure budgets.

References

  1. P^2-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors

來源:DEV Community · dev.to