跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 5 天前AI 評分47

M3OPD-Reasoner:以多教師 on-policy 蒸餾進行多模態推理後訓練的開放方案

M3OPD-Reasoner: An Open Recipe for Post-Training Multimodal Reasoning with Multi-Teacher On-Policy Distillation

AI 導讀

M3OPD-Reasoner提出一套完全開放的多模態推理後訓練方案,涵蓋圖像 STEM、圖像感知及影片三個領域。評測 21 個基準後,研究發現從 Qwen3.5-9B 直接以 GSPO 訓練各領域教師,再用 MOPD 蒸餾的方案表現最佳;接續 MixRL 仍可提升學生模型,形成 M3OPD-Reasoner-9B。

正文

Published on Nov 17

Authors:

,

,

,

Abstract

Multi-Teacher On-Policy Distillation (MOPD) enables a single student to absorb the skills of several domain teachers by learning from its own generated responses. We present Multimodal Model Multi-Teacher On-Policy Distillation Reasoner (M3OPD-Reasoner), a fully open recipe for post-training vision-language models (VLMs) over three reasoning domains: image STEM, image perception, and video. We curate two datasets, M3OPD-SFT-1.35M for supervised fine-tuning (SFT) and M3OPD-RL-193K for reinforcement learning (RL) with verifiable rewards. On these data, we compare post-training recipes that start from Qwen3.5-9B: training one model on the mixed data of all domains (MixSFT and MixRL), and training one teacher per domain with Group Sequence Policy Optimization (GSPO), directly or after an SFT warm-up, and distilling the teachers into one student with MOPD. We evaluate on 21 image STEM, image perception, and video reasoning benchmarks after we decontaminate the training data against them. The comparison establishes MOPD of teachers trained with GSPO directly from Qwen3.5-9B as the best recipe: the student closely matches each teacher on its own domain and is on average above the mixed-training baselines on the same prompts. Since MOPD already merges the domain capabilities that MixRL would otherwise have to learn, its student is a stronger starting point for RL, and we ask whether the same data can carry the model further from there: we add MixRL on top of the MOPD student, on the same prompts and with the same settings, which, to our knowledge, no prior VLM recipe has studied. The same prompts still improve the MOPD student, with no collapse, and give our main model, M3OPD-Reasoner-9B. MOPD and MixRL can therefore work as consecutive stages that get more out of the same data. We release all data, the training code, and the checkpoints of every stage.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2611.23034

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 16

ngqtrung/M3OPD-MixSFT-9B

Image-Text-to-Text •

9B •

Updated about 1 hour ago

•

1

ngqtrung/M3OPD-Warmup-SFT-Video-9B

Image-Text-to-Text •

9B •

Updated about 1 hour ago

•

46

ngqtrung/M3OPD-Teacher-ImageSTEM-9B

Image-Text-to-Text •

9B •

Updated about 1 hour ago

•

38

ngqtrung/M3OPD-Teacher-ImagePerception-9B

Image-Text-to-Text •

9B •

Updated about 1 hour ago

•

36

Browse 16 models citing this paper

Datasets citing this paper 2

ngqtrung/M3OPD-SFT-1.35M

Viewer •

Updated about 1 hour ago

•

1.35M

•

120

ngqtrung/M3OPD-RL-193K

Viewer •

Updated about 1 hour ago

•

193k

•

116

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.23034 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co