跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-09-29AI 評分44

不止提示詞數量:數據如何影響在策略蒸餾中的遷移

Beyond Prompt Count: How Data Shapes Transfer in On-Policy Distillation

AI 導讀

研究發現,在策略蒸餾(OPD)的遷移效果受提示詞數量、來源和選擇影響,提示詞效用亦取決於教師—學生模型組合及目標任務。僅以“What is 1+1?”作為訓練提示詞,Qwen3-4B 的數學評分由 15.47% 升至 47.93%,接近教師的 48.33%;訓練仍包含反覆學生生成和教師監督。均勻隨機抽樣仍具競爭力,針對性選取未能持續改善遷移效果。

正文

We study how prompt count, source, and selection shape transfer in on-policy distillation (OPD).

One motivating result: using “What is 1+1?” as the only training prompt raises Qwen3-4B’s math score from 15.47% to 47.93%, close to the teacher’s 48.33%. This still involves repeated student rollouts and teacher supervision—not a single training update.

But prompt efficiency does not imply prompt interchangeability. Our main findings:

• Prompt utility depends on the teacher–student pair and target task. Changing only the teacher can reverse the relative effectiveness of math and code prompts.

• Teacher-aligned prediction changes can emerge through very different parameter updates.

• We probe failure and recovery at the representation level. Hidden-state interventions partially improve a degraded model on a small probe. In a separate experiment, continued OPD on effective prompts restores code performance from 50.03% to 63.16%, without retracing a successful run’s parameter path.

• Uniform random sampling remains competitive. Targeted selection—including semantic diversity, difficulty, response length, teacher–student disagreement, and gradient-based selection—does not consistently improve transfer in our experiments. Filtering out a source that performs poorly alone also gives no consistent gain.

Have you observed similar sensitivity to prompt choice in your own OPD experiments? Happy to discuss the results and implementation!

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co