PAR:以進度對齊循環提升狀態追蹤的長度泛化能力
PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking
PAR 改進循環語言模型的長度外推能力,在每次循環預測相應步驟後的狀態,並重新縮放隱藏狀態。在 swap-S5 任務中,0.5B 模型以最多 32 步的序列訓練,在 512 步達到 95.5% 完整狀態準確率;同一設定下,RLT 為 55.7%,CoT 為 0.8%。PAR 亦可只用最終答案監督訓練,毋須中間狀態標註。
🚀 Introducing PAR, our new work on length extrapolation in looped language models.
📦 The task is simple: five boxes, one object in each. Follow a sequence of swaps, then answer what’s inside. We train the model on short sequences and test whether it can keep up as the sequences get much longer.
Simply running more loops is not enough: performance breaks down at just 2× the training length. Scaling up the model alone doesn’t solve the problem either. On single-query S5, fixed-depth Qwen2.5 models from 0.5B to 7B achieve only around 20% accuracy at 16× the training length—roughly random guessing.
We identify two key obstacles: supervising only the final answer leaves the model without clear guidance on how far to advance in each loop; repeatedly applying the same layers hundreds of times causes the hidden-state scale to drift.
💡 PAR makes two straightforward changes: at loop k, predict the state after the first k steps; after each loop, rescale the hidden states to match their scale in the first loop. This aligns the loops with task progress: one loop, one step.
📈 On swap-S5, a 0.5B model trained on sequences of at most 32 steps achieves 95.5% full-state accuracy at 512 steps—a 16× length extrapolation. For comparison, the previous best method, RLT, achieves 55.7%, while the same model using chain-of-thought (CoT) reaches just 0.8%.
On single-query S5, PAR substantially outperforms fixed-depth models at every tested size, spanning Qwen2.5 0.5B, 1.5B, 3B, and 7B, and Qwen3 models from 0.6B to 4B. Across all four Qwen2.5 sizes, the best training runs achieve 100% accuracy at 16× the training length.
The model can also learn when to stop. We further show that PAR can be trained with final-answer supervision alone, without intermediate-state annotations.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co