跳到正文
DEV Community· Papers Mache·· 12 天前AI 評分45

TailSteer 減少多輪圖片編輯漂移;History Forcing 穩定長時程影片生成

TailSteer reduces drift in multi‑turn image editing; History Forcing stabilizes long‑horizon video generation

AI 導讀

TailSteer 透過從 on-policy rollout 學習針對尾段的修正,減少連續 15-20 次圖片編輯累積的漂移,並大致保留原模型能力。History Forcing 將 15 秒影片生成的 VBench 質素提升至 85.07,MemoBench 物件一致性達 62.18,Evolution Quality 由 2.35 升至 2.70;每步訓練時間和記憶體開銷只增加約 2%。

正文

Papers Mache

Multi‑turn image editing and long‑horizon video rollouts still suffer from drift that compounds after just a handful of steps, making iterative creative tools brittle. TailSteer preserves the early denoising dynamics while History Forcing injects historic context, effectively halting that drift.

Earlier solutions either retrain on multi‑turn data—rewriting the pretrained priors—or rely on inference hacks that do not scale to large diffusion backbones. Those workarounds trade off fidelity for stability and often require hand‑crafted rollout pipelines.

TailSteer cuts cumulative drift across 15‑20 successive edits. It learns tail‑focused corrections from on‑policy rollouts, leaving the early denoising trajectory untouched; experiments across the FLUX, SenseNova, and Qwen families show a substantial reduction while largely preserving base‑model performance [1].

TailSteer maintains the generative and editing abilities of the original model. Its tail‑focused design “improves long‑horizon editing robustness while largely preserving the generative and editing capabilities acquired through earlier training,” even surpassing GPT‑Image‑2.5 Sunburst on all three preservation metrics, proving that long‑horizon stability can be unlocked without erasing prior knowledge [1].

History Forcing lifts the VBench quality of 15‑second video generation to 85.07. By feeding the residual between teacher predictions with and without informative history into the student’s distillation target, the method aligns the student to a corrected prediction that respects long‑range constraints.

Object consistency rises to 62.18 on MemoBench when the camera returns, while Evolution Quality improves from 2.35 to 2.70, demonstrating more plausible scene evolution during out‑of‑sight intervals; training overhead grows by only ~2 % in time and memory per step [2].

TailSteer has only been validated up to 20 edit turns, leaving behavior on longer sequences uncertain, and History Forcing depends on a teacher that retains visibility of the scene—a condition that may not hold for arbitrary content changes; together this suggests further research is needed to scale tail‑steering beyond dozens of edits and to generalize historical forcing without specialized teachers.

Future iterative video editors should replace naïve autoregressive rollouts with a History Forcing‑distilled student and fine‑tune diffusion backbones using TailSteer before exposing them to user‑driven edit sequences, ensuring stable 60‑second generation and consistent visual quality.

References

  1. TailSteer: Trajectory-Tail Steering for Drift-Resistant Multi-turn Image Editing
  2. History Forcing: Distilling History-Dependent Continuations for Long-Horizon Video World Models

來源:DEV Community · dev.to