跳到正文
DEV Community· AI-KSK·· 5 天前AI 評分60

LTX 2.3 Kit 重建解析:三階段渲染、10 秒人物身份保持與 Flux 2 八步圖像編輯

LTX 2.3 Kit Rebuild: Three-Stage Rendering, 10-Second Likeness Hold, and an 8-Step Flux 2 Editor

AI 導讀

AI-KSK 重建並解析包含九個工作流的套件,涵蓋 LTX 2.3 三階段影片渲染、約 10 秒人物身份保持,以及 Flux 2 八步圖像編輯。三階段流程先以低解像度建立構圖和動作,再升頻及分段精修;音訊亦在同一工作流中生成,與畫面合併後一併取樣。

正文

LTX 2.3 Kit Rebuild: Three-Stage Rendering, 10-Second Likeness Hold, and an 8-Step Flux 2 Editor

A video model that produces audio and picture in a single pass changes how you structure a workflow. LTX 2.3 does exactly that, and this kit rebuild is mostly about learning to work with that rather than around it.

Nine workflows are in the set. They are not variations on one graph — they are separate jobs that happen to share a model family. This article walks the ones with the most interesting internals.

What the model actually loads

Before the graphs make sense, three files matter:

File Role
ltx-2.3-22b-dev-dare-ties-distilled-1.1.safetensors The 22B AV checkpoint, distilled
ltx-2.3-spatial-upscaler-x2-1.1.safetensors Latent-space 2× upscaler used between stages
ltx-2.3-22b-distilled-lora-384.safetensors Distillation LoRA that makes the step counts viable

The distilled checkpoint is what makes a three-stage render practical. A non-distilled 22B at full step count would make the three-pass structure unusable in a workstation context.

Three-stage rendering, and why it is three and not one

The text-to-video 3.1 graph is 88 nodes with 155 links, and it runs SamplerCustomAdvanced three times with separate ManualSigmas and RandomNoise nodes per stage. The structure is deliberate:

  1. Stage 1 — structure. Low resolution (EmptyLTXVLatentVideo starts around 320×288 at 97 frames), full guidance. This pass decides composition and motion.
  2. Stage 2 — upscale. LTXVLatentUpsampler with the spatial-upscaler-x2 model lifts the latent, then a second sampler refines at the higher resolution with a reduced noise budget.
  3. Stage 3 — finish. The final pass at full resolution, again with a smaller denoise budget than the stage before it.

The reason to split rather than run one long pass at target resolution: at high resolution the sampler spends its budget on pixel work instead of motion, and the result is a softer action than the same prompt produces at low resolution.

Two details in this graph are worth stealing for other pipelines:

  • LTXLatentAnchorAware is loaded with an explicit parameter set (anchor strength, schedule mode, step counts). This is what keeps the subject from drifting between passes — the anchor is re-applied at each stage boundary rather than conditioned once at the start.
  • Audio latents are generated in the same graph. LTXVEmptyLatentAudio takes frame count, sample rate and batch, and LTXVConcatAVLatent joins picture and audio latents before sampling. The audio track is part of the render, not a post-process.

Sampler throughout is euler_ancestral_cfg_pp, and decode is tiled (VAEDecodeTiled at 512/64/512/4) because a full 97×24fps AV decode at this size does not fit in one allocation.

10-second likeness hold: what changed

The single-person digital-human graph is the largest in the set at 98 nodes, and its job is specific: hold a character's face and voice for roughly ten seconds of speech or singing.

Three mechanisms do the work:

  • Ltx2.3-Licon-VBVR-I2V-240K-R32.safetensors — the image-to-video likeness LoRA. This is the part that carries identity rather than just appearance.
  • MelBandRoFormerSampler + LTXVAudioVAEEncode — the audio path. Instead of conditioning on a plain audio latent, the graph runs a vocal-separation sampler and re-encodes the isolated voice. That matters for singing, where the instrumental bed would otherwise bleed into the mouth timing.
  • SetLatentNoiseMask with SolidMask — region-controlled re-noising. The graph pushes noise back selectively, so the face region can be refined without re-rolling the background.

The honest limitation: this is a hold, not a lock. At ten seconds the identity is stable; past roughly that, drift accumulates in the same way it does in any reference-conditioned video pipeline. If you need longer, chain two generations with an anchor handoff rather than stretching one.

The 8-step Flux 2 editor

The Flux 2 graph is the small one in the set — 26 nodes — and it is a different kind of tool: single-image editing at roughly 1 megapixel, using a step-distilled checkpoint.

What it loads:

  • flux2_distilled_8step_fp8mixed.safetensors — the distilled UNET
  • mistral_3_small_flux2_fp8.safetensors — the text encoder
  • flux2-vae.safetensors
  • linflux_000004500.safetensors — a LoRA at weight 1.0

The conditioning prompt is itself a JSON object describing the edit task, the source image reference and the test focus. That is worth noting: the editor is driven by a structured instruction, not a sentence.

The pipeline is LoadImage → ImageScaleToTotalPixels → VAEEncode → ReferenceLatent → SamplerCustomAdvanced → VAEDecode, with BasicGuider and euler. Two ImageScaleToTotalPixels + ReferenceLatent branches exist, which is the mechanism for supplying both a source and a reference target.

Where 8-step distillation costs you: fine texture and small text degrade before composition does. At this resolution that is usually acceptable for a working draft, and the trade is that iteration is fast enough to actually explore.

视频扩展 3.0 / 视频重说话 3.0

Two utilities round out the set:

  • Video extension with aspect widening. Uses OmniNFT + Relay + a distillation LoRA to widen a shot to vertical framing. This is not a crop — the width is generated.
  • Re-talk. Same stack, applied to replacing spoken words while keeping the lip sync aligned to the new audio.

Both are the kind of tool you reach for at the end of a project, not the start.

When this kit is the right choice

  • You need picture and audio from one graph, with the timing between them generated rather than aligned after the fact.
  • You are willing to run three passes for a shot that matters.
  • You need a character to hold for roughly ten seconds of speech.

It is the wrong choice for a quick single-clip test where a lighter model would iterate faster, and it is the wrong choice if you need identity held past ten seconds — that is where you chain generations.

Troubleshooting notes

  • Soft or mushy output after upscale — check that stage 2 and 3 are using reduced sigmas. Running full noise at the upscaled resolution re-rolls the composition instead of refining it.
  • Audio drift against mouth movement — the audio latent and the picture latent must be concatenated before sampling. Generating audio separately and muxing afterwards breaks the timing the model produced.
  • Identity drifts early — the anchor params on LTXLatentAnchorAware are the first thing to check; a low anchor strength shows up as drift within the first few seconds, not at the end.
  • OOM at decode — tiled decode settings (512/64/512/4 in this graph) are the lever, not lowering the render resolution.

Workflows

Nine graphs are published on RunningHub with invite code rh-v1111:

  • Single-person digital human with 10s likeness hold
  • Image + text video with likeness hold
  • Text-to-video 3.1 with three-stage HD refinement
  • Multi-image reference update with 10s likeness lock
  • Storyboard director video (character, action, scene control)
  • Flux 2 timestep distillation
  • Video extension 3.0 (OmniNFT + Relay + distillation, vertical widening)
  • Video re-talk 3.0 (word replacement with lip sync)
  • IC Edit watermark and subtitle removal

Files

The ComfyUI workflow exports live in the companion repository:

https://huggingface.co/AI-KSK/ksk-video-255-comfyui-workflows

Video walkthrough (Chinese): https://youtu.be/JflBem1RPZc
Bilibili: https://www.bilibili.com/video/BV1nVVr6QEd8/

Notes

  • Workflow files only. Model weights are not included and each model keeps its own licence.
  • These are tutorial exports, not a packaged product. Node parameters match the video walkthrough.
  • Where a performance characteristic is described, it is derived from the graph structure and the models it loads; measured throughput numbers are not claimed here.
  • Independent AI-KSK release. Not affiliated with or endorsed by Lightricks, Black Forest Labs or RunningHub.

來源:DEV Community · dev.to