跳到正文
DEV Community· AI-KSK·· 42 分鐘前AI 評分52

Qwen3 TTS 與 LTX 2 如何串接兩個工作流製作對話鏡頭

LTX 2 lip sync plus a cloned voice: building a talking shot from two workflows

AI 導讀

作者示範將 Qwen3 TTS 聲音克隆與 LTX 2 唇形同步串接,從樣本生成新語音,再用圖像或影片及音軌製作配合語音的對話鏡頭。建議先克隆聲音並生成完整台詞,再用最終音訊進行唇形同步;爆破音和齒擦音是同步容易出錯的位置。兩者以音訊檔交接而非單一管線;聲音會承襲樣本錄音品質,唇形同步在大幅轉頭時可能於畫面邊緣失準,且此做法面向 talking-head,並非全身表演。

正文

AI-KSK

Two workflows, one result. The first clones a voice from a sample with Qwen3 TTS. The second drives lip movement with LTX 2. Together they make a talking shot where both the face and the voice are derived from your own material.

Watch the episode: https://youtu.be/iHM2VOtUAZ0

The two halves

  1. Qwen3 TTS voice clone - a short sample becomes a voice you can generate new speech with. Clean audio in, usable audio out.
  2. LTX 2 lip sync - an image or clip plus an audio track becomes a moving mouth that matches the words.

Running them in sequence is what makes the result feel like one performance instead of a dub.

The order that works

  1. Clone the voice first, generate the full line.
  2. Render the lip sync against that final audio - not against a placeholder.
  3. Check the hard consonants: plosives and sibilants are where lip sync visibly fails.

What to watch for

  • The clone inherits the recording. A noisy sample gives a noisy voice.
  • Lip sync is convincing on the face and falls apart at the frame edges when the head turns hard.
  • Mouth shapes follow the phonemes, not the emotion; flat delivery reads as flat even with perfect sync.

Honest limits

  • These are two separate workflows, not one pipeline; the handoff is the audio file.
  • No model weights are bundled here.
  • This is a talking-head tool. Full-body performance is a different problem.

來源:DEV Community · dev.to