跳到正文
DEV Community· Mohamed Mokbel·· 6 天前AI 評分40

Cutora 如何決定從鏡頭前講話影片中剪掉哪些內容

Deciding what to cut in a talking-head video: pauses are easy, retakes are not

AI 導讀

Cutora 以逐字稿中的詞語起止時間規劃鏡頭前講話影片的初剪,刪除長停頓、口誤重説及重複錄製片段,並保留短停頓。判斷重複錄製時會按五項因素評分;若不確定兩段是否相同,便保留兩段,所有刪除內容也會列在時間軸上供還原。初剪仍可能漏掉改寫過的重複句或保留錯誤版本。

正文

I'm building Cutora, an editor that makes the first cut of a talking-head video. Most of the interesting problems turned out not to be "AI" problems at all. They're timing problems. Here's how the cut is decided, and where it still goes wrong.

Step 1: words with times, not just text

Everything starts from a transcript where every word has a start and end time. The cut is planned on words, then rendered on media. If the word times are wrong, every later rule inherits the error, so most of our bugs live here, not in the "smart" parts.

Step 2: pauses

Long pauses get cut, with a short beat left around the words so speech doesn't sound stapled together. Short pauses stay, because they're how people breathe. We don't expose a millisecond threshold; creators pick a pacing and can restore any single cut on the timeline.

The trap: a loudness threshold alone will happily eat a quiet last word. Planning on word times instead of on the waveform avoids most of that.

Step 3: false starts

A false start is a sentence you begin, drop, and restart with the same opening words. The abandoned attempt goes, the finished sentence stays.

The bug that taught us the most: someone says the same greeting three times in a row before starting. Our rule compared a restart only with the words right after it, so a triple opener slipped through. The fix was to treat a run of near-identical openings as one group and keep the last complete one. Obvious in hindsight. It took a real recording to find it.

Step 4: repeated takes

When a line is said more than once in nearly the same words, we score each take on five things: completeness, fluency, how close its pace is to the other takes, how clearly the words were recognised, and loudness. The highest score stays. When two takes score close, a model also listens and looks before deciding.

Two rules matter more than the scoring:

  • If we're not sure two takes are the same line, we keep both. A missed cut costs the creator one click; a wrong cut costs them a sentence they meant to say.
  • Takes that say different things are never treated as repeats, even if they start the same way.

Step 5: guards before render

Before anything renders, a check looks for cuts that land inside a word and moves them to the word boundary. It sounds trivial. It caught real clicks.

Where it still fails

  • Word timestamps from alignment sometimes stretch a word across a silence, so a short stumble hides inside a "long" word and survives the cut. We're working on it; it isn't solved.
  • It can't know what you meant, so it can keep the wrong take for a reason it can't see.
  • It doesn't find every repeated line, especially when the second take is reworded.

That's why every removed piece is listed on the timeline and can be put back. The edit is a proposal, not a verdict.


Disclosure: I build Cutora. The full write-up of how the first cut decides is here: How the first cut decides what to remove

來源:DEV Community · dev.to