Tiny Desk AI 影片如何以 ffmpeg 將五段 AI 剪輯合併成 571 影格影片

TL;DR

The Tiny Desk AI video trend turns five source shots into one mock concert that runs 19 seconds using edge-detected outlines. The ffmpeg work is three jobs: turn each shot into a line drawing, trim and join five AI edits at 30 fps into 571 frames, and lay NPR's live recording back under it. Here's the one trap: trim by seconds, never by frame count, because the two models hand back different frame rates and lengths.

Here are the commands in the order we run them, with the numbers we got. You can run them directly in your shell next to this post.

Turn each shot into a line drawing with two filters

# band shots
ffmpeg -i shot.mp4 -vf "scale=720:1280:flags=lanczos,gblur=sigma=2,edgedetect=low=0.1:high=0.25,format=gray" -r 24 -an lines.mp4
# soft close-ups
ffmpeg -i shot.mp4 -vf "scale=720:1280:flags=lanczos,unsharp=7:7:1.5,gblur=sigma=1.2,edgedetect=low=0.05:high=0.15,format=gray" -r 24 -an lines.mp4

We send line drawings as the reference files for the AI models. The model never sees filmed footage. It gets an edge-detected line drawing of each shot: grey outlines on black, 720×1280, 9:16, muted. No face, outfit or colour of the original people survives the filter.

Band shots use gblur=sigma=2,edgedetect=low=0.1:high=0.25,format=gray. That's a light blur first so fabric texture doesn't turn into lines, then edge detection, then grey.

The two close-ups are soft in the AI clip, so the band filter lit only about 0.4–0.9% of their pixels. The sharper chain uses unsharp=7:7:1.5,gblur=sigma=1.2,edgedetect=low=0.05:high=0.15,format=gray. It sharpens first and lowers the edge thresholds.

That brings them to about 2%, which gives the model enough outline to follow the head turns and the laugh. We scale drawings to 720×1280 with lanczos, encode them at 24 fps (-r 24), and mute them with -an.

The model gets a drawing, not the video, because stripping colour and clothes leaves only motion.

For the Wan 3.0 route, we also black out 22 objects across the five shots on every frame. These include caps, headphones and headsets, chest logos, a printed slogan, a hood, and an out-of-focus person in a corner. We use keyframed masks that keep the lines of the eyes, nose and mouth.

Between 62% and 81% of the lit pixels stay. If you leave anything in the drawing, it ends up in the video, which is why the masks exist.

Five shots, two models: the layout the commands follow

The shot layout traces the viral 19-second motion source shot for shot. TikTok creator @theburnerfromhe11 posted College Football's Week 8 All-Loss Tiny Desk Concert on 2026-10-18; it passed 2.3 million views.

Our motion source is that viral AI clip, not NPR's footage, so our video matches it shot for shot and second for second. That clip is vertical 9:16, 720×1280, 24 fps, and 457 frames (19.042 seconds). Hard cuts after frames 33, 161, 312 and 393 give five shots.

The lead singer's in shots 1 and 3, the band (keyboard player and two backup singers) is in shots 2 and 5, and the sax player's only in shot 4. One to five photos fill those five spots.

The resolution picks the engine: 480p and 720p run on Wan 3.0, while 1080p runs on Seedance 2.5. Each edit gets its own shot's photos and drawing, with the model's sound turned off.

• Wan 3.0 route: We run four reference edits side by side, keeping the drawing's motion and swapping in people from photos. Shots 2 and 5 are sent together as one 8.0 s band drawing alongside shot 1, shot 3, and shot 4.
• Seedance 2.5 route: We render five segments side by side, one per shot, requested in whole seconds. The requests are 4 s for shot 1, 6 s for shot 2, 7 s for shot 3, 4 s for shot 4, and 4 s for shot 5 (4 seconds is the model's shortest).

Sending the two band shots as one drawing keeps the same three people on both sides of the cut. We pad Seedance drawings with their last frame to hit those whole seconds. You can grab the copy-ready Tiny Desk AI prompts for every shot and role on our blog.

Trim by seconds, because each model returns a different length

AI models return different lengths and frame rates from the exact same drawings. Seedance segments come back at 24 fps and exactly one frame longer than you asked for. That means 4 s comes back as 4.0417 s, 6 s as 6.0417 s, and 7 s as 7.0417 s, so there's always one extra frame at 24 fps.

Wan edits come back at 30 fps and follow the drawing's own length. Shot 1 is 33 frames (1.375 s), but its Wan drawing is 48 frames (2.0 s). That stays clear of the shortest clip the model handles well, and we keep only the first 1.375 s in the join.

The 8 s Wan band edit holds shot 2 (128 frames) and shot 5 (64 frames) back to back. Wan drew no visible cut at 5.333 s, so the motion runs straight through. We take shot 2 from 0 s and shot 5 from 5.333 s by time, feeding the band file to ffmpeg twice as input 1 and input 4.

So a frame count means different things in different files (24 fps source, 24 fps Seedance, 30 fps Wan); seconds mean the same thing in all of them. Every trim in our join is a duration in seconds.

Join five shots with one filter graph

# Wan route, 480p: four edits, the 8 s band edit used twice (shot 2 from 0 s, shot 5 from 5.333 s)
FIT="scale=480:854:force_original_aspect_ratio=decrease,pad=480:854:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30,format=yuv420p"
ffmpeg -y \
  -i shot1.mp4 -i band.mp4 -i shot3.mp4 -i shot4.mp4 -i band.mp4 -i song.m4a \
  -filter_complex "\
[0:v]trim=duration=1.375,setpts=PTS-STARTPTS,$FIT[s0];\
[1:v]trim=duration=5.333,setpts=PTS-STARTPTS,$FIT[s1];\
[2:v]trim=duration=6.292,setpts=PTS-STARTPTS,$FIT[s2];\
[3:v]trim=duration=3.375,setpts=PTS-STARTPTS,$FIT[s3];\
[4:v]trim=start=5.333:duration=2.667,setpts=PTS-STARTPTS,$FIT[s4];\
[s0][s1][s2][s3][s4]concat=n=5:v=1:a=0[v]" \
  -map "[v]" -map 5:a:0 \
  -c:v libx264 -preset veryfast -crf 20 -c:a aac -b:a 128k \
  -shortest -movflags +faststart tiny-desk.mp4

One ffmpeg filter graph does the whole join with one re-encode.

• trim and setpts: We trim each shot to its length, start shot 5 at 5.333 s, and reset timestamps to zero with setpts=PTS-STARTPTS.
• scale and pad: We fit each shot to the first shot's probed size with decrease aspect ratio scaling, pad with black bars, and set setsar=1.
• fps=30: Sets every shot to 30 fps and sets format=yuv420p.
• concat: Joins the five video segments with n=5:v=1:a=0.
• -map: We take the joined picture from [v] and the song from the sixth input 5:a:0.
• libx264: Encodes the video stream with -preset veryfast -crf 20.
• AAC: Encodes the soundtrack at 128k.
• -shortest: Ends the output file on the shorter of picture and sound.
• faststart: Passes -movflags +faststart so playback starts before the file's fully downloaded.

We force the output to 30 fps, so every finished file is 571 frames and 19.033 seconds, at either 480×854 or 720×1280. We probe the first shot to set that target size.

The block above is the Wan route at 480p. The Seedance route runs the same graph on its five segment files, each trimmed from 0 s to the same shot lengths.

A join gets 3 tries and 75 seconds per try. Locally, our join finished 5.45 seconds after the last shot downloaded.

This is the file the join makes.

Find the song's 19 seconds by cross-correlation

NPR's live recording replaces the silent model outputs. We always turn the model's own audio off (audio: false on Wan, generate_audio: false on Seedance).

What you'll hear is NPR's live recording from Charlie Wilson's 2023 NPR Tiny Desk concert, posted on YouTube on 2023-06-05. The closing number is "Outstanding", The Gap Band's 1982 song, running at about 98 BPM. We take roughly 23:45.886 to 24:04.928 of their upload (19.042 s), lay it under the joined video from 0 s, and add a 0.3 s fade-out at the end.

We found that span by audio cross-correlation (sliding one soundtrack along the other and scoring how well they line up at each offset) against the AI clip's soundtrack. The match peak stood 10.2 times above the next best, so there's one clear answer, not a shortlist.

# NPR's live recording, 23:45.886 for 19.042 s, 0.3 s fade-out at the end
ffmpeg -i npr-upload.m4a -ss 23:45.886 -t 19.042 -vn \
  -af "afade=t=out:st=18.742:d=0.3" -c:a aac -b:a 128k song.m4a

Check the file before you ship it

# real frame count (571 on a finished file)
ffprobe -v error -count_frames -select_streams v:0 \
  -show_entries stream=nb_read_frames,width,height,r_frame_rate -of compact tiny-desk.mp4
# any freeze of 0.3 s or more prints a freeze_start line
ffmpeg -i tiny-desk.mp4 -vf freezedetect=d=0.3 -map 0:v -f null - 2>&1 | grep freeze_start

You check the output before you ship it. Reading every frame with ffprobe -count_frames confirms the real count: 571 on a finished file.

The freezedetect filter flags any frozen video lasting 0.3 s or more. We found no freeze of 0.3 s or more in any shipped test file across 25 test renders we ran on 26 and 27 October 2026.

Prep the photos, or skip the terminal

Input portrait photos need clean geometry before the video models see them. Our free Tiny Desk AI Photo Prep Space runs the photo steps right in your browser, without storing anything:

• Upright: Saves an upright copy, because a phone photo is sideways pixels plus an EXIF Orientation tag that the model ignores.
• Size: Keeps each side between 300 and 6000 px.
• Shape: Pads very tall or wide photos with black bars instead of cropping.
• One face: Checks there's one clear face in each photo.

If you want to skip the terminal, Seadanse's Tiny Desk AI effect turns 1 to 5 photos into the 19-second 9:16 video with the original "Outstanding" sound in a few minutes. The drawings and join described above are the exact ones the site runs today. A 720p video costs $9.90 (one $9.90 credit pack, no subscription), with no sign-up needed to order.

Follow this checklist on your next multi-shot join

Every multi-shot AI video assembly follows the same timing principles in ffmpeg.

1. Feed line drawings to the model instead of filmed footage so outlines guide the motion without carrying over source faces.
2. Trim every AI generation by seconds rather than frame counts to avoid length drift across different engines.
3. Force one uniform frame rate like 30 fps across all inputs inside the join filter graph.
4. Scale and pad each clip to match the probed dimensions of the first shot.
5. Map NPR's original audio recording under the video timeline and end it with -shortest.
6. Count total frames to verify 571 and run freezedetect to catch any stalls of 0.3 s or more.

Disclosure: we build Seadanse, which sells this Tiny Desk AI effect. The numbers here come from 25 test renders we ran on 26 and 27 October 2026, and every person in our test photos is AI-generated.