YouTube 自動生成字幕是否準確到可以直接引用？

Most YouTube videos have a transcript, and for most of them it was written by speech recognition. That's fine for following along. For a quote in a paper, a news story or a legal note, the question is how far the automatic text is from what was said, and whether its timestamps point at the right second. One video with both kinds of captions gives a direct answer.

The video is NASA's upload of President Kennedy's 1962 speech at Rice University (WZyRbnpGyzQ, 18 minutes). It carries two English tracks: one written by a person and one generated by YouTube. Both can be listed and downloaded with the youtube-transcript-api Python package:

from youtube_transcript_api import YouTubeTranscriptApi

tracks = YouTubeTranscriptApi().list("WZyRbnpGyzQ")
for t in tracks:
    print(t.language_code, t.is_generated, t.language)
# en False English - Caption
# en True English (auto-generated)

manual = tracks.find_manually_created_transcript(["en"]).fetch().to_raw_data()
auto = tracks.find_generated_transcript(["en"]).fetch().to_raw_data()

Each entry is a line of text with a start time and a duration in seconds.

How far apart the two tracks are

Compared word by word (lowercased, punctuation stripped, aligned with Python's difflib.SequenceMatcher), the two tracks agree on about 95% of the speech:

Six percent sounds small until you look at which words they are. Speech recognition does well on common words and badly on names, numbers and anything technical, which are exactly the parts a quote tends to hinge on:

There is also a hole. Around 4:47 the automatic track jumps from "both must be" straight to "determined and cannot be deterred". The 28 words in between, "enterprised and overcome with answerable courage. If this capsule history of our progress teaches us anything, it is that man, in his quest for knowledge and progress, is", aren't in it at all. Nothing in the automatic track marks the gap.

What happens to the famous line

The line people quote from this speech sits at 9:18. In the human track it's one line, punctuated, with one start time:

558.90  3.65  not because they are easy, but because they are hard,

In the automatic track it's spread across three overlapping lines, and the first word belongs to the line before:

555.64  4.80  decade and do the other things not
558.20  6.52  because they are easy but because they
560.44  7.04  are hard because that goal will serve to

Automatic captions are built for rolling display, a few words at a time, so their lines follow the timing of speech rather than sentences. Two things follow for quoting. Copying the line at 9:18 gives a fragment that starts mid-sentence, and the start time of the line that contains "not" is 9:15, three seconds before the quote. Neither is wrong as a caption; both are wrong as a citation.

The grammar survives in this case, "they are", unlike the version most people remember ("because it is hard"). But "we are willing" became "we're willing" a few lines later, and a quote taken from the automatic track would put a contraction in Kennedy's mouth that he didn't say.

Picking the track, and the link

The transcript tool in the osFoundry desktop app, which fetches captions from the computer it runs on, makes the choice in a few lines of Rust. YouTube marks automatic tracks with kind: "asr", and the app takes a human English track first, then any English track, then any human track, then whatever is there:

fn pick_track(tracks: &[CaptionTrack]) -> Option<&CaptionTrack> {
    tracks
        .iter()
        .find(|t| t.language_code.starts_with("en") && t.kind != "asr")
        .or_else(|| tracks.iter().find(|t| t.language_code.starts_with("en")))
        .or_else(|| tracks.iter().find(|t| t.kind != "asr"))
        .or_else(|| tracks.first())
}

When a line is saved as a reference, its start time becomes the t parameter of the link. The time is rounded down, so the video starts a fraction of a second before the words rather than just after the first one:

url: `https://www.youtube.com/watch?v=${transcript.video_id}&t=${Math.floor(segment.start)}s`

For the 9:18 line that's &t=558s, for a line that starts at 558.9 seconds.

When there is only an automatic track

Plenty of videos have nothing but the automatic track. Then it's a search index, not a source: use it to find roughly where something was said, play that stretch, and write the quote down from the audio. Check names and numbers against another source, and expect missing stretches with no marker. The link to the second is still worth keeping, because it lets a reader do the same check in a few seconds.

The desktop app used here is free from the osFoundry download page.