跳到正文
DEV Community· xusheng·· 6 天前AI 評分64

TranscribeLink 如何先在瀏覽器抽取音訊,避免為轉錄上載 5 GB 影片

Uploading a 5 GB video just to transcribe it? Extract the audio in the browser first

AI 導讀

TranscribeLink 在瀏覽器抽取影片音軌,並比較原檔剩餘上載量與音訊檔大小,決定改傳音訊或保留原檔。實作以 Mediabunny 配合 WebCodecs 處理容器與音訊編碼,優先輸出單聲道 16 kHz、32 kbps 的 Opus,約每小時 14 MB,並以 AAC 作回退方案,每小時約 22 MB。

正文

If someone wants a transcript of a 2-hour screen recording, the file on their disk might be 4 GB. But the part a speech-to-text model needs is the sound, and speech models work at 16 kHz mono anyway. At 32 kbps Opus, an hour of mono speech is about 14 MB. Ten hours is about 144 MB.

So for TranscribeLink I stopped asking people to upload the video. The browser pulls the audio track out first and uploads only that. Here's how it works and what I had to get right.

The tools: WebCodecs + Mediabunny

WebCodecs gives the browser access to native audio encoders and decoders (AudioDecoder, AudioEncoder). On its own it doesn't read containers like MP4, MOV, MKV or WebM. For that I use Mediabunny, a small library that demuxes and muxes those formats in the browser and drives WebCodecs for you.

The core of it is a "conversion" that drops the video track and re-encodes the audio:

const input = new Input({ source: new BlobSource(file), formats: ALL_FORMATS });
if (!(await input.getPrimaryAudioTrack())) throw new Error('No audio track');

const output = new Output({ format: new OggOutputFormat(), target: new BufferTarget() });
const conversion = await Conversion.init({
  input, output,
  video: { discard: true },
  audio: { codec: 'opus', numberOfChannels: 1, sampleRate: 16000, bitrate: 32000, forceTranscode: true },
});
conversion.onProgress = (p) => showProgress(p);
await conversion.execute();

const audio = new File([output.target.buffer], 'audio.ogg', { type: 'audio/ogg' });

BlobSource reads the file in slices, so a 5 GB video never has to fit in memory.

Fallbacks matter more than the happy path

Not every browser can encode Opus. I try two targets in order and check each one first with canEncodeAudio:

  1. Opus, mono, 16 kHz, 32 kbps, in an Ogg file (about 14 MB an hour)
  2. AAC, mono, 44.1 kHz, 48 kbps, in an MP4 file (about 22 MB an hour)

If the browser has no AudioEncoder at all (older Safari), or neither codec works, nothing breaks: it just uploads the original file. Extraction is an optimisation, never a requirement.

Don't wait for extraction: race it

Extracting audio from a long file takes time, and on a fast connection the original might upload before extraction finishes. So both start at once:

  • The original file starts uploading in chunks right away.
  • Extraction runs in parallel.
  • When extraction finishes, compare: if the bytes still left to upload are more than the size of the extracted audio, stop the upload, tell the server to replace it, and send the audio instead. Otherwise cancel extraction and let the original finish.
const first = await Promise.race([
  sending.then((result) => ({ sent: result })),
  extraction.promise.then((audio) => ({ audio }), (error) => ({ audio: null, error })),
]);

if (first.audio && file.size * (1 - uploadedFraction) > first.audio.size) {
  stopUpload();
  await replaceOnServer(first.audio.size);
  await upload(first.audio);
}

On a slow connection the audio almost always wins, and that's exactly where it saves the most time.

Small details that caught me

  • Resumed uploads keep the original. I resume interrupted uploads by byte offset. Extracted audio isn't byte-identical between runs, so a resumed upload continues with the original file instead of re-extracting.
  • Skip tiny files. Below about 1 MB, extraction isn't worth the overhead, so those go up as they are.
  • Load the library early. Mediabunny is about 690 KB minified. I start loading it when the user opens the file picker, so it's ready by the time they've chosen a file.
  • Use XMLHttpRequest for chunks. fetch doesn't report upload progress, so a big chunk shows 0% until it's done. xhr.upload.onprogress lets the progress bar move smoothly, which matters when people are watching a multi-GB upload.

Result

A long video becomes an audio file of tens of megabytes before it leaves the user's machine. The upload that used to be the slowest step is now usually the shortest, and the server never has to store or process the video.

I use this in TranscribeLink, where you can upload files up to 10 hours or 5 GB, or paste a video link, and get a transcript in 100 languages. If you've done something similar, I'd like to hear how you handled browsers without WebCodecs.

來源:DEV Community · dev.to