跳到正文
DEV Community· x3-team·· 2 小時前AI 評分60

Nibbo 作者如何構建 60 多款不會上傳檔案的工具

How I built 60+ file tools that never upload your files

AI 導讀

Nibbo 開發者 Vitalii 説明如何以瀏覽器端處理打造 67 款檔案工具,令檔案毋須上傳至伺服器。以 1 GB 1080p VP9 測試檔為例,串流解封裝令分頁記憶體峯值由約 3.0 GB 降至約 1.9 GB。文章亦介紹離線快取設計,以及透過 DevTools Network 檢查是否存在大型檔案上傳請求的方法。

正文

Disclosure up front: I'm Vitalii, and I built Nibbo, the project this post is about. I'm a solo developer. What follows is how it works, including the parts that still don't work well.

Most "free online converters" follow the same pattern: you pick a file, it gets uploaded to their server, a job runs there, and you download the result. That's fine for a meme GIF. It's less fine for a passport scan, a signed contract, or a private video.

So I set myself one rule: the site has no server that could receive a file. Not "we delete it after an hour", but no endpoint for it at all. Today that rule covers 67 tools: video compression to an exact size, photo compression to a target KB, PDF merge/split/compress/sign, background removal, subtitles from speech, HEIC→JPG, MP4→MP3 and so on.

This post covers the architecture, the memory problems you hit with 3 GB inputs, offline support, and how you can check the "no upload" claim yourself in about 30 seconds.

The stack: a static site with heavy islands

  • Astro builds a fully static site, with Preact islands for the interactive tools.
  • Cloudflare Pages hosts it. The only server-side code is two small Pages Functions: a language redirect on / and an endpoint that returns the visitor's country (for a flag in the UI). Neither one accepts a request body.
  • All processing happens in the browser, using:
    • WebCodecs + mp4box.js (demux) + mp4-muxer for hardware-accelerated video
    • ffmpeg.wasm (single-threaded core) as the fallback for containers WebCodecs can't handle
    • jSquash (AVIF/WebP/JPEG XL) and libheif-js (HEIC) for images
    • pdf-lib, qpdf compiled to wasm, and pdf.js for PDFs
    • tesseract.js for OCR
    • ONNX Runtime Web / transformers.js for on-device models (Whisper for subtitles, BiRefNet-lite and MODNet for background removal), running on WebGPU with a WebAssembly fallback

One constraint shapes everything else. Each of those libraries weighs megabytes, and plenty of my visitors are on slow, expensive mobile data. So the build fails if any page's first load (HTML + CSS + statically imported JS, gzipped) goes over 200 KB. Every wasm module sits behind a dynamic import() and is only downloaded when you actually start a tool that needs it.

WebCodecs first, ffmpeg.wasm second

ffmpeg.wasm is the obvious way to do video in a browser, and it's great. It's also slow for encoding: x264 on a single wasm thread is no match for the hardware encoder your GPU already has. So video compression uses a hybrid engine:

  1. Probe. MP4/MOV/M4V/3GP (anything ISO-BMFF) is demuxed with mp4box.js. Everything else (MKV, AVI, WMV, FLV, TS…) goes to the fallback.
  2. Encode with VideoEncoder (H.264, hardware where available). Some Chromium builds ship without an H.264 encoder, so those get VP9-in-MP4 instead, which is still far faster than software x264.
  3. Fall back to x264 in ffmpeg.wasm only if WebCodecs is missing, can't decode the input, or fails. The ~32 MB FFmpeg core is downloaded only in that case.

I deliberately use the single-threaded FFmpeg core. The multi-threaded one needs SharedArrayBuffer, which means cross-origin isolation (COOP/COEP headers), and that breaks a lot of ordinary embedding and third-party behavior across the whole site. Being slower on rare formats is the better trade.

Hitting an exact size

"Compress to under 20 MB for Discord" is a bitrate budget problem:

// target bytes → total bitrate, keeping ~6% headroom for encoder overshoot
export function totalBitrateBudget(targetBytes: number, durationSec: number, overheadBytes: number, safety = 0.94) {
  const payload = Math.max(0, targetBytes * safety - overheadBytes);
  return Math.floor((payload * 8) / durationSec);
}

Audio gets a small tiered slice, and the rest goes to video. Then the planner walks a resolution ladder (2160 → 240) and picks the highest rung where bits-per-pixel stays above a minimum that depends on motion. Motion is estimated by measuring spatial and temporal complexity (SI/TI) on a few sampled GOPs. It's a cheap stand-in for a proper quality metric, but it beats shipping a 1080p blur fest. If the output still overshoots, the bitrate is scaled by target / actual × 0.97 and the job runs again, up to 3 attempts. If the target can't work at all, the tool says so up front and tells you the minimum size that would.

Images use the same idea with a cheaper loop: binary search on encoder quality, and only if the quality floor still doesn't fit, shrink the pixel dimensions and search again.

The 3 GB problem: never hold the file

The free limit is 3 GB per file. The first naive version did await file.arrayBuffer(), and that's how you crash a tab with a big video.

What helped, roughly in order of impact:

1. Read only the index. In most MP4s the moov box (the sample table) sits after the giant mdat payload. mp4box.js can be fed slices and tells you which offset it needs next, so it jumps over mdat entirely. Opening a multi-GB file reads only box headers and moov.

2. Stream payloads in windows. Sample data is read on demand, grouping nearby samples into one 8 MB Blob.slice() read:

export async function* readSamples(file: Blob, samples: readonly Sample[], windowBytes = 8 * 1024 * 1024) {
  let winStart = 0;
  let win: Uint8Array | undefined;
  for (const s of samples) {
    const start = s.offset, end = start + s.size;
    if (!win || start < winStart || end > winStart + win.byteLength) {
      winStart = start;
      win = new Uint8Array(await file.slice(start, start + Math.max(windowBytes, s.size)).arrayBuffer());
    }
    yield { sample: s, data: win.subarray(start - winStart, end - winStart) };
  }
}

3. Backpressure on the codecs. VideoDecoder and VideoEncoder happily accept thousands of queued chunks, which turns into thousands of decoded frames in GPU/CPU memory. I wait until decodeQueueSize / encodeQueueSize drops below 24 before feeding more.

The non-obvious part: my first version polled with setTimeout. In a hidden tab Chrome throttles timers to once per second (and much more after a few minutes), so compression slowed to a crawl whenever the user switched tabs. Listening for the codec's dequeue event fixed it. The timer stays only as a safety net.

4. Don't copy into the wasm heap. For the FFmpeg path, ff.writeFile() would copy the whole input into wasm memory, which is 32-bit (4 GiB at most) and shared with everything else FFmpeg needs. Mounting the File read-only via WORKERFS lets FFmpeg read it in place:

await ff.createDir('/input').catch(() => {});
await ff.mount('WORKERFS', { files: [file] }, '/input');
try {
  await ff.exec(['-i', `/input/${file.name}`, /* … */ 'output.mp4']);
} finally {
  await ff.unmount('/input').catch(() => {});
}

With the streaming demux, my notes show a 1 GB 1080p VP9 test file going from about 3.0 GB peak tab memory to about 1.9 GB. The honest limits right now: the encoded output is still assembled in memory (about the size of the target, so it's fine for a "make it 25 MB" job), and fragmented MP4s are still loaded whole.

Offline: a boring service worker, on purpose

Once you've opened a page, it should work with Wi-Fi off. The service worker uses three plain strategies:

  • Navigations: network-first, falling back to cache, so deploys show up right away when you're online.
  • Hashed build assets and vendored wasm (/_astro/, /vendor/): cache-first. They're immutable by URL.
  • Everything else from the site: stale-while-revalidate.

Two things made this harder than the tutorials suggest:

  • Cloudflare Pages has a 25 MiB per-file limit. The FFmpeg wasm (~32 MB) and ONNX Runtime don't fit, so they come from jsDelivr at a pinned version, and the browser refuses them unless the SHA-256 matches the copy from npm baked in at build time. Model weights come from Hugging Face at pinned commits, are checked the same way, and are then kept in Cache Storage, so the second run (and offline runs) skip the network.
  • The service worker only handles same-origin requests. Third-party files rely on the cache I manage myself (models) or on the browser's HTTP cache (the FFmpeg core). Pre-warming those is still on my roadmap. Caveat noted.

The PWA manifest also registers file_handlers and a share_target. Files shared from your phone's share sheet go into the site's own IndexedDB on the device, not to a server.

Don't trust me, check it

"We don't upload your files" is a claim. Here's how to test it on any site, including mine:

  1. Open DevTools → Network, tick Preserve log, and clear the list.
  2. Drop in a large file and run the tool.
  3. Look for requests with a request body about as big as your file (POST/PUT, or chunked uploads to /upload, /api/…, S3, etc.). Sorting by size and checking the Payload tab makes this quick.

On Nibbo you'll see GET requests for code and, the first time, for wasm or model weights. If you accepted the cookie banner, you'll also see analytics hits (GA4 / Yandex Metrica), which carry events like "tool started, mp4, ~120 MB". They never include file names or contents. If you declined, those aren't loaded at all.

  1. Even simpler: load the tool page, turn on airplane mode, then process a file. If it works, nothing was sent.

On top of that, there's a Content-Security-Policy with an explicit connect-src allowlist (the site itself, jsDelivr, Hugging Face, and the two analytics hosts). Even a buggy or compromised script on the page can't open a connection to an arbitrary server. The browser enforces it, so it isn't just a promise.

Lessons

  • Feature-detect and fall back in layers. WebCodecs H.264 → WebCodecs VP9 → ffmpeg.wasm covers the browsers I've tested. Each layer needs its own error path, because "supported" in isConfigSupported doesn't always mean it works on a given file.
  • Memory is the real limit, not CPU. Design for Blob.slice() from day one. Retrofitting streaming onto code that assumed an ArrayBuffer was the most painful refactor I did.
  • Hidden tabs are hostile. Anything long-running that depends on timers will stall. Use events.
  • Single-threaded wasm is a valid choice if threads would force cross-origin isolation on your whole site.
  • Put a page-weight budget in CI early. Seven wasm-heavy libraries will creep into your landing page bundle otherwise.
  • Pin and hash everything you load from a CDN. It's a few lines of crypto.subtle.digest and it closes a whole class of supply-chain problems.

Try it, break it

Nibbo is free, needs no account, adds no watermark, and is available in 16 languages. The code isn't public yet. If you want to poke at it, nibbo.org is the place. I'm most interested in files that break it and in how fast the video encoder runs on your hardware. Comments here work, and so do bug reports with a sample file.

來源:DEV Community · dev.to