如何建立可處理大型音訊及影片檔案的遠端 MCP 伺服器

Posted on Nov 1 Edited on Nov 1

The rise of AI agents (like Cursor and Claude Desktop) is incredible. But if you've ever tried to build a custom tool for them to process large files—like a 50MB audio track or a video—you already know the pain.

LLMs have strict token and payload limits. You can't just pass a local .mp3 file into a prompt.

In this post, I'll share the architectural pattern I used to build a Remote MCP (Model Context Protocol) Server that allows an AI agent to process heavy media files entirely hands-free.

The Core Problem: File Size & Context Switching

When you want an AI to process a file, the naive approach is to base64 encode it and send it. This blows up the context window instantly.

Alternatively, you can build a local MCP server that reads the file directly. But what if the processing requires heavy GPU compute that your local laptop doesn't have?

The Solution: The Async Upload Pattern

To solve this, I designed an MCP server that uses an asynchronous pipeline with presigned URLs.

Here is the exact flow the AI agent follows:

get_upload_url: The agent calls this tool. The server returns a presigned URL (e.g., an S3/R2 URL).
Local Upload: The agent (or a local script it triggers) uploads the file directly to cloud storage. No file data ever touches the LLM context window.
submit_task: The agent tells the server, "Hey, process this uploaded file."
check_task_status: The agent polls the server (or uses webhooks) to check when the GPU processing is done.
get_download_url: Finally, the agent gets a URL to download the processed stems, MIDI, or denoised audio back to the local workspace.

Why This Rocks for Agentic Workflows

This pattern completely decouples the AI's reasoning from the heavy lifting.

Here is what a prompt looks like in Cursor:

"Extract the vocals from ./demo.mp3 and save the result locally."

The AI handles the file system, calls the tools, waits for the GPU, and drops the finished file right back into your project folder. It feels like magic.

Implementation Tips

Use Streamable HTTP: If you are building for remote clients like Claude or Cursor, stdio isn't enough. Host your server and expose it via HTTP.
Handle Polling Gracefully: LLMs can get stuck in loops if a task fails. Make sure your check_task_status returns clear error states, not just "pending".
Authentication: Protect your server with OAuth. Exposing a GPU-backed API without auth is a quick way to get a massive bill.

Open Source Example

I built this entire architecture for Sonicaura MCP (an audio processing engine). It's completely open-source and published in the official MCP Registry if you want to dig into the code or try it out.

Repo: warmcoding/sonicaura-mcp
Registry: Official MCP Registry

If you are building agents, I'd love to know: How are you handling large file processing in your MCP servers? Let me know in the comments!