音樂分軌如何拆分歌曲：原理與 Python 示例

By Tania Alwan

Stem separation, also called music source separation, takes a finished stereo mix and estimates the individual parts that went into it: vocals, drums, bass and everything else. It powers karaoke apps, remix tools and AI cover workflows. It's also a nice example of an "impossible" inverse problem that deep learning made practical.

This post covers how separation works, how quality is measured, and how to run an open-source model yourself in a few lines of Python.

The problem: un-mixing a sum

A stereo mix is, roughly, a sum of sources:

mix(t) = vocals(t) + drums(t) + bass(t) + other(t)

Given only mix, we want to estimate each term. Mathematically that's underdetermined: infinitely many combinations sum to the same signal. Models solve it by learning what each kind of source usually sounds like from large sets of multitrack recordings.

Approach 1: spectrogram masking

The classic deep-learning approach works in the time-frequency domain.

1. Compute a spectrogram with a short-time Fourier transform (STFT).
2. Predict a mask per source: a value between 0 and 1 for every time-frequency bin, saying how much of that bin belongs to the source.
3. Multiply the mix spectrogram by each mask.
4. Invert back to audio with the inverse STFT, usually reusing the mix's phase.

import numpy as np
import librosa

y, sr = librosa.load("mix.wav", sr=44100, mono=True)
S = librosa.stft(y, n_fft=4096, hop_length=1024)
mag, phase = np.abs(S), np.angle(S)

# a trained model would predict this; here it's a placeholder
vocal_mask = model.predict(mag)          # shape == mag.shape, values in [0, 1]

vocals = librosa.istft(vocal_mask * mag * np.exp(1j * phase), hop_length=1024)

Weakness: reusing the mixture phase limits quality. When two sources overlap in the same bin, a single mask can't fully separate them, which is where "watery" artefacts come from.

Approach 2: waveform and hybrid models

Waveform models like the original Demucs operate directly on raw audio with a convolutional encoder-decoder, sidestepping the phase problem. Hybrid models combine a spectrogram branch and a waveform branch. Hybrid Transformer Demucs (HT Demucs), released by Meta's research team, adds transformer layers between the two branches to model longer-range context.

Other strong families include band-split architectures, which process different frequency bands with dedicated sub-networks. The open research community iterates on these quickly, often through public challenges.

How quality is measured

The standard metric is SDR (signal-to-distortion ratio), in decibels; higher is better. Benchmarks typically use the MUSDB18 dataset, which contains multitrack songs split into vocals, drums, bass and other.

Keep two caveats in mind:

• SDR doesn't fully match perception. Two models with similar SDR can sound different.
• Benchmarks use four stems. Many products offer more stems (guitar, piano), which aren't covered by the classic benchmark.

Always listen, not just read numbers.

Running Demucs locally

Demucs is open source and installable with pip.

python -m pip install -U demucs
demucs song.mp3                       # default model, 4 stems
demucs --two-stems=vocals song.mp3    # vocals + no_vocals only

Outputs are written to a separated/ folder, organised by model name and track. Check demucs --help for the model options available in your installed version.

Practical notes:

• A GPU helps a lot. CPU-only works, but a full song can take a while.
• Long files use more memory; the CLI has options for splitting into segments.
• Input quality matters. Separation can't recover detail lost to heavy compression.

Batch processing a folder

import subprocess, pathlib

for f in pathlib.Path("tracks").glob("*.wav"):
    subprocess.run(["demucs", "--two-stems=vocals", str(f)], check=True)

Evaluating your own outputs

If you're comparing models for a project, a simple, repeatable test beats impressions.

• Pick 5 to 10 reference tracks across genres: a sparse acoustic song, a dense rock mix, an electronic track with heavy sidechain, a rap track.
• Process all of them with each model using identical settings.
• Listen blind if you can: rename files so you don't know which model produced which.
• Check the residual: sum the stems and subtract the original mix. A large residual means energy went missing.
• Log notes per track, not just a single overall score.

import soundfile as sf, numpy as np
mix, _ = sf.read("song.wav")
stems = [sf.read(f"separated/htdemucs/song/{s}.wav")[0] for s in ("vocals","drums","bass","other")]
residual = mix - np.sum(stems, axis=0)
print("residual RMS:", np.sqrt(np.mean(residual**2)))

and why they happen

• Bleed: hi-hats in the vocal stem. High-frequency transients overlap with sibilants.
• Phasiness: a swirly sound on sustained notes, typical of masking with mixture phase.
• Missing reverb tails: models tend to assign reverb ambiguously, so tails can end up split between stems.
• Backing vocals: harmonies may land in "vocals" or "other" depending on the model's training data.

Choosing between local and hosted separation

Run it yourself when:

• You want zero cost and full control.
• You're batch-processing many files.
• You need to keep audio on your own machine.

Use a hosted service when:

• You don't have a GPU or don't want to manage environments.
• You need more than four stems out of the box.
• You want downstream tools in the same place.

On the hosted side there are options like LALAL.AI, Moises and the Voicestars stems splitter, which outputs six stems (vocals, drums, bass, guitars, piano, other). Pricing models differ (credits, subscriptions, per-minute rates), so compare against your expected volume.

Where stem separation is going

• More stems: separating individual instruments beyond the classic four.
• Real-time separation: lower-latency models for live use and DJ software.
• Separation as a preprocessing step: many voice-conversion and transcription pipelines now start by isolating vocals first. I wrote about that pipeline in Vocal Isolation and Voice Conversion.

Try it

Take a song you own, run demucs --two-stems=vocals on it, and listen to the outputs on headphones. Then compare against a hosted tool to see which artefacts each one leaves. If you want a browser comparison point, the Voicestars stems splitter is one option.

Disclosure: this article was written in partnership with Voicestars.