Hum to Song 改以瀏覽器分析 Google Lyria 伴奏，不再只依賴提示詞

Hum to Song has one promise: the tune you hummed is really in the finished song. Not "inspired by", not a new melody in the same mood. Your notes.

The pipeline looked simple on paper:

1. Transcribe the hum in the browser (Spotify's open-source Basic Pitch model) into notes, plus tempo, key and a chord for every bar.
2. Ask a text-to-music model (we use Google's Lyria) for a backing band in exactly that key, tempo and chord loop.
3. Play the melody on top.

Step 2 is where it broke. The model doesn't accept audio input, so the only way to steer it is the prompt. On our test sheet ("Twinkle", A major, 96 BPM) real backings showed:

• Tempo: usually exact, but not always (we got 95 and 100).
• Key: followed 1 time out of 3.
• Chords: the loop often started on its own bar. One backing opened its body on the IV chord.
• Intro: we asked for 4 bars and got 2-, 3- and 8-bar intros, often with a drum fill in the last one.

Laying the melody on at bar 5, in A, at 96 BPM produced exactly what you'd expect: a melody fighting the band. So we stopped trusting the prompt and started measuring the audio we got back. Everything below runs in an OfflineAudioContext in the browser, with no samples to download, and gives the same result every time.

1. Tempo: a comb filter near the tempo we asked for

We compute an onset envelope (~10 ms hop), then score candidate tempi by summing the envelope at every beat position, for every phase. The search stays within ±8% of the requested BPM, because a free-range tempo search on pop music happily locks onto half or double time.

function findTempo(f: Features, targetBpm: number) {
  const evalBpm = (bpm: number) => {
    const period = 60 / bpm / f.frameSec;
    let bestPhase = 0, best = -1;
    for (let ph = 0; ph < period; ph += 0.5) {
      let s = 0;
      for (let p = ph; p < f.onset.length; p += period) s += sampleAt(f.onset, p);
      if (s > best) { best = s; bestPhase = ph; }
    }
    // Per beat, so faster tempi don't win just by counting more beats.
    return { bpm, phaseFrames: bestPhase, strength: best / (f.onset.length / period) };
  };
  // ...search target ±8%, then half and double time, which only win when
  // they're clearly stronger (> 1.35×), because the melody should follow
  // the tempo it was written in.
}

Normalising the score per beat was the fix that mattered. Without it, double time wins almost every time.

2. Where does the body start?

The downbeat is the beat (of 4) that carries the kick and the chord changes, so each candidate offset is scored on low-end energy plus how much the chroma changes across it.

Finding the end of the intro was harder. The obvious rule ("the biggest jump in level") failed on a real backing: the drum fill in the last intro bar was the biggest jump, one bar early. A fill is loud but has no low end. So a bar only counts as the start of the body when the level and the kick/bass energy reach what the song holds for most of its length:

if (level(k) >= 0.8 && barLow(k) >= 0.75 * lowRef
    && quietSoFar / k < 0.65 && level(k + 1) >= 0.7) {
  introBars = k;
  break;
}

lowRef and the level reference are medians over bars in the middle of the song, so a loud final chorus doesn't skew them.

3. Key: correlate pitch classes, all 12 ways

Take the melody's pitch classes weighted by note length, rotate them through −6…+5 semitones, and correlate each rotation with the backing's average chroma over the body. The requested key gets a tiny 0.02 bonus as a tie-breaker. The best rotation is the transposition.

This is why the finished song can end up a few semitones away from what you hummed. We'd rather move your tune than have it clash with the band.

4. Chords: one Goertzel filter per semitone

The model often plays its own chords rather than the ones we asked for. To keep the tune off clashes we need to know what the band plays every half bar, and here the analysis FFT let us down: at 11 kHz with 1024-sample frames, a bin is ~11 Hz wide, wider than the gap between neighbouring notes below about A3.

So the harmony map uses one Goertzel filter per semitone from E2 to C6. Goertzel is a single-bin DFT: cheap when you only need ~45 specific frequencies, and it can be tuned exactly to each note.

function goertzel(x: Float32Array, from: number, to: number, freq: number, rate: number) {
  const w = (2 * Math.PI * freq) / rate;
  const coeff = 2 * Math.cos(w);
  let s1 = 0, s2 = 0;
  for (let i = from; i < to; i++) {
    const s0 = x[i] + coeff * s1 - s2;
    s2 = s1; s1 = s0;
  }
  return s1 * s1 + s2 * s2 - coeff * s1 * s2; // power at freq
}

Fold the 45 powers into 12 pitch classes, normalise to max = 1, and you have a chroma vector per half bar.

5. Shift the phrase, then fix only the real clashes

Two last moves:

• Phrase shift: try starting the melody on the bar the band comes in, and up to 3 bars later. Score each by how much of the melody's duration sits on pitch classes the band is playing. Only move if it's at least 5% better; otherwise keep the natural entrance.
• Clash fix: a note moves only when its pitch class is nearly absent from the band (< 0.3) and a note a semitone away rings strongly (≥ 0.55). Then it moves by one semitone toward it. Whole-tone neighbours are left alone, so passing notes survive. Only minor-second rubs get fixed.

On one real backing these steps took the melody's consonance with the band from 0.50 to 0.63. A unit-test script with 60 checks pins the maths so the next tweak doesn't quietly undo it.

What I'd tell anyone building on generative audio

1. Treat the model's output as data to measure, not a contract. The prompt is a request. The audio is the truth.
2. Plain prompts beat clever ones. One plain chord loop ("A | D | D | E, I–IV–IV–V in A major") was followed better than a detailed per-section timeline.
3. Let users hear the cheap step first. Before anything is charged, Hum to Song plays back the transcribed notes on their own: "this is the tune we heard". If the transcription is wrong, nobody pays to find out.

If you want to try it, hum 5–20 seconds into Hum to Song and listen to what it heard. That part is free; the full song needs a paid SongUp AI plan. The non-technical version of this post, with recording tips, is the hum-to-song guide.