跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 9 天前AI 評分58

Marimba:面向語音的 System One 模型

Marimba: A System One Model for Speech

AI 導讀

Marimba 是一個面向語音的 System One 模型,用戶可對即時音訊輸入文字問題,並每 250 ms 收到文字答案。單一凍結的 Qwen3-ASR-0.6B 處理所有問題,可在手提電腦 CPU 上運行;説話者計數在 AMI 和 VoxConverse 上優於 pyannote 3.1,300 段 FLEURS 音訊的語言識別率為 100%。

正文

Marimba is a System One model for speech: ask typed questions about live audio, get typed answers back every 250 ms.

Is someone speaking? Did the turn end? Which language? Did they say "refund"? Did the speaker change? How many speakers?
Answers come back as yes/no, a choice, or a score.

  • One frozen Qwen3-ASR-0.6B serves every question, so it runs on a laptop CPU.
  • Counts speakers better than pyannote 3.1 on AMI and VoxConverse; 100% language ID on 300 FLEURS clips.
  • Open weights, benchmark and code (Apache-2.0).

Latency (p50 per question, 8 s window, batch 1):

Question Mac M4 Pro CPU NVIDIA L4 GPU
Turn end ~100 ms ~12 ms
Voice activity ~105 ms ~14 ms
Language ~1.3 s ~150 ms
Term ("did they say X?") not measured ~1.4 s

Turn end and voice activity are fast on CPU; term detection needs ASR decoding, so a GPU helps there. Protocol: https://github.com/Charcot-AI/marimba-api/blob/main/docs/benchmarks.md

Try it live: https://huggingface.co/spaces/charcot-ai/marimba
Model: https://huggingface.co/charcot-ai/marimba
Benchmark: https://huggingface.co/datasets/charcot-ai/marimba-bench
Code: https://github.com/Charcot-AI/marimba-api

Author here, ask me anything!

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co