SLAM-ASR 的單語 ASR 訓練線性投影器能實現甚麼零樣本口語理解？
Zero-Shot Spoken Language Understanding with SLAM-ASR: What Do Monolingual ASR-Trained Linear Projectors Enable?
研究測試以 11 種語言訓練的 SLAM-ASR 單語投影器能否讓模型執行零樣本口語理解，結果顯示投影器未能有效傳遞相關能力。回答是非問題的平均指令遵循率為 49.8%，要求以單一數字選項作答時則升至 81.3%。主題分類平衡準確率為 23.2%，低於同一解碼器使用參考轉錄文本時的 71.5%。
Rerank 5 is here: sharper search, fewer tokens. Now with Context Focus.
Oct 14, 2026
The SLAM-ASR framework connects the features of a frozen speech encoder as input to a frozen LLM decoder to enable speech transcription. In this work we ask whether this training-on speech transcription-provides the LLM with the capability to also perform spoken language understanding (SLU) tasks, by training monolingual projectors on 11 languages
Authors
Edwin Arkel Rios, Sneh Sourabh, Jocelyn Zaruma, Girish Ewoorkar, Julian Mack, Bo-Cheng Lai
Abstract
The SLAM-ASR framework connects the features of a frozen speech encoder as input to a frozen LLM decoder to enable speech transcription. In this work we ask whether this training-on speech transcription-provides the LLM with the capability to also perform spoken language understanding (SLU) tasks, by training monolingual projectors on 11 languages. When prompted to answer a yes/no question about the input speech, the mean instruction following rate (IFR) is 49.8%. When not following the instruction, these models default back to transcription. The transcription quality-measured by the character error rate (CER)-is not sufficient to predict IFR. Remarkably, when the model is instructed to answer the topic question as a single digit option, the mean IFR rises to 81.3%. Our models score a mean 23.2% topic balanced accuracy (BAcc) against 14.3% chance. However, the same decoder scores a mean 71.5% BAcc when using reference transcripts as text input. This indicates a failure of the projector to transfer information for the SLU tasks.
Related works
來源：Cohere Labs：官方研究博客 · cohere.com