跳到正文
LiveKit 產品與工程博客·· 1 天前AI 評分60

LiveKit Turn Detector v1.1 加快回合結束偵測,並推出 CPU 版 v1.1-mini

LiveKit Turn Detector v1.1: end-of-turn detection that reads and hears

AI 導讀

LiveKit 發佈 Turn Detector v1.1,在相同誤打斷率下,偵測延遲較 v1 降低 15–34%,並提供可在 CPU 執行的開放權重版本 v1.1-mini。

正文

LiveKit Turn Detector v1.1 detects the end of a user's turn 15–34% sooner than v1, at the same false-interruption rate. It ships with v1.1-mini, an open-weight version that runs on CPU.

Like v1, v1.1 listens to the audio of the user's turn. It also reads the agent's last message as text. v1.1-mini decides from audio alone. Both models improve on two cases that are hard for end-of-turn detection: short replies and non-native English speakers.

Short turnsLink to this section

When an agent asks a direct question, users often answer in a few words, such as a number, a name, or a short "uh-huh". But the same words can start a much longer answer to other questions:

  • Agent: "How many guests?"

  • User: "Seven."

  • Agent: "What is your phone number?"

  • User: "Seven… [pause]"

To tell the two apart, v1.1 also reads the agent's last message.

Non-native English speakersLink to this section

Non-native speakers often pause in the middle of a sentence to find a word. A model trained mostly on native speech can take that pause for the end of the turn, and the agent cuts in too early. Both models are trained on additional non-native English speech from many accents, so they wait through these pauses.

Benchmark resultsLink to this section

To measure progress on these two cases, we're adding two English evaluation sets to eot-bench: 1,088 short user turns and 800 turns from non-native English speakers. They join the original 400-turn English set and up to 400 turns in each of the 13 other languages. Use them to test any end-of-turn model on these cases, with the same harness behind the results below.

Latency is how long the model waits after the user stops speaking before it ends the turn. The cut-off budget caps false interruptions, where the agent cuts in before the user is done, at 5% or 10%. Lower latency is better.

EnglishLink to this section

Cut-off budget 5%

Eval setv1.1v1v1.1-miniv1-mini
Original set

400 turns

462 ms

−15%

543 ms812 ms

−24%

1,070 ms
Short turns

1,088 turns

1,096 ms

−23%

1,427 ms1,237 ms

−24%

1,634 ms
Non-native speakers

800 turns

700 ms

−19%

867 ms999 ms

−19%

1,228 ms
All

2,288 turns

810 ms

−19%

996 ms1,124 ms

−17%

1,352 ms

Percentages compare each model with its predecessor.

Cut-off budget 10%

Eval setv1.1v1v1.1-miniv1-mini
Original set

400 turns

250 ms

−15%

295 ms504 ms

−28%

698 ms
Short turns

1,088 turns

635 ms

−32%

932 ms893 ms

−19%

1,100 ms
Non-native speakers

800 turns

360 ms

−31%

523 ms676 ms

−24%

884 ms
All

2,288 turns

414 ms

−34%

632 ms738 ms

−20%

922 ms

Percentages compare each model with its predecessor.

MultilingualLink to this section

Cut-off budgetv1.1v1v1.1-miniv1-mini
≤5%521 ms

−17%

631 ms715 ms

−18%

867 ms
≤10%329 ms

−19%

406 ms492 ms

−22%

629 ms

The mean of 13 languages, with up to 400 turns each. Percentages compare each model with its predecessor.

At the same cut-off budget, v1.1 cuts latency by 15–34% on English and by 17–19% across the 13 other languages, compared with v1. v1.1-mini cuts latency by 17–28% on every set, compared with v1-mini.

The eot-bench leaderboard ranks models on each set. v1.1 ranks first on every English set at both 5% and 10% false-interruption rates. Most end-of-turn models don't cover all 13 other languages, so the leaderboard also has an interactive heatmap of per-language results. v1.1 has the lowest latency across all languages at both rates.

AvailabilityLink to this section

LiveKit Turn Detector v1.1 is free for agents hosted on LiveKit Cloud, and self-hosted agents can use it through LiveKit Inference. If you already use the LiveKit turn detector through LiveKit Inference, you get v1.1 with no code change, and it decides from audio alone. Upgrade to Agents 1.8.12 (Python) or 1.10.0 (Node.js) so your agent also sends its last message and the user's turn length, which v1.1 uses for short replies.

v1.1-mini runs on CPU, decides from audio alone, and covers the same 14 languages as v1.1. It ships with open weights in the Agents SDKs from the same versions, under the same LiveKit Model License as v1-mini.

Get startedLink to this section

You can use v1.1 now:

Questions, benchmarks, or pull requests? Join us in the LiveKit community.

來源:LiveKit 產品與工程博客 · livekit.com