Vapi 上線流量拆分功能，支援按比例測試語音 Agent 新版本

If you’ve been building on voice agents for a while, there’s probably a list of changes you would like to make. This could include the bloated prompt you’d love to cut down, the model swap you’ve been curious about, or the voice you think would land better with customers but haven’t tried.

Most of that list came down to one thing: making a change meant routing your entire customer base to something new, all at once. Until now, publishing a new version of an assistant sent 100% of your traffic to it, immediately. So teams got careful, then slow, and eventually they stopped iterating once things were good enough, leaving behind the changes that could have taken good enough to great.

Traffic splitting changes how you iterate on your agents. Send 0.1% of calls to a new version, or 1%, or 5%, watch what happens, and roll forward or back with confidence.

We sat down with Peter Gomez, a Member of the Technical Staff at Vapi, who helped build it. Below is the conversation on what it is, what was hard, and where it goes next.

In plain terms, what is traffic splitting, and why does it matter?

Traffic splitting lets you divide inbound and outbound call traffic across different versions of your agents.

You could already create versions of an assistant. The problem was that as soon as you published a new one, all your traffic moved to it at once. Say you change a voice or a prompt. You had no reliable way to confirm the change was doing what you wanted with live customers, or whether it was outperforming the previous version. That matters most with LLMs, where the outcome of a change isn’t deterministic. Adding a single keyword to a prompt can have downstream effects you never planned for.

We felt this ourselves while iterating on our own assistants. Teams slow down their publishing and iterate less and less. Once an assistant gets good enough, they stop touching it, even when they know there’s a problem. The config becomes fragile, and the risk of a small update outweighs the reward.

With this release, you send a small slice of traffic to a new version and watch it before you commit. You confirm it meets your standards by looking at the metrics, then roll it out to 100% of callers. It’s essentially a canary for your assistant. Sims and evals get you to maybe 90% confidence before you go live. Traffic splitting gets you the rest of the way with real calls before the whole customer base lands on it.

How were teams working around it before?

People got creative. Some run duplicate assistants and route between them. Some version through our GitOps repository, which is still a good practice. Others built routing logic on their side using the assistant request as an escape hatch.

The honest take is that most of those solutions are more complex and less reliable than what’s now built into the platform.

What was the hardest problem to get right, and how did you solve it?

Two things.

First, changing the code safely. It lives in the hot path, so every call that comes in touches it, whether or not it’s using the feature. We put everything behind feature flags, did extensive internal testing before it reached anyone else, and made sure it was easy to roll back.

The second was identity - to keep repeat callers on the same version, a concept we call "stickiness". Everyone receives and routes calls differently. The real technical challenge was identifying a caller across phone calls, web calls, inbound, and outbound, so when someone calls back, they get the same version wherever possible instead of a different one, creating a consistent experience. Identity can be toggled on and off for the deployments where this is a requirement.

What use cases have you seen customers run with traffic splitting?

The main one is safe rollout. Make a new version, send a small slice of traffic to it, watch, then roll it out to everyone.

The more interesting thing we found looking at customer data is that splits are temporary. There’s a moment when you have full confidence in a new version. Rather than dialing it to 100%, people set the assistant back to “follow latest,” meaning new versions go straight to all traffic again. Split, confirm, flip back, repeat. Short cycles, one variable at a time.

That tells me customers know the risk level of each change. Swapping the LLM gets a split. A small prompt tweak goes to 100%, and they move on with their day, then check in a few hours later to confirm it’s working correctly.

What best practices should teams follow when they start splitting traffic?

Start with two-way splits. The feature supports more, but we recommend two at a time so you know what’s driving the difference. We try to teach this in the UI as well.

Also, in many cases, you can keep experiments short. Every call is pinned to its version, so you can attribute results cleanly without a long testing window. Some changes will need a week or more to read with lower volume or spiky traffic, but many reach statistical significance much faster.

And before you start, look at how you’ve been building. If you’ve got duplicate assistants or custom routing that only exists to avoid the all-or-nothing publish, that’s a candidate to simplify.

What did early users do with it that you didn’t expect?

I expected people to treat it as a long-running experiment tool. Instead, they treat it as a gate.

The bigger surprise was how much hesitation to adopt came from people already engineering around the problem. Replacing a working, if clunky, in-house solution takes a deliberate decision.

Where does this go next? And what does it unlock for the platform more broadly?

Right now it unlocks the safe rollouts we want every builder to have. Building should be fun. You should be able to see something new, like GPT-Live, want to try it, and try it without breaking production.

Longer term, this is the primitive for outcome-based features. Once calls are pinned to versions, automatic rollouts become feasible alongside statistical experiments. If you have a structured output that measures your deflection rate, you could tell Vapi to keep a version only if deflection holds, and roll back if it drops. While we are evaluating the future of this, there are promising signs that this will be the foundation for much more automated experimentation and automation down the line.

If you’re building on Vapi, take this as a moment to evaluate your agents, pull up that list of changes you’ve been sitting on, and send a slice of traffic their way. To learn how to set up your first split, head to the docs.