ElevenLabs 與 Twilio：構建語音通話應用程式

Why combine ElevenLabs with Twilio?

If you’ve ever wanted to give your users a truly human‑like voice experience—whether it’s a customer‑support bot, an interactive story, or a language‑learning assistant—two services together can do the heavy lifting for you:

When you pair ElevenLabs’ high‑quality audio generation with Twilio’s call routing, you can spin up voice‑first applications in minutes instead of weeks. Below is a practical walkthrough that shows how to generate speech on the fly and pipe it straight into a phone call.

Prerequisites

• ElevenLabs API key – sign up and grab your key from the dashboard. You’ll also need the affiliate link for this guide: https://try.elevenlabs.io/kr07zfuqn1bp.
• Twilio account – get a trial number and note your ACCOUNT_SID, AUTH_TOKEN, and a phone number you’ll use as the caller ID.
• Python 3.9+ (or Node.js if you prefer). The example uses Flask, requests, and twilio libraries.
• ngrok (or any public HTTPS tunnel) so Twilio can reach your local server.

# Python dependencies
pip install flask requests twilio

1️⃣ Generating Speech with ElevenLabs

ElevenLabs exposes a simple REST endpoint: POST /v1/text-to-speech/{voice_id}. The request body contains the text you want spoken, and the response is an MP3 (or WAV) audio stream.

import requests

ELEVEN_API_KEY = "YOUR_ELEVENLABS_API_KEY"
VOICE_ID = "EXAMPLE_VOICE_ID"  # you can list your voices via the API

def generate_speech(text: str) -> bytes:
    url = f"https://api.elevenlabs.io/v1/text-to-speech/{VOICE_ID}"
    headers = {
        "xi-api-key": ELEVEN_API_KEY,
        "Content-Type": "application/json"
    }
    payload = {
        "text": text,
        "model_id": "eleven_monolingual_v1",  # default high‑quality model
        "voice_settings": {
            "stability": 0.75,
            "similarity_boost": 0.85
        }
    }
    response = requests.post(url, json=payload, headers=headers)
    response.raise_for_status()
    return response.content  # binary MP3 data

Pro tip: Use ElevenLabs’ voice cloning feature to create a custom voice that matches your brand. The process is documented in their UI, and once you have a voice_id you can plug it straight into the code above. Check out the official docs or sign up via the affiliate link: https://try.elevenlabs.io/kr07zfuqn1bp.

2️⃣ Setting Up a Twilio Voice Webhook

Twilio expects a URL that returns TwiML (Twilio Markup Language) to tell it what to do during a call. In our case, we’ll return a <Play> verb that points to a temporary URL serving the MP3 we just generated.

from flask import Flask, request, send_file, jsonify
from twilio.twiml.voice_response import VoiceResponse
import io
import uuid

app = Flask(__name__)

# In‑memory store for generated audio (good enough for a demo)
audio_store = {}

@app.route("/voice", methods=["POST"])
def voice():
    # Extract the caller's speech or a preset prompt
    prompt = "Welcome to the ElevenLabs + Twilio demo. How can I help you today?"
    audio_bytes = generate_speech(prompt)

    # Store audio with a short‑lived UUID
    audio_id = str(uuid.uuid4())
    audio_store[audio_id] = audio_bytes

    # Build TwiML that plays the generated audio
    response = VoiceResponse()
    response.play(url=f"{request.url_root}audio/{audio_id}")
    return str(response)

@app.route("/audio/<audio_id>", methods=["GET"])
def serve_audio(audio_id):
    audio = audio_store.get(audio_id)
    if not audio:
        return "Audio not found", 404
    # Return MP3 with proper MIME type
    return send_file(
        io.BytesIO(audio),
        mimetype="audio/mpeg",
        as_attachment=False,
        download_name="speech.mp3"
    )

Exposing the Flask app

# Run the server locally
flask run --port 5000

Then start an ngrok tunnel:

ngrok http 5000

Copy the HTTPS forwarding URL (e.g., https://abcd1234.ngrok.io) – you’ll need it in the Twilio console.

3️⃣ Wiring Twilio to Call Your Endpoint

1. In the Twilio Console, go to Phone Numbers → Manage → Active Numbers.
2. Click your trial number and set the Voice & Fax → A CALL COMES IN webhook to:

https://abcd1234.ngrok.io/voice

1. Save the changes.

Now, whenever someone dials your Twilio number, Twilio will POST to /voice, your Flask app will generate speech via ElevenLabs, and the caller will hear the generated audio instantly.

4️⃣ Adding Interactivity (Optional)

If you want the caller to respond and have the system speak back, you can combine Twilio’s <Gather> with the same TTS pipeline:

@app.route("/voice", methods=["POST"])
def voice():
    response = VoiceResponse()
    # Prompt the user
    response.say("Press 1 for sales, 2 for support, or say something after the beep.")
    # Gather DTMF or speech
    gather = response.gather(
        input="speech dtmf",
        timeout=5,
        speech_timeout="auto",
        action="/process_input",
        method="POST"
    )
    # Play a brief hold tone while waiting
    gather.play(url="https://api.twilio.com/cowbell.mp3")
    return str(response)

@app.route("/process_input", methods=["POST"])
def process_input():
    user_input = request.form.get("SpeechResult") or request.form.get("Digits")
    if not user_input:
        reply = "I didn't catch that. Goodbye!"
    else:
        reply = f"You said: {user_input}. Thanks for trying the demo."
    audio_bytes = generate_speech(reply)
    audio_id = str(uuid.uuid4())
    audio_store[audio_id] = audio_bytes

    resp = VoiceResponse()
    resp.play(url=f"{request.url_root}audio/{audio_id}")
    resp.hangup()
    return str(resp)

Now the call becomes a two‑way conversation, with ElevenLabs handling all the spoken responses.

5️⃣ Testing with cURL (Quick sanity check)

If you just want to verify that ElevenLabs returns audio correctly, you can hit the endpoint directly:

curl -X POST "https://api.elevenlabs.io/v1/text-to-speech/EXAMPLE_VOICE_ID" \
  -H "xi-api-key: YOUR_ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text":"Hello from ElevenLabs!", "model_id":"eleven_monolingual_v1"}' \
  --output speech.mp3

Play speech.mp3 locally—if it sounds natural, you’re ready to integrate with Twilio.

6️⃣ Scaling Considerations

• Cache audio: For static prompts (e.g., welcome messages), cache the MP3 on disk or a CDN to avoid repeated API calls.
• Rate limits: ElevenLabs enforces request limits based on your plan. Batch generate common phrases ahead of time.
• Security: Store API keys in environment variables or secret managers. Never commit them to source control.
• Concurrency: Use a production WSGI server (Gunicorn, uWSGI) behind a reverse proxy (NGINX) for high‑traffic scenarios.

7️⃣ Where to Go Next

• Voice cloning: Upload a few minutes of a speaker’s voice to ElevenLabs and generate a custom voice_id. Perfect for brand mascots or personalized assistants.
• Multilingual support: ElevenLabs now supports several languages; combine with Twilio’s language attribute in <Gather> for global reach.
• Audio post‑processing: Add background music or sound effects using pydub before sending the final MP3 to Twilio.

TL;DR

• ElevenLabs gives you lifelike TTS and voice cloning via a simple API.
• Twilio handles the telephony plumbing, letting you route calls to your own webhook.
• A few lines of Flask code glue them together, turning any phone line into an AI‑powered voice interface.

Ready to give your users a voice they’ll actually want to listen to? Grab your ElevenLabs API key (and a free trial) at https://try.elevenlabs.io/kr07zfuqn1bp, spin up the Flask app, point Twilio at your webhook, and start calling. Happy building!