# Connecting Twilio Calls to a Voice AI with Media Streams

> Stream Twilio call audio to a voice AI with <Connect><Stream>, pass context with <Parameter>, and route the call afterwards with the Connect action. TwiML and failover.

Published: 2026-10-17
Source: https://voicast.jp/en/blog/twilio-media-streams-voice-ai

## Key points

- To connect a Twilio call to a voice AI, return TwiML with <Connect><Stream>, which opens a bidirectional WebSocket. <Start><Stream> is one-way and can't play the agent's voice back into the call.
- The Stream url doesn't accept query strings, so pass tenant IDs and keys as <Parameter> elements. They arrive in the WebSocket start message.
- Add an action URL to <Connect>. When the WebSocket closes, Twilio requests it, and you decide what happens next (transfer, hang up) based on the agent's result.

To put a voice AI on a Twilio call, respond to the incoming-call webhook with TwiML containing `<Connect><Stream>`. Twilio opens a bidirectional WebSocket to the agent and streams call audio both ways. When the conversation ends and the WebSocket closes, Twilio requests the `action` URL on `<Connect>`, where you look up the agent's result and decide whether to transfer the caller, hang up or fall back to your normal flow. Your numbers stay on Twilio; only the conversation goes to the AI.

## Connect vs. Start

For a conversational agent, **use `<Connect><Stream>`**. Twilio offers two ways to stream ([TwiML Stream](https://www.twilio.com/docs/voice/twiml/stream)):

| | `<Start><Stream>` | `<Connect><Stream>` |
|---|---|---|
| Direction | Call → WebSocket only | Bidirectional (you can send audio into the call) |
| Next TwiML verb | Runs immediately | Waits until the WebSocket closes |
| Use for | Recording, transcription, analytics | Conversational voice AI |

`<Start><Stream>` lets you listen, but not talk back. `<Connect><Stream>` lets the agent's audio flow back over the same socket.

## The TwiML

```xml
<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect action="https://saas.example.com/twilio/after">
    <Stream url="wss://voice.example.com/ws">
      <Parameter name="tenant" value="ten_123" />
      <Parameter name="from" value="+15555550123" />
    </Stream>
  </Connect>
</Response>
```

Three things to know:

1. **`url` must be `wss`, and it doesn't support query strings** ([TwiML Stream](https://www.twilio.com/docs/voice/twiml/stream)). Anything the agent needs to know (which customer this is, which configuration to use) goes in `<Parameter>` elements.
2. Parameter values arrive in the `start` message on the WebSocket.
3. With an `action` URL, Twilio makes a request to it when `<Connect>` ends. Without one, Twilio moves on to the next verb, and if there isn't one, the conversation ends ([TwiML Connect](https://www.twilio.com/docs/voice/twiml/connect)).

## What flows over the WebSocket

Your media server exchanges these messages with Twilio ([WebSocket messages](https://www.twilio.com/docs/voice/media-streams/websocket-messages)):

| Direction | Message | Contents |
|---|---|---|
| Twilio → you | `connected` | Socket established |
| Twilio → you | `start` | Stream metadata and your `<Parameter>` values |
| Twilio → you | `media` | Caller audio (base64) |
| Twilio → you | `dtmf` | Keypad presses |
| Twilio → you | `mark` | Audio you sent has finished playing |
| Twilio → you | `stop` | Stream ended |
| You → Twilio | `media` | Audio to play into the call |
| You → Twilio | `mark` | A label to be echoed back when playback reaches it |
| You → Twilio | `clear` | Discard buffered audio (for barge-in) |

Audio is always 8,000 Hz μ-law, so you'll likely need to convert it for your STT and TTS providers. To stop the agent mid-sentence when the caller starts talking, send `clear`.

## Routing the call with action

When the conversation is over, the agent closes the WebSocket. Twilio then requests your `action` URL with the standard voice parameters, including `CallSid`. The TwiML you return decides what happens next:

- Agent collected the details and said goodbye → `<Hangup/>`
- Agent decided to hand off → `<Dial>` a staff number
- Agent unreachable or errored → your usual flow, such as ringing staff

The pattern: in the `action` handler, fetch the agent's result for this `CallSid` and branch on it.

## Failing safe

Design so that calls still reach a human when the AI doesn't:

1. Have the agent's WebSocket server **close immediately** on a bad key or an internal problem. Twilio ends `<Connect>` and goes to `action`.
2. In the `action` handler, **treat "no result" as a failure** and route to staff.
3. Set a **Fallback URL** on the Twilio number so that if your own webhook fails, Twilio still has somewhere to go.

With that in place, an outage costs you automation, not calls. More in [Fallbacks: keeping calls flowing when the voice AI fails](https://voicast.jp/en/blog/voice-ai-fallbacks).

## Connecting to voicast

voicast's voice endpoint is `wss://voice.voicast.jp/ws`. Pass the tenant ID (the business taking the call), its connection key (`vts_…`) and the caller's number as parameters. After the conversation, call `GET /v1/calls/by-provider/{CallSid}` from your `action` handler and branch on `outcome`. An Express example:

```ts
import express from 'express';

const app = express();
app.use(express.urlencoded({ extended: false }));

const esc = (s: string) =>
  s.replace(/[&<>"']/g, (c) => ({ '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&apos;' })[c]!);

// Incoming call: find the tenant for this number and stream to voicast
app.post('/twilio/voice', async (req, res) => {
  const t = await findTenantByNumber(req.body.To); // your DB: { voicastTenantId, voicastSecret, staffNumber }
  res.type('text/xml').send(`<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect action="https://saas.example.com/twilio/after">
    <Stream url="wss://voice.voicast.jp/ws">
      <Parameter name="tenant" value="${esc(t.voicastTenantId)}" />
      <Parameter name="secret" value="${esc(t.voicastSecret)}" />
      <Parameter name="from" value="${esc(req.body.From ?? '')}" />
    </Stream>
  </Connect>
</Response>`);
});

// After the conversation: route based on the result
app.post('/twilio/after', async (req, res) => {
  const t = await findTenantByNumber(req.body.To);
  const r = await fetch(
    `https://api.voicast.jp/v1/calls/by-provider/${encodeURIComponent(req.body.CallSid)}`,
    { headers: { Authorization: `Bearer ${process.env.VOICAST_API_KEY}` } },
  );
  const call = r.ok ? await r.json() : null;

  let twiml: string;
  if (call?.outcome === 'completed' || call?.outcome === 'answered' || call?.outcome === 'hangup') {
    twiml = '<Hangup/>';
  } else {
    // handoff, error, or no result: ring staff
    twiml = `<Dial>${esc(t.staffNumber)}</Dial>`;
  }
  res.type('text/xml').send(`<?xml version="1.0" encoding="UTF-8"?><Response>${twiml}</Response>`);
});

app.listen(3000);
```

Keep connection keys on your server and inject them only when rendering TwiML. Validate `X-Twilio-Signature` on incoming Twilio webhooks ([Webhooks security](https://www.twilio.com/docs/usage/webhooks/webhooks-security)). voicast doesn't own numbers or lines, so number configuration and the transfer flow stay in your app. See [how it works](https://voicast.jp/en#how).

## FAQ

### Can I put the tenant ID in the Stream URL?

No. Twilio's Stream `url` doesn't support query strings. Pass values like tenant IDs and keys as `<Parameter>` elements and read them from the WebSocket `start` message.

### What happens if I leave out action?

When `<Connect>` ends, Twilio continues with the next verb in the document, or ends the call if there isn't one. If you want to branch on the agent's result, use `action` and decide in the handler.

### What happens to calls if the voice AI is down?

If the agent's WebSocket closes right away, Twilio proceeds to your `action` URL. Route "no result" to staff there, and callers still reach a person.
