Telephony

The Ultravox API allows you to create AI-powered voice applications that can interact through various protocols:

WebRTC → Default protocol for browser and mobile applications.
Regular Phone Numbers → Connect Ultravox to phone calls you make or receive (via Telnxy, Twilio, Plivo, or jambonz).
WebSockets → Direct server-to-server integration.

Choosing a Protocol

Choose your integration method based on your needs:

WebRTC: Best for most integrations, especially for any client deployment (for example, browsers or mobile clients). This is the default. Get started with the Ultravox client SDK.
Phone: For traditional phone network integration. Ultravox integrates directly with Telnyx, Twilio, Plivo, and jambonz (bring your own carrier).
WebSocket: For server-to-server integration, especially when you already have high-bandwidth connections between your server and clients. Check out the WebSocket Integration guide for more information.

Phone Integration

Ultravox integrates with multiple telephony providers, enabling you to create AI-powered voice applications that can interact through regular phone networks. You can build AI agents that can make outgoing calls and answer incoming calls, opening up possibilities for customer service, automated outreach, and other voice-based AI applications.

Supported Providers

Twilio → Uses Twilio Media Streams.
Telnyx → Uses Telnyx Media Streaming.
Plivo → Uses Plivo AudioStream.
jambonz → Uses jambonz llm verb and connects to the carrier of your choice.
Voximplant → See the integration guide.

Prerequisites

Prerequisites

Make sure you have:

An active account with your chosen provider (Telnyx, Twilio, Plivo, or any carrier when connecting via jambonz)
A phone number purchased from your provider
A jambonz account (if connecting via jambonz)

Connecting Ultravox to a Phone Call

”Calls” is Overloaded

It can be a bit confusing because Ultravox Realtime uses the concept of a “call” to mean a voice session between an AI agent and another party. For phone calls, you will accept incoming or make outgoing calls via your chosen telephony provider and then connect those calls to an Ultravox call.

Creating an Ultravox call that you can connect with a phone call is similar to creating a WebRTC call, but requires specific parameters in the Create Call command:

medium

object

default:"{'webRtc': {}}"

Tells Ultravox which protocol to use. For phone calls, must be set to one of:

{"telnyx": {}}, {"twilio": {}}, or {"plivo": {}}.

Defaults to {"webRtc": {}}.

firstSpeaker

string

default:"FIRST_SPEAKER_AGENT"

Tells Ultravox who should speak first.

For outgoing calls, typically set to "FIRST_SPEAKER_USER".

The default is "FIRST_SPEAKER_AGENT".

Example: Outgoing Call via Telnyx
{
  "systemPrompt": "You are a helpful assistant...",
  ...
  "medium": {
    "telnyx": {} // or "twilio": {} or "plivo": {}
  },
  "firstSpeaker": "FIRST_SPEAKER_USER"
}

jambonz have integrated Ultravox into their llm verb, so you won’t need to create an Ultravox call—this is all done for you. Just follow the instructions in the “Provider-Specific Integration” section below.

Provider-Specific Integration

Outgoing Calls with Telnyx

Create an Ultravox Call

Create a new call as shown above with medium: { "telnyx": {} }, firstSpeaker: "FIRST_SPEAKER_USER", and get a joinUrl.

Connect Ultravox to the Telnyx Phone Call

Use the joinUrl with a TeXML <Stream>:

// Example using the telnyx node library
const call = await telnyx.calls.create({
  connection_id: "uuid",
  to: phoneNumber,
  from: telnyxPhoneNumber,
  stream_url: joinUrl,
  stream_track: "both_tracks",
  stream_bidirectional_mode: "rtp"
});

Or using TeXML:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="${joinUrl}" bidirectionalMode="rtp"  codec="PCMU" bidirectionalCodec="PCMU"/>
  </Connect>
</Response>

Incoming Calls with Telnyx

Create an Ultravox Call

Create a new call with medium: { "telnyx": {} } and firstSpeaker set to "FIRST_SPEAKER_AGENT".

Connect the Inbound Telnyx Call to Ultravox

Use the joinUrl with a TeXML <Stream>:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="${joinUrl}" bidirectionalMode="rtp" codec="PCMU" bidirectionalCodec="PCMU"/>
  </Connect>
</Response>

Telnyx codec

Telnyx allows setting both codec and bidirectionalCodec. The former controls user audio while the latter controls agent audio. When using with Ultravox, these must have the same value because Telnyx only tells us about one of them! If your users are mostly in Europe, you’ll likely want to set both to PCMA. Otherwise setting both to PCMU is preferred but leaving them both unset is fine to get started.

For more details, see the Telnyx documentation.

Outgoing Calls with Telnyx

Create an Ultravox Call

Create a new call as shown above with medium: { "telnyx": {} }, firstSpeaker: "FIRST_SPEAKER_USER", and get a joinUrl.

Connect Ultravox to the Telnyx Phone Call

Use the joinUrl with a TeXML <Stream>:

// Example using the telnyx node library
const call = await telnyx.calls.create({
  connection_id: "uuid",
  to: phoneNumber,
  from: telnyxPhoneNumber,
  stream_url: joinUrl,
  stream_track: "both_tracks",
  stream_bidirectional_mode: "rtp"
});

Or using TeXML:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="${joinUrl}" bidirectionalMode="rtp"  codec="PCMU" bidirectionalCodec="PCMU"/>
  </Connect>
</Response>

Incoming Calls with Telnyx

Create an Ultravox Call

Create a new call with medium: { "telnyx": {} } and firstSpeaker set to "FIRST_SPEAKER_AGENT".

Connect the Inbound Telnyx Call to Ultravox

Use the joinUrl with a TeXML <Stream>:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="${joinUrl}" bidirectionalMode="rtp" codec="PCMU" bidirectionalCodec="PCMU"/>
  </Connect>
</Response>

Telnyx codec

For more details, see the Telnyx documentation.

Outgoing Calls with Twilio

Create an Ultravox Call

Create a new call as shown above with medium: { "twilio": {} }, firstSpeaker: "FIRST_SPEAKER_USER", and get a joinUrl.

Connect Ultravox to the Twilio Phone Call

Use the joinUrl with a Twilio <Stream>:

// Example using the twilio node library
const call = await client.calls.create({
    twiml: `<Response>
                <Connect>
                    <Stream url="${joinUrl}"/>
                </Connect>
            </Response>`,
    to: phoneNumber,
    from: twilioPhoneNumber
});

Incoming Calls with Twilio

Create an Ultravox Call

Create a new call with medium: { "twilio": {} } and firstSpeaker set to "FIRST_SPEAKER_AGENT".

Connect the Inbound Twilio Call to Ultravox

Use the joinUrl with a Twilio <Stream>:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Connect>
        <Stream url="your_ultravox_join_url" />
    </Connect>
</Response>

For more details, see the Twilio documentation.

Outgoing Calls with Plivo

Create an Ultravox Call

Create a new call as shown above with medium: { "plivo": {} }, firstSpeaker: "FIRST_SPEAKER_USER", and get a joinUrl.

Connect Ultravox to the Plivo Phone Call

Use the joinUrl with AudioStream:

// Example using the plivo node library
// This assumes our server exposes an endpoint at `answerUrl`
const call = await plivo.calls.create({
  to: phoneNumber,
  from: plivoPhoneNumber,
  answer_url: answerUrl, // URL that returns the XML below
  answer_method: "GET"
});

The answer URL should return:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Stream keepCallAlive="true" 
            contentType="audio/x-l16;rate=16000"
            bidirectional="true">
        ${joinUrl}
    </Stream>
</Response>

Note: For best audio quality, we recommend audio/x-l16;rate=16000. However, any contentType supported by Plivo will work with Ultravox.

Incoming Calls with Plivo

Create an Ultravox Call

Create a new call with medium: { "plivo": {} } and firstSpeaker set to "FIRST_SPEAKER_AGENT".

Connect the Inbound Twilio Call to Ultravox

Use the joinUrl with AudioStream:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Stream keepCallAlive="true"
            contentType="audio/x-l16;rate=16000" 
            bidirectional="true">
        ${joinUrl}
    </Stream>
</Response>

For more details, see the Plivo documentation.

jambonz Portal Setup

jambonz is a “bring your own everything” open-source telephony platform that integrates Ultravox directly via their llm verb. This gives you the flexibility to choose your carrier of choice, you’ll just need to add it in your jambonz dashboard.

Add Your Carrier in jambonz

In jambonz, we use the terms “carrier” and “SIP trunk” interchangeably. jambonz is a “Bring your own carrier” platform, which means that you can connect any sip network provider or device. Add your carrier of choice in your jambonz dashboard to get started.

Add a Speech Provider in jambonz

Next, you need to add speech credentials for your chosen vendor.

Create a New jambonz Application

A jambonz application configured via the jambonz portal defines how calls are handled by linking them to your custom logic through webhooks or WebSocket endpoints. When you create an application, you specify:

Call webhook URL: Where jambonz sends call events.
Call status webhook URL: For receiving call status updates.
Speech vendors: Your chosen TTS/STT providers.

Once saved, you can associate phone numbers or SIP trunks with this application, ensuring that incoming calls are routed to your specified logic. This setup allows you to implement features like speech recognition, text-to-speech, call routing, and integration with AI services.

Add a Phone Number in jambonz

Finally, you need to add a phone number provisioned from your carrier of choice. At the bottom of the page select the jambonz application you just created to link your new virtual number to that application.

Incoming Calls with jambonz

// Example using the @jambonz/node-client-ws library
session
.pause({length: 1.5})
.llm({
  vendor: 'ultravox',
  model: 'fixie-ai/ultravox',
  auth: {
    apiKey
  },
  actionHook: '/final',
  eventHook: '/event',
  llmOptions: {
    systemPrompt: 'You are an agent named Karen. Greet the user and ask how you can help.',
    firstSpeaker: 'FIRST_SPEAKER_AGENT',
    initialMessages: [{
      medium: 'MESSAGE_MEDIUM_VOICE',
      role: 'MESSAGE_ROLE_USER'
    }],
    model: 'fixie-ai/ultravox',
    voice: 'Tanya-English',
    transcriptOptional: true,
  }
})
.hangup()
.send();

For more details see the llm verb in the jambonz docs.

Outgoing Calls with jambonz

In addition to the inbound scenario, you’ll have to create a call that connects to the destination number (phoneNumber) and points to the jambonz application that defines how the call should be handled. Find the APPLICATION_SID in the jambonz portal by clicking on the application you created during the setup process.

  const JambonzClient = require('@jambonz/node-client');

  const client = JambonzClient(
    process.env.JAMBONZ_ACCOUNT_SID,
    process.env.JAMBONZ_API_KEY,
    {baseUrl: process.env.JAMBONZ_REST_API_BASE_URL || 'https://api.jambonz.cloud/v1'}
  );

  const call = await client.calls.create({
      from: process.env.FROM_NUMBER,  
        to: {
          type : 'phone',
          number: phoneNumber,
          trunk: process.env.CARRIER
          },
      application_sid: process.env.APPLICATION_SID
  });

For more details, see the jambonz documentation and example code.

Provider Support

We currently integrate with Telnyx, Twilio, Plivo, and jambonz (bring your own carrier voice platform). Please let us know if there’s another integration you’d like to see.

DTMF

Ultravox provides comprehensive support for DTMF (Dual-Tone Multi-Frequency) tones, enabling both sending and receiving tones during phone calls. This enables AI agents to interact with traditional phone systems and allows you to build voice applications that can respond to keypad inputs.

DTMF and WebRTC

Due to the audio codec used in WebRTC connections, DTMF tones are inaudible when using WebRTC. The playDtmfSounds tool is intended for use with telephony integrations.

Receiving DTMF Tones

Ultravox automatically converts incoming DTMF tones to text, making it easy to build interactive voice applications that respond to keypad input. When a caller presses keys on their phone keypad, the tones are converted to text that your AI agent can understand and respond to.

For example, if a caller presses “5” on their keypad, your agent will receive this as text and can respond accordingly:

// Example system prompt for an agent that handles DTMF input
{
  "systemPrompt": `You are an automated phone system.
    When a caller joins, say: "Welcome! Press 1 for sales, 2 for support, or 3 for billing."
    If they press 1, transfer them to sales using the transfer tool.
    If they press 2, transfer them to support.
    If they press 3, transfer them to billing.
    If they press any other key, ask them to try again with a valid option."`
}

Sending DTMF Tones

The built-in playDtmfSounds tool allows your AI agent to send DTMF tones, which is useful for navigating Interactive Voice Response (IVR) systems or other phone trees. To enable the tool, add it to the selectedTools array when creating a call or call stage:

// Example request body for creating a call with DTMF capability
{
  "systemPrompt": "You are a helpful assistant. When prompted to dial an extension, use the 'playDtmfSounds' tool to send the appropriate tones.",
  "selectedTools": [
    { "toolName": "playDtmfSounds" }
  ]
}

The playDtmfSounds tool accepts a string parameter named digits and works with the following tones: 0-9, *, #, A-D.

For example:

// Example of using the playDtmfSounds tool to dial an extension
{
  "digits": "123#"  // Will play tones for 1, 2, 3, and # in sequence
}

Note: the playDtmfSounds tool uses an automatic parameter that sends the proper sample rate of the source audio and should be treated as an implementation detail.

Common Use Cases

Building interactive phone trees or IVR systems
Creating agents that can navigate existing phone systems
Enabling quick responses through keypad input
Collecting numeric input (e.g., account numbers, PIN codes)
Building hybrid voice/keypad interfaces

Getting Started

Essentials

Advanced Guides + Tutorials

Choosing a Protocol

Phone Integration

Supported Providers

Prerequisites

Connecting Ultravox to a Phone Call

Provider-Specific Integration

Outgoing Calls with Telnyx

Incoming Calls with Telnyx

Outgoing Calls with Telnyx

Incoming Calls with Telnyx

Outgoing Calls with Twilio

Incoming Calls with Twilio

Outgoing Calls with Plivo

Incoming Calls with Plivo

jambonz Portal Setup

Incoming Calls with jambonz

Outgoing Calls with jambonz

DTMF

Receiving DTMF Tones

Sending DTMF Tones

Common Use Cases

Getting Started

Essentials

Advanced Guides + Tutorials

​Choosing a Protocol

​Phone Integration

​Supported Providers

​Prerequisites

​Connecting Ultravox to a Phone Call

​Provider-Specific Integration

​Outgoing Calls with Telnyx

​Incoming Calls with Telnyx

​Outgoing Calls with Telnyx

​Incoming Calls with Telnyx

​Outgoing Calls with Twilio

​Incoming Calls with Twilio

​Outgoing Calls with Plivo

​Incoming Calls with Plivo

​jambonz Portal Setup

​Incoming Calls with jambonz

​Outgoing Calls with jambonz

​DTMF

​Receiving DTMF Tones

​Sending DTMF Tones

​Common Use Cases

Choosing a Protocol

Phone Integration

Supported Providers

Prerequisites

Connecting Ultravox to a Phone Call

Provider-Specific Integration

Outgoing Calls with Telnyx

Incoming Calls with Telnyx

Outgoing Calls with Telnyx

Incoming Calls with Telnyx

Outgoing Calls with Twilio

Incoming Calls with Twilio

Outgoing Calls with Plivo

Incoming Calls with Plivo

jambonz Portal Setup

Incoming Calls with jambonz

Outgoing Calls with jambonz

DTMF

Receiving DTMF Tones

Sending DTMF Tones

Common Use Cases