Protocol & Data Messages

Data messages are used to communicate non-audio information during Ultravox calls. These messages enable real-time control and interaction with ongoing conversations.

Communication Methods

Client Data Channels → Used by our SDKs and WebSocket connections for bi-directional, real-time message exchange during calls. This is the primary method for client apps to interact with calls.
Data Connection → Add a data connection to your call to receive messages via a separate WebSocket connection. This is particularly useful for:
- Telephony integrations where the client doesn’t support WebRTC
- Server-side applications that need to monitor call events or route data to external systems
REST API → Inject messages into active calls via HTTP POST requests. See Sending Messages to Live Calls via REST API below for detailed implementation guidelines.

Messages at a Glance

Details on each message type appear below in Data Message Details.

Client-to-Server Messages

Type	Message	Description
Agent Behavior	ForcedAgentMessage	Forces the agent to say a specific message or invoke tools.
Call Control	HangUp	Instructs the agent to end the call with an optional farewell message.
Call Control	SetOutputMedium	Sets server’s output medium to text or voice.
System	Ping	Measures round-trip data latency.
Tools	ClientToolResult and DataConnectionToolResult	Contains the result of a tool invocation.
User Input	UserTextMessage	Used to send a user message to the agent.
Threads	SpawnThread	Spawns a new parallel thread. See Threads.

Server-to-Client Messages

Type	Message	Description
Conversation	CallStarted	Provides some basic information about the call at its start.
Conversation	Transcript	Contains text for an utterance made during the call.
System	Debug	Useful for application debugging. Excluded by default.
System	PlaybackClearBuffer	Used to clear buffered output audio. WebSocket only.
System	Pong	Server reply to a ping message.
System	State	Indicates the server’s current state.
Threads	ThreadSpawned	Confirms a thread was successfully created.
Threads	ThreadRejected	Indicates a thread spawn request was rejected.
Threads	ThreadTerminated	Indicates a thread has stopped running.
Threads	SideGenerationDelta	Streaming text delta from a thread’s generation.
Threads	SideGenerationCompleted	A thread’s generation has completed.
Tools	ClientToolInvocation and DataConnectionToolInvocation	Asks the client or data connection to invoke a tool.

Data Message Details

All messages are JSON objects with camelCase keys containing:

A required type field identifying the message type
Additional fields specific to each message type

Ping

A message sent by the client to measure round-trip data message latency. Message Structure

{
  "type": "ping",
  "timestamp": 1234567890.123
}

Fields

timestamp

float

required

Unix timestamp with millisecond precision. Client timestamp for latency measurement.

Pong

A message sent by the server in response to a PingMessage. The timestamp is copied from the PingMessage. Message Structure

{
  "type": "pong",
  "timestamp": 1234567890.123
}

Fields

timestamp

float

required

Echoed timestamp from the original ping message.

State

A message sent by the server to indicate its current state. Message Structure

{
  "type": "state",
  "state": "listening"
}

Fields

state

string

required

Current session state. One of: idle, listening, thinking, or speaking.

Transcript

A message containing text transcripts of user and agent utterances. Message Structure

{
  "type": "transcript",
  "role": "agent",
  "medium": "voice",
  "text": "Full transcript so far",  // Either text or delta will be set
  "delta": null,
  "final": false,
  "ordinal": 1
}

Fields

role

string

required

Who emitted the utterance. Must be either “user” or “agent”.

medium

string

default:"voice"

The medium through which the utterance was emitted. Either “text” or “voice”.

text

string

The full text of the transcript so far. Either this or delta will be set, but not both.

delta

string

The additional transcript text added since the last transcript message. Either this or text will be set, but not both.

final

boolean

required

Whether to expect additional transcript messages for this conversation round.

ordinal

integer

required

The ordinal of the message within the current call, used for ordering transcripts.

UserTextMessage

A user message sent via text. The message appears to the agent as if it came from the user. Message Structure

{
  "type": "user_text_message",
  "text": "Your message here",
  "urgency": "soon"  // Optional, defaults to "soon"
}

Fields

text

string

required

The content of the user message.

urgency

string

default:"soon"

Determines whether this message can interrupt the agent and whether it should trigger a generation. Options:

immediate → Interrupts the agent if speaking and starts a new generation immediately.
soon → Doesn’t interrupt but starts a generation at the next opportunity.
later → Message is considered during the next natural generation without forcing a new generation.

threadId

string

default:"UI"

Optional thread ID to send this message to. Defaults to “UI” (the main conversation).

SetOutputMedium

Message sent by the client to set the server’s output medium. Message Structure

{
  "type": "set_output_medium",
  "medium": "voice"
}

Fields

medium

string

required

Output medium to use. Must be either “voice” or “text”.

ClientToolInvocation and DataConnectionToolInvocation

Sent by the server to ask the client or data connection to invoke a tool with the given parameters. The client or data connection is expected to send back a ClientToolResultMessage or DataConnectionToolResultMessage with a matching invocationId. Message Structure

{
  "type": "client_tool_invocation", // Or "data_connection_tool_invocation" for data connections
  "toolName": "get_weather",
  "invocationId": "unique-invocation-id",
  "parameters": {
    "location": "Seattle"
  }
}

Fields

toolName

string

required

Name of the tool to invoke.

invocationId

string

required

Unique identifier for this invocation. Must be included in the corresponding result.

parameters

object

required

Tool-specific parameters as a JSON object.

ClientToolResult and DataConnectionToolResult

Contains the result of a tool invocation. Message Structure

{
  "type": "client_tool_result", // Or "data_connection_tool_result" for data connections
  "invocationId": "matching-invocation-id",
  "result": "Tool execution result",
  "responseType": "tool-response",
  "agentReaction": "speaks",
  "errorType": null,
  "errorMessage": null,
  "updateCallState": null
}

Fields

invocationId

string

required

Must match the invocationId from the corresponding invocation.

result

string

Typically the tool execution result as viewed by the agent, which is often a JSON string. May be omitted for errors. For responseTypes other than tool-response, this may be a JSON string for an object that further specifies how the response should be handled. See special response types.

responseType

string

default:"tool-response"

Type of response being provided. See special response types.

agentReaction

string

default:"speaks"

How the agent should react. Options: “speaks” (default), “listens”, or “speaks-once”. See Agent Responses to Tools for more.

errorType

string

Error classification if the tool failed. Should be omitted when result is set.Options:

undefined → Tool with the given name does not exist
implementation-error → Tool exists but execution failed

errorMessage

string

Human-readable error description if the tool failed. This is not seen by the model but may be used for debugging.

updateCallState

object

Optional state updates to apply to the call. See Tool State for more.

Debug

A message sent by the server to communicate debug information. Disabled by default. Message Structure

{
  "type": "debug",
  "message": "Debug information here"
}

Fields

message

string

required

Debug information or diagnostic details.

Debug messages are disabled by default and must be explicitly enabled for debugging purposes.

CallStarted

Basic call metadata shared by the server when a call begins. Message Structure

{
  "type": "call_started",
  "callId": "550e8400-e29b-41d4-a716-446655440000"
}

Fields

callId

string

required

The UUID of the call that has started.

PlaybackClearBuffer

Message sent by the server to clear buffered output audio. Integrators should drop as much unplayed output audio as possible for interruptions to function properly. Message Structure

{
  "type": "playback_clear_buffer"
}

This message is only used with WebSocket connections. Handling this message allows for larger client buffers while maintaining responsive interrupts. Larger client buffers make choppy audio less likely in the presence of network disruption or resource contention.

ForcedAgentMessage

Forces the agent to say a specific message or invoke tools. Message Structure

{
  "type": "forced_agent_message",
  "content": "Text for the agent to say",  // Optional (default: "")
  "toolCalls": [  // Optional array of tool calls
    {
      "id": "unique-invocation-id",  // Optional, generated if not provided
      "name": "tool_name",
      "arguments": {
        "param1": "value1"
      }
    }
  ],
  "knownToolResults": [  // Optional, skips execution of matching tool calls using these results instead
    {
      "invocationId": "unique-invocation-id",
      "result": "Tool execution result",
      "responseType": "tool-response",
      "agentReaction": "speaks",
    }
  ],
  "uninterruptible": false,  // Optional (default: false)
  "urgency": "soon"  // Optional: "immediate" or "soon" (default: "soon")
}

Fields

content

string

default:""

Text content the agent should speak.

toolCalls

array

Array of tool invocations to execute.

knownToolResults

array

Array of known tool results. If specified, any toolCalls with matching invocationIds will use these results instead of executing the tool. See ClientToolResult for the structure of each result object.

uninterruptible

boolean

If true, prevents interruption while the agent speaks this message. (Note that tools are always uninterruptible.)

urgency

string

default:"soon"

Controls when the message is processed. Must be either “immediate” (may interrupt the user or agent) or “soon” (process at next opportunity).

threadId

string

default:"UI"

Optional thread ID to send this message to. Defaults to “UI” (the main conversation).

HangUp

Instructs the agent to end the call with an optional farewell message. Message Structure

{
  "type": "hang_up",
  "message": "Goodbye!"
}

Fields

message

string

default:""

Final message to speak before ending the call.

Thread Messages

The following messages are used to manage Threads — parallel conversations that run alongside the main call. For conceptual background and usage examples, see the Threads guide.

SpawnThread

Spawns a new parallel thread that forks from the parent’s conversation history. See Threads for a full explanation. Message Structure

{
  "type": "spawn_thread",
  "newThreadId": "my-thread-1",       // Optional, auto-generated if omitted
  "parentThreadId": "UI",             // Optional, defaults to "UI" (main conversation)
  "ifExists": "reject",              // Optional, defaults to "reject"
  "additionalMessages": [             // Optional
    {
      "type": "user_text_message",
      "text": "Perform this background task."
    }
  ],
  "toolFilter": {                     // Optional
    "allowedTools": ["searchDatabase"]
  },
  "limits": {                         // Optional
    "generationLimit": 5,
    "threadOutputTokenLimit": 2000
  }
}

Fields

newThreadId

string

Unique identifier for the new thread. If omitted, an id will be auto-generated.

parentThreadId

string

default:"UI"

The thread to fork from. Defaults to “UI” (the main conversation). If the referenced thread has failed or doesn’t exist, this spawn attempt will be rejected.

ifExists

string

default:"reject"

Controls behavior when newThreadId matches an existing thread. Options:

reject (default) → The spawn request is rejected and a ThreadRejected message is sent.
replace → The existing thread is terminated and a new thread is spawned with the same ID. Useful for re-syncing a thread with the current conversation state.

additionalMessages

array

Messages to append to the forked history before the thread starts generating. Each element must be either a UserTextMessage or ForcedAgentMessage. Every ForcedAgentMessage except the last must have known results for each tool call. The last may have unmatched calls, in which case the thread will begin by executing those.

toolFilter

object

Restricts which tools the thread can use. Specify either allowedTools (allowlist) or disallowedTools (blocklist), or both. If both are provided, the allowlist is applied first and then the blocklist is applied to the remaining tools. If neither is specified (or this field is omitted), the thread will be allowed to use all tools.

limits

object

Resource constraints for the thread. Available fields:

threadOutputTokenLimit (integer) → Max total output tokens for the thread.
threadFuzzyInputTokenLimit (integer) → Approximate total (uncached) input token limit.
generationLimit (integer) → Max LLM generation rounds.
generationOutputTokenLimit (integer) → Max output tokens per generation.
generationFuzzyInputTokenLimit (integer) → Approximate (uncached) input token limit per generation.

Limits are a useful way to constrain model usage (and thus costs) for a thread, but leaving threads unconstrained is also totally reasonable, especially when the context provided to them keeps them focused on a specific task.

ThreadSpawned

Sent by the server to confirm a thread was successfully created. Message Structure

{
  "type": "thread_spawned",
  "threadId": "my-thread-1"
}

Fields

threadId

string

required

The ID of the newly created thread.

ThreadRejected

Sent by the server when a thread spawn request cannot be fulfilled. Message Structure

{
  "type": "thread_rejected",
  "threadId": "my-thread-1",
  "reason": "Thread with this ID already exists"
}

Fields

threadId

string

required

The thread ID from the spawn request.

reason

string

required

Why the thread could not be created.

ThreadTerminated

Sent by the server when a thread stops running. Message Structure

{
  "type": "thread_terminated",
  "threadId": "my-thread-1",
  "reason": "limit reached"
}

Fields

threadId

string

required

The ID of the terminated thread.

reason

string

required

Why the thread was terminated. Common reasons include: limit reached or cancellation.

SideGenerationDelta

Streaming text delta from a (non-UI) thread’s LLM generation, similar to a transcript delta. Message Structure

{
  "type": "side_generation_delta",
  "threadId": "my-thread-1",
  "delta": "Here is the next chunk of text..."
}

Fields

threadId

string

required

The ID of the thread producing this generation.

delta

string

required

The incremental text produced since the last delta message.

SideGenerationCompleted

Sent when a (non-UI) thread’s generation round finishes successfully. Message Structure

{
  "type": "side_generation_completed",
  "threadId": "my-thread-1",
  "text": "Full generated response text",
  "toolCalls": []
}

Fields

threadId

string

required

The ID of the thread that completed generation.

text

string

required

The complete generated text for this generation round.

toolCalls

array

Any tool calls the LLM requested during this generation. The thread will execute these automatically.

Sending Messages to Live Calls via REST API

The Send Data Message to Call endpoint allows you to inject messages into active calls (calls that are joined and not yet ended).

Responses

Successful messages sent via the REST API will receive a 204 No Content response with an empty body. Potential error responses include:

401 Unauthorized: Missing or invalid API key
403 Forbidden: Insufficient authorization
422 Unprocessable Entity: Call is not active (either not joined yet or already ended)

Getting Started

Agents & Calls

Telephony

Web, Apps, Websockets

Tools

Voices

Webhooks

Noise & VAD

Protocol & Data Messages

Communication Methods

Messages at a Glance

Client-to-Server Messages

Server-to-Client Messages

Data Message Details

Ping

Pong

State

Transcript

UserTextMessage

SetOutputMedium

ClientToolInvocation and DataConnectionToolInvocation

ClientToolResult and DataConnectionToolResult

Debug

CallStarted

PlaybackClearBuffer

ForcedAgentMessage

HangUp

Thread Messages

SpawnThread

ThreadSpawned

ThreadRejected

ThreadTerminated

SideGenerationDelta

SideGenerationCompleted

Sending Messages to Live Calls via REST API

Supported Message Types

Responses

Getting Started

Agents & Calls

Telephony

Web, Apps, Websockets

Tools

Voices

Webhooks

Noise & VAD

​Communication Methods

​Messages at a Glance

​Client-to-Server Messages

​Server-to-Client Messages

​Data Message Details

​Ping

​Pong

​State

​Transcript

​UserTextMessage

​SetOutputMedium

​ClientToolInvocation and DataConnectionToolInvocation

​ClientToolResult and DataConnectionToolResult

​Debug

​CallStarted

​PlaybackClearBuffer

​ForcedAgentMessage

​HangUp

​Thread Messages

​SpawnThread

​ThreadSpawned

​ThreadRejected

​ThreadTerminated

​SideGenerationDelta

​SideGenerationCompleted

​Sending Messages to Live Calls via REST API

​Supported Message Types

​Responses

Communication Methods

Messages at a Glance

Client-to-Server Messages

Server-to-Client Messages

Data Message Details

Ping

Pong

State

Transcript

UserTextMessage

SetOutputMedium

ClientToolInvocation and DataConnectionToolInvocation

ClientToolResult and DataConnectionToolResult

Debug

CallStarted

PlaybackClearBuffer

ForcedAgentMessage

HangUp

Thread Messages

SpawnThread

ThreadSpawned

ThreadRejected

ThreadTerminated

SideGenerationDelta

SideGenerationCompleted

Sending Messages to Live Calls via REST API

Supported Message Types

Responses