Integrate with your server via direct WebSocket connections.
Server-to-Server Only WebSocket connections are designed for server-to-server communication. For browser or mobile applications, use our client SDKs with WebRTC for optimal performance. WebSocket connections over TCP can experience audio blocking and ordering constraints that make them unsuitable for direct client use.
Creating a WebSocket-based call with Ultravox requires setting medium to serverWebSocket and passing in parameters for sample rates and buffer size.
inputSampleRate (required): Sample rate for input (user) audio (e.g., 48000).
outputSampleRate (optional): Sample rate for output (agent) audio (defaults to inputSampleRate).
clientBufferSizeMs (optional): Size of the client-side audio buffer in milliseconds. Smaller buffers allow for faster interruptions but may cause audio underflow if network latency fluctuates too greatly. For the best of both worlds, set this to some large value (e.g. 30000) and implement support for PlaybackClearBuffer messages. (Defaults to 60).
import websocketssocket = await websockets.connect(join_url)audio_send_task = asyncio.create_task(_send_audio(socket))async for message in socket: if isinstance(message, bytes): # Handle agent audio data else: # Handle data message. See "Data Messages"...async def _send_audio(socket: websockets.WebSocketClientProtocol): async for chunk in some_audio_source: # chunk should be a bytes object containing s16le PCM audio from the user self._socket.send(chunk)
See Data Messages for more information on all available messages.
Data Messages WebSocket connections use the same message format as WebRTC data channels. See our Data Messages documentation for detailed message specifications.