HTTP vs. Client Tools
Choose the right tool implementation for your use case.
Unlike using tools with single-generation LLM APIs, Ultravox Realtime actually calls your tool. This means you need to do a bit more work upfront in defining tools with the proper authentication and parameters.
Ultravox supports three primary types of tool implementations: HTTP tools, Client tools, and Data Connection tools. Each has distinct advantages and use cases.
HTTP Tools
HTTP tools (AKA “server tools”) are the most common and flexible option. Your tool runs on your server, and Ultravox calls it via HTTP requests during conversations.
How HTTP Tools Work
- Agent triggers tool during conversation.
- Ultravox sends HTTP request to your server.
- Your server processes the request and returns a response.
- Agent continues conversation with the tool result.
HTTP Tool Advantages
✅ Server-side logic: Full access to databases, APIs, and business logic
✅ Any call medium: Works with WebRTC, telephony, and websockets
✅ Scalable: Runs on your infrastructure with your scaling strategies
✅ Secure: Keep sensitive data and credentials on your servers
✅ Language agnostic: Implement in any programming language
HTTP Tool Implementation
Error Handling
Return appropriate HTTP status codes:
Client Tools
Client tools run directly in the client application using our SDKs. They’re perfect for UI interactions and client-side operations.
Client tools work best with our client SDKs, which are designed for the webrtc call medium. See Client Tools to learn how those are registered and used with the Ultravox Client SDK.
You can also use client tools with a websocket medium. See the ClientToolInvocation
and ClientToolResult
data messages.
If you want a similar experience to client tools with a telephony medium, you have two options:
- Handle telephony using voximplant and define your tools in your voximplant session code.
- Use a Data Connection Tool.
How Client Tools Work
- Agent triggers tool during conversation.
- Ultravox sends tool invocation to your client.
- Your client code executes the tool logic.
- Client sends result back to Ultravox.
- Agent continues conversation with the tool result.
Client Tool Advantages
✅ UI integration: Direct access to update interface elements
✅ Low latency: No network round trip to your servers
✅ Client-side data: Access to local storage, camera, microphone
✅ Real-time updates: Immediate visual feedback
Client Tool Implementation
Error Handling
Return error information in the response:
Data Connection Tools
A third option combines benefits of both: Data Connection tools run on your server but communicate via websocket, enabling both server-side logic and real-time capabilities.
Data connections are like another participant in your call. Like the client, they can receive tool invocation messages and can send back tool result messages. Implementation lives in your websocket server and can be used regardless of the call medium used.
Data connection tools are ideal for:
- Long-running operations
- Real-time data streaming
- Complex server operations that need immediate feedback
Choosing the Right Tool Type
Use HTTP Tools When:
- Accessing databases or external APIs
- Processing sensitive data
- Performing server-side calculations
- Sending emails or notifications
- Working with telephony (Twilio, etc.)
- Need authentication with external services
Use Client Tools When:
- Updating user interface elements
- Accessing client device features (camera, microphone)
- Performing client-side validation
- Managing local application state
- Need immediate visual feedback
- Working with WebRTC calls primarily
Use Data Connection Tools When:
- Need both server logic and real-time feedback
- Handling long-running operations
- Streaming real-time data
- Complex workflows requiring immediate updates
Call Medium Compatibility
Tool Type | WebRTC | Websocket | Telephony |
---|---|---|---|
HTTP | ✅ | ✅ | ✅ |
Client | ✅ | ✅ | ❌ |
Data Connection | ✅ | ✅ | ✅ |
Authentication
HTTP Tools: Full authentication support including API keys, tokens, and custom headers. See Tools Authentication →
Client Tools: No built-in authentication - handle security in your client application.
Data Connection Tools: Authentication handled via websocket connection setup.