Tools in Ultravox
Temporary vs. Durable Tools
Ultravox supports two types of tools: temporary and durable. There is much more information below but there are a few things to consider right upfront:
- Creation → Temporary tools are created in the request body when a new call is created. Durable tools are created using the Ultravox REST API.
- No Functional Difference → There is no functional difference within the context of an Ultravox call between the two tool types.
- Iteration Speed → Temporary tools are great when you are building out a new application and need to iterate.
- Reuse & Collaboration → Durable tools are best when you have things dialed in and want to reuse tools across applications and/or work with a team and want to divide ownership of tools from the rest of your app.
Temporary Tools
Temporary tools are created each time you create a new Call and exist exclusively within the context of that call. (Temporary tools aren’t visible in the List Tools response for example.)
Iteration is faster when using temporary tools because you don’t have to create/update/delete tools as you build out your application. You can simply adjust the JSON in the request body and start a new call.
Creating & Using Temporary Tools
Temporary tools are defined and passed in the request body of the Create Call
endpoint. They are available during the current call.
Durable Tools
In addition to temporary tools, Ultravox supports the creation of durable tools. There is no functional difference between durable and temporary tools within the context of a call.
Durable tools are persisted and can be reused across calls or applications. They shine once you have things dialed in, when you want to share tools across multiple applications, or if you have split responsibilities on the team.
Creating Durable Tools
You create durable tools either by uploading an OpenAPI spec or via the request body in the Create Tool
endpoint. Your OpenAPI spec must be either json
or yaml
format.
The /tools
endpoint in the Ultravox API is for working with durable tools.
Using Durable Tools
To use a durable tool in a call, set the toolName or toolId field instead of temporaryTool. For example:
Tool Authentication
Ultravox has rich support for tools auth. When creating a tool, you must specify what is required for successful auth to the backend service. Three methods for passing API keys are supported and are used when creating the tool:
- Query Parameter → The API key will be passed via the query string. The name of the parameter must be provided when the tool is created.
- Header → The API key will be passed via a custom header. The name of the header must be provided when the tool is created.
- HTTP Authentication → The API key will be passed via the HTTP Authentication header. The name of the scheme (e.g.
Bearer
) must be provided when the tool is created.
You then pass in the key(s) in the authTokens
property of selectedTools
when creating a call.
Tool Parameters
Tool parameters define what gets passed in to your backend service when the tool is called. When creating a tool, parameters are defined as one of three types:
- Dynamic → The model will choose which values to pass. These are the parameters you’d use for a single-generation LLM API. Can be overridden (see below).
- Static → Value is known when the tool is defined and is unconditionally set on invocations. Not exposed to or set by the model.
- Automatic → Like “Static”, except that the value may not be known when the tool is defined but will instead be populated by the system when the tool is invoked.
Dynamic Parameters
Dynamic parameters will have their values set by the model. Creating a dynamic parameter on a tool looks like this:
Parameter Overrides
You can choose to set static values for dynamic parameters when you start a call. The model won’t see any parameters that you override. When creating a call simply pass in the overrides with each tool:
Parameter overrides don’t make sense for temporary tools. Instead of overriding a dynamic parameter, use a static parameter instead.
Static Parameters
If you have parameters that are known at the time you create the tool, static parameters can be used.
Automatic Parameters
Automatic parameters are used when you don’t want the model to specify the value and you don’t know the value when the tool is created. The primary use case for automatic parameters today is for using the call_id
that is generated for the current call and then passing it as a unique identifier to your tool. Can also be used to get the current conversation history.
Additional Information
Debugging
The Ultravox SDK enables viewing debug messages for any active call. These messages include tool calls. You can see a live demo of this on our website (make sure to toggle “Debug View” on at the bottom).
Tool Definition Schema
The definition
object in the tool creation and update requests follows the BaseToolDefinition schema. Here’s a breakdown of its main components:
description
(string): A clear, concise description of what the tool does.dynamicParameters
(array, optional): List of parameters that can be set by the AI model when using the tool. Each parameter is an object containining:name
(string): The name of the parameter.location
(string): Where the parameter is used (e.g., “PARAMETER_LOCATION_QUERY”, “PARAMETER_LOCATION_PATH”).schema
(object): JSON Schema definition of the parameter. This typically includes things like type, description, enum values, format, other restrictions, etc.required
(boolean): Whether the parameter is required.
staticParameters
(array, optional): List of parameters that are always set to a known, fixed value when the tool is used. These are unconditionally added when the tool is invoked. These parameters are not exposed to or set by the model. Example: you use an API for various things but want to track which requests come from your Ultravox app so you always appendutm=ultravox
to the query parameters.automaticParameters
(array, optional): Additional parameters automatically set by the system. Used when the value is not known when the tool is created but that will be known when the tool is called. Example: you want to use the uniquecall_id
from ultravox as a key in your backend and you have the tool includecall_id
in the request body when your tool’s API is called.requirements
(object, optional): Any specific requirements for using the tool. Currently this is used for security (e.g. API keys or HTTP Auth).http
(object): Details for invoking the tool via HTTP.baseUrlPattern
(string): The base URL pattern for the tool, possibly with placeholders for path parameters.httpMethod
(string): The HTTP method for the tool (e.g., “GET”, “POST”).
Best Practices for Creating Tools
-
Clear Naming: Choose a descriptive and unique name for your tool that clearly indicates its function.
-
Detailed Description: Provide a comprehensive description of what the tool does, including any important details about its usage or limitations. This and the name will help the model decide when and how to use your tool.
-
Precise Parameters: Define your dynamic parameters carefully, ensuring that the AI model has all the information it needs to use the tool effectively.
-
Error Handling: Consider how your tool will handle errors or unexpected inputs, and document this behavior in the tool’s description.
-
Iterate Faster: Use temporary tools when you are building your application. Persist durable tools in the system when things have stabilized.
-
Version Control: When updating tools, consider creating a new version (e.g., “stock_price_v2”) rather than modifying the existing tool. This allows testing of the new tool before impacting new calls made with the prior version of the tool.
-
Security: Be mindful of security when creating tools, especially when they interact with external APIs. Use appropriate authentication methods and avoid exposing sensitive information.
-
Testing: Thoroughly test your tools before deploying them in production conversations to ensure they function as expected.