Be in the Know

All Ultravox customers automatically receive email updates. Create a free account to start building with the the best voice AI and to stay in the loop.

​
Latest Update

​
2025-01-03 - Upgrades to Model, Tools Performance, and Docs

We’re kicking off 2025 with exciting improvements and community-driven developments build on our leading platform for voice AI (5 cents per minute, highest quality voices, low latency, and SDKs for all major languages).

​
What’s Hot

  1. Model Upgrades and Performance Improvements
  2. New Dashboard Features and Documentation
  3. Community Hack Day Results
Model Upgrades and Performance Improvements
  • Model Upgrade → Upgraded Ultravox 0.4.1 to run on the newest Llama 3.3 model, delivering significantly improved instruction following and tool usage capabilities. This new model is now the default in the Ultravox Realtime service and the HF model and model card are available.
  • Improved Tool Performance → Updated our vLLM tool parser with more lenient processing to better handle Llama’s tool calling patterns, resulting in more consistent performance. We fixed an issue with large tool calls being unstable.
  • More Improvements → Added vadSettings parameters for more control, new public hang-up tool, and added timeout for tools.
New Dashboard Features and Documentation
  • Call History → Introducing the Call History page at app.ultravox.ai - dive deep into detailed call analytics, including tool error tracking. This is just the first of many dashboard improvements coming your way.
Community Hack Day Results

Last week’s impromptu hack day brought our community together to share their vision for future voice agents.

Members were particularly interested in implementing features like knowledge lookup/RAG, human agent handoff, end-of-call transcript retrieval, Make.com integration, and Cal.com API calendar availability checking.

In response, we’ve released a new sample application that implements all these features as tools for the voice AI agent to call when needed (with end-of-call transcripts handled via webhooks). You can find the complete implementation on our GitHub repo, along with a detailed video walkthrough.

​
What’s Next

Dashboard Evolution → The new Call History page is just the beginning. We’re developing a no-code builder that will allow you to create and customize voice AI agents without writing any code. This will make it easier than ever to implement complex features like those showcased in our latest sample application, while still maintaining the flexibility for developers who prefer to code their own solutions.

Other Improvements → Building on our recent Llama 3.3 integration, we’re focusing on further enhancing tool usage reliability and model performance. We are also working on providing automatic conversation summaries on call end.

​
Prior Updates

​
2024-12-10 - WebSockets and More

We’re excited to announce several new features and improvements to the Ultravox platform, including new integration options, model support, and infrastructure updates.

​
What’s Hot

  1. New Features: WebSockets, Telnyx, and Plivo
  2. SDK and Other Improvements
  3. Docs Updates (Including “News” and “Deprecation” pages)
New Features: WebSockets, Telnyx, and Plivo
  • WebSockets: You can now integrate on the server side via WebSockets.
  • Telnyx & Plivo: New telephony integrations for Telnyx and Plivo are now available in addition to our existing support for Twilio. Check the docs.
SDK and Other Improvements
  • SDK Updates: New client version tracking allows you to set an arbitrary value that is tied to calls (retrieve with GET on /calls endpoint).
  • Enhanced Call Transcripts: For more accurate transcripts, you can now pass in the languageHint at call creation time to help guide the model.
  • Bug Fix: Fixed an issue where errant connections could affect proper call termination.
Docs Updates

​
What’s Not

We have one active deprecation: initiator will be deleted at the end of the month. This has been replaced with firstSpeaker. Not using initiator? You can ignore this. Otherwise, check out the migration guide.

​
What’s Next

We’re actively working on several exciting features and improvements:

Language and Voice Expansion: Finnish language support is up next. We welcome your input on additional language requirements. Pop into #feature-requests to let us know which voices you’d like to see added to our roadmap!

Infrastructure and Compliance: EU datacenter planning in progress and GDPR compliance implementation is underway. If these initiatives are important to your operations, please schedule a meeting using my calendar link to discuss your specific requirements.

Platform Enhancements: Enhanced call visibility in the dashboard to help you more easily monitor usage and debug issues. We are working on adding a low latency RAG service and are continuing to work on additional optimizations for transcripts and function calling.

Holiday Schedule Update: Our team will be operating on a reduced schedule between Christmas and New Year’s. While we’ll maintain system health and provide emergency support, response times on Discord and email may be longer than usual during this period. Rest assured that all critical systems will remain fully monitored and supported by our on-call team.

​
2024-11-14 - Ultravox v0.4.1 Release

We’re excited to announce the release of Ultravox v0.4.1, which brings significant improvements to the model you’re already using. We’ve also added a new web console and have enabled your agents to start conversations via text.

​
What’s Hot

  1. Ultravox v0.4.1: Six new languages, higher quality, new variants.
  2. Ultravox Console: Your web playground and place to manage your account.
  3. initialOutputMedium: Agents can now start conversations via text.
Ultravox v0.4.1

Expanded Language Coverage

  • Added 6 new languages (Chinese, Dutch, Hindi, Swedish, Turkish, and Ukrainian).
  • Total of 15 languages are now supported by the model.

Enhanced Performance

  • Improved BLEU scores across all languages.
  • Now achieving average BLEU score of 38.97 (vs. GPT-4’s 40.35).

New Model Variants

  • Added Mistral NeMo variant.
  • Updated Llama variants (8B model and 70B model) trained on 8xH100s.

The 0.4.1 updates are now live as the default on our managed Ultravox Realtime APIs. Pricing starts at just 5 cents per minute (⅓ the cost of GPT-4o).

The model weights are available on Hugging Face, and you can find detailed release notes on our GitHub repository. If you need on-premises support for end-to-end data sovereignty, please reach out via email or set-up a call to discuss.

For insights into our roadmap and strategy and to see a live demonstration of the new model in action, check out our latest blog post.

Ultravox Console

There’s now a web-based console application at https://app.ultravox.ai that you can use for keeping track of usage, generating API keys, managing your subscription, and playing around with different voices and system prompts. The console is a work-in-progress so don’t hesitate to reach out with requests for new features!

initialOutputMedium

This new property can be set at call creation to have the agent’s initial output be text (voice remains the default). This enables text-based scenarios and can be used with the SDK’s setOutputMedium() to toggle between text and voice. Check out the Create Call docs for more info.

​
What’s Next

We’re already working on the next major release of Ultravox with even more exciting features. Your feedback has been invaluable in shaping our development, and we’d love to hear your thoughts on these latest improvements.

​
2024-10-18 - Call Stages and Client-Implemented Tools

We’re thrilled to share the latest updates we’ve made to the Ultravox APIs. All of these enhancements have been made due to feedback from our community. Please keep the feedback coming! If there’s anything we can do to make things work better for you, don’t hesitate to get in touch!

​
What’s Hot

  1. Call Stages: Dynamic, Multi-Stage Conversations
  2. Client-Implemented Tools: Implement Tools in Your App
  3. More Improvements: setOutputMedium + Webhooks
Call Stages: Dynamic, Multi-Stage Conversations
  • What’s new: Stages enable more complex and nuanced agent interactions, giving you fine-grained control over the conversation flow.
  • Why it matters: Each stage can have a new system prompt, a different set of tools, a new voice, an updated conversation history, and more.
  • Where to use: Stages are designed for complex conversational flows like data gathering (job applications, medical intake forms, applying for a mortgage) or context switching (customer support escalation, triaging IT issues).
  • Where to start: Check our docs for the details on how to get started.
Client-Implemented Tools: Implement Tools in Your App
  • What’s new: In our previous update we added support for tools. Those were “server” tools and required you to implement the logic on a server and expose things via a URL. Client-implemented tools enable putting all the logic in your client application and are still called by your agent.
  • Why it matters: Enable dynamic UI or other interactivity in your app without having to rely on putting all the logic on a server.
  • Learn more: Visit our SDK page for more info.
More Improvements: setOutputMedium + Webhooks
  • setOutputMedium(): Added to our SDKs to give you more control over how your agents respond. Allows toggling the agent’s output between text and voice. See the docs.
  • Webhooks: Ultravox now has webhooks for two key events: call.started and call.ended. This opens up new opportunities for triggering external processes when calls start/end, logging call data in real-time to your own systems, or integrating Ultravox more deeply with other workflows.

​
What’s Not

  1. Breaking Change: SDK SessionState
  2. Deprecation Notice: initiator on new call creation

We recognize that breaking changes and deprecation notices are not fun and we try to avoid them when possible. However, we are committed to having our APIs and SDKs work better and be as clear as possible. That means we will inevitably need to revisit some choices early on.

Breaking Change: SDK SessionState

In the latest versions of our client SDKs, the UltravoxSession joinCall() method no longer returns an object. UltravoxSession now exposes properties for status and transcripts.

Deprecation: initiator is now firstSpeaker

This change is being made because firstSpeaker is more descriptive of what is happening when the call starts. For example, if you are making an outbound call, you expect the user to answer the call and be the first to speak. When creating a new call, you should start using firstSpeaker and choose either “FIRST_SPEAKER_AGENT” (the default) or “FIRST_SPEAKER_USER” (for outbound calls) as the value.

initiator will be removed at the end of November, 2024.

​
What’s Next

We are working on a new version of the Ultravox model that will add new language support for Chinese, Dutch, Hindi, Swedish, Turkish, and Ukrainian. We are also creating a web-based application for the Ultravox service (sign-up, API key management, usage tracking) and are adding a Swift client SDK for iOS developers.

If you have any suggestions for new features or improvements, please don’t hesitate to reach out.

​
2024-09-30 - 70B and Tools

We are continuing to get great feedback (thank you!) and have been working to add more capabilities.

​
What’s Hot

  1. Ultravox 70B: Our Smartest Model Yet
  2. Tools Support: Give Your Agents New Abilities
  3. Expanded SDK Coverage: Flutter, Kotlin, and Python
1. Ultravox 70B: Brains Meet Brawn
  • Why it matters: More complex reasoning, better understanding
  • How to use: It’s now the default! Just use ‘fixie-ai/ultravox’ in your API calls
  • Pro tip: Need the 8B version? Use ‘fixie-ai/ultravox-8B’ (Note: Tools not supported)
  • Model weights: Available on HuggingFace
2. Tools: Your AI’s New Superpowers
  • What’s new: Durable tools (create once, use often) and Temporary tools (perfect for iterating)
  • Where to start: Check our docs for the how-to
  • See it in action: Try our tools demo on our website
3. New SDKs: Code Your Way
  • New additions: Flutter, Kotlin, and Python join our JavaScript SDK
  • Cool features: Debug Messages, mic/speaker controls
  • Learn more: Visit our SDK page for details

​
In Case You Missed It

  • Price drop: Now just $0.05/minute (cheaper than coffee, and way more talkative!)
  • Voice cloning: Create customized voices for your agents
  • Conversation continuity: Because why start over?

​
What’s Next?

You tell us! We’re all ears for your suggestions to make Ultravox even better for you.

​
2024-09-04 - Price Reduction, Resume Calls, Voice Cloning

  • Our managed Ultravox APIs are getting much cheaper. We’re decreasing our price to $0.05/min. That’s full-on, real-time, speech-to-speech voice chat. We think this is the highest quality, lowest cost system out there.
  • We continue to offer 30 minutes of free usage to try it out for yourself. If you’d like to continue using our managed APIs after that, you’ll need to set up a Stripe subscription. You can now do that by accessing the billingUrl from the new /accounts API.
  • We’ve added the ability to seamlessly continue a prior conversation. This is as simple as passing in a priorCallId parameter when starting a call.
  • We’ve added support for Voice Cloning.
  • We released a new version of the Ultravox Model, v0.4.
  • Tool support is coming very soon!