LLMs have tremendous knowledge about the world, but they don’t have all the up-to-date specifics about your organization, your products, or other relevant information you might want to provide to your AI voice agents.

RAG is a common technique for grounding agents in the relevant information for your use case.

​
Examples of Knowledge Sources

Let’s consider some of the content that might be useful to serve some popular use cases:

  • Product Documentation → user guides, FAQs, troubleshooting steps.
  • Onboarding Materials → getting started guides, best practices, transcripts from training videos.

​
Adding RAG to Ultravox

As we saw in the Using Tools guide, tools provide power-ups for your agents. To use RAG with an Ultravox agent, it’s as simple as creating a tool and instructing the agent on how to use the tool.

​
The Easy Way

Ultravox provides the corpus service for RAG.

1

Create a Corpus

Use the Create Corpus endpoint. Give your new corpus a name and (optional) description. This returns a corpusId.

2

Create a Source

Add a website to crawl using Create Corpus Source. Each source is given a unique sourceId. We will crawl the URL(s) and ingest all the content.

3

Query the Corpus

After everything is loaded, try some queries using the Query Corpus endpoint.

4

Use the queryCorpus Tool

Give the built-in queryCorpus tool. to your agents and provide the corpusId. For example, if we wanted to create a voice agent to answer questions about Seattle, we could provide the tool like this:

{
  "systemPrompt": "Use the queryCorpus tool to answer questions about Seattle.",
  "selectedTools": [
    {
      "toolName": "queryCorpus", 
      "parameterOverrides": {
        "corpus_id": "<your_corpus_id_here>",
        "max_results": 5
      }
    }
  ]
}

​
The Other Way

Let’s assume we have already stored our product documentation in a vector database and can search that content at https://foo.bar/lookupProductInfo.

Here’s how we might create a tool for our Ultravox agent to use:

Example: Adding a RAG tool
{
  "systemPrompt": "You are a helpful assistant. You have a tool called 'lookupProductInfo' that you must use to find answers.",
  "model": "fixie-ai/ultravox",
  "selectedTools": [
    {
      "temporaryTool": {
        "modelToolName": "lookupProductInfo",
        "description": "Searches official product documentation using semantic similarity to find relevant information. Use this tool to look up specific product features, specifications, limitations, pricing, or support information. The tool returns the most relevant text chunks from the documentation.",
        "dynamicParameters": [
          {
            "name": "query",
            "location": "PARAMETER_LOCATION_BODY",
            "schema": {
              "description": "A specific, focused search query to find relevant product information",
              "type": "string"
            },
            "required": true
          }
        ],
        "http": {
          "baseUrlPattern": "https://foo.bar/lookupProductInfo",
          "httpMethod": "POST"
        }
      }
    }
  ]
}
Using ToolsSDKs