LLMs have tremendous knowledge about the world, but they don’t have all the up-to-date specifics about your organization, your products, or other relevant information you might want to provide to your AI voice agents.

RAG is a common technique for grounding agents in the relevant information for your use case.

​
Examples of Knowledge Sources

Let’s consider some of the content that might be useful to serve some popular use cases:

​
Adding RAG to Ultravox

As we saw in the Using Tools guide, tools provide power-ups for your agents. To use RAG with an Ultravox agent, it’s as simple as creating a tool and instructing the agent on how to use the tool.

​
The Easy Way

Ultravox provides the corpus service for RAG.

1

Create a Corpus

Use the Create Corpus endpoint. Give your new corpus a name and (optional) description. This returns a corpusId.

2

Create a Source

Add a website to crawl using Create Corpus Source. Each source is given a unique sourceId. We will crawl the URL(s) and ingest all the content.

3

Query the Corpus

After everything is loaded, try some queries using the Query Corpus endpoint.

4

Use the queryCorpus Tool

Give the built-in queryCorpus tool. to your agents and provide the corpusId. For example, if we wanted to create a voice agent to answer questions about Seattle, we could provide the tool like this:

{
  "systemPrompt": "Use the queryCorpus tool to answer questions about Seattle.",
  "selectedTools": [
    {
      "toolName": "queryCorpus", 
      "parameterOverrides": {
        "corpus_id": "<your_corpus_id_here>",
        "max_results": 5
      }
    }
  ]
}

​
The Other Way

Let’s assume we have already stored our product documentation in a vector database and can search that content at https://foo.bar/lookupProductInfo.

Here’s how we might create a tool for our Ultravox agent to use:

Example: Adding a RAG tool
{
  "systemPrompt": "You are a helpful assistant. You have a tool called 'lookupProductInfo' that you must use to find answers.",
  "model": "fixie-ai/ultravox",
  "selectedTools": [
    {
      "temporaryTool": {
        "modelToolName": "lookupProductInfo",
        "description": "Searches official product documentation using semantic similarity to find relevant information. Use this tool to look up specific product features, specifications, limitations, pricing, or support information. The tool returns the most relevant text chunks from the documentation.",
        "dynamicParameters": [
          {
            "name": "query",
            "location": "PARAMETER_LOCATION_BODY",
            "schema": {
              "description": "A specific, focused search query to find relevant product information",
              "type": "string"
            },
            "required": true
          }
        ],
        "http": {
          "baseUrlPattern": "https://foo.bar/lookupProductInfo",
          "httpMethod": "POST"
        }
      }
    }
  ]
}

​
Add Static Files as Corpus Sources

You can use files as sources for any of your corpora. The process requires using the Create Corpus File Upload API to generate an upload URL.

Follow these steps:

1

Step 1: Request Upload URL

  • Use the Create Corpus File Upload API
  • Include the MIME type string in the request body
  • This returns the URL to use for upload and the unique ID for the document
  • URLs expire after 5 minutes. Request a new one if it expires before using it
The URL that is returned is tied to the provided MIME type. The same MIME type must be used during upload.
2

Step 2: Upload File

  • Use the presignedUrl from Step 1 to upload the document
  • Ensure the MIME type in the upload matches what was specified in Step 1

For example, if we requested an upload URL for a text file (MIME type text/plain):

FILE_PATH="/path/to/your/file"
UPLOAD_URL="https://storage.googleapis.com/fixie-ultravox-prod/..."

curl -X PUT \
  -H "Content-Type: text/plain" \
  --data-binary @"$FILE_PATH" \
  "$UPLOAD_URL"
3

Step 3: Create New Source with Uploaded Document

You can provide an array of Document IDs to bulk create a source.

​
Supported File Types

Here’s the information in a markdown table format:

File ExtensionType of FileMIME Type
docMicrosoft Word Documentapplication/msword
docxMicrosoft Word Open XML Documentapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
txtPlain Text Documenttext/plain
mdMarkdown Documenttext/markdown
pptMicrosoft PowerPoint Presentationapplication/vnd.ms-powerpoint
pptxMicrosoft PowerPoint Open XML Presentationapplication/vnd.openxmlformats-officedocument.presentationml.presentation
pdfPortable Document Formatapplication/pdf