Overview

ℹ️

Dust does not synchronize or store any of your BigQuery data. We only store metadata, such as datasets and table names.

You can connect Dust to your BigQuery data warehouse to enable your agents to perform Table queries on your BigQuery tables.

Dust agents are able to use the output from these queries to answer quantitative questions:

The agents can also leverage the results to create visualizations, such as charts:

The "Tools inspection" button allows to view the reasoning and SQL query used by the agent to retrieve the data:

Setting up the Connection

Key BigQuery Concepts

Let's review some essential concepts:

• Project: A container for your BigQuery resources
• Dataset: A container for tables and other objects (similar to a schema in other databases)
• Table: A structure that stores data in rows and columns
• Service Account: An account for programmatic access to BigQuery
• IAM Role: A collection of permissions that can be assigned to users and service accounts

Step-by-Step Guide

Setup on Google Cloud's side

📘

We recommend creating a dedicated service account for Dust.

Create a service account

1. Go to the IAM & Admin > Service Accounts section of the Google Cloud Console
2. Click "Create Service Account"
3. Enter a name (e.g., "dust-service-account")
4. Click "Create and Continue"

Assign roles to the service account

The service account needs at least two roles:

• roles/bigquery.user (to run queries)
• roles/bigquery.dataViewer (to access data)
• Note: if you use certains types of underlying storage, such as Delta Lake tables in GCS, you may need additional roles.

1. In the IAM section, find your service account
2. Click the edit (pencil) icon
3. Click "Add another role"
4. Add both required roles
5. Click "Save"

Create and download credentials

1. Go back to the service account details
2. Go to the "Keys" tab
3. Click "Add Key" > "Create new key"
4. Choose JSON format
5. Click "Create"

The key file will download automatically. Keep it secure as it provides access to your BigQuery data.

Setup on Dust's side

1. Navigate to Dust's "Connection Admin" (in the Knowledge tab)
2. Click on "Add Connections" and select BigQuery
3. Upload your service account key file
4. [If you have tables in several locations] Pick the location where the tables you want to use on Dust are located
5. Select the datasets and tables you want to make available in Dust