BigQuery

Last updated: February 27, 2026

With Rally's BigQuery integration, enriched participant data — like company name, subscriber status, device type, customer tier, and custom properties — can automatically sync from BigQuery into Rally every 24 hours. This keeps your research panel up to date with your most current first-party product and customer data, without manual effort.

By configuring sync settings and property mappings, you can tailor the integration to bring in exactly the data your team needs to build targeted segments, recruit the right participants, and launch studies faster.

  • Benefits for Researchers: Syncing BigQuery data into Rally means you can build highly specific participant segments and recruit from them on demand, without relying on data science or engineering teams to pull lists.

  • Benefits for Ops and Data Teams: The integration allows you to leverage your existing BigQuery tables and data pipelines to keep Rally participant profiles fresh and accurate, using data that's already being maintained in your warehouse.

How does the BigQuery integration work?

Rally connects to your Google Cloud project and syncs data from a specified BigQuery table into Rally on a set schedule.

After connecting BigQuery, you configure sync settings and map BigQuery fields to properties in Rally. Once configured, Rally will automatically sync data every 24 hours. You can also trigger a manual sync at any time from Settings > Integrations.

Rally uses the Identifier Field (typically email address) to match BigQuery records to people in Rally. If a record from BigQuery is not found in Rally, Rally will either ignore it or create a new person, depending on your sync preferences.

The BigQuery integration is one-directional: BigQuery syncs data into Rally only. Data entered or updated in Rally will not be written back to BigQuery.

Required user permissions

Before setting up the BigQuery integration, make sure you have the correct permissions in both Rally and Google Cloud.

Rally user permissions

You'll need to be an Ops Manager, Developer/IT, or Admin role type in Rally.

Google Cloud permissions

Rally connects to BigQuery using a Google Cloud service account that you create and configure in your Google Cloud account. The service account will need the following IAM roles:

  • BigQuery Data Viewer — to read data from your BigQuery datasets or tables

  • BigQuery Job User — to run query jobs on your behalf

See the Preparation section next for step-by-step instructions on creating the service account before connecting.

Preparation

Before connecting BigQuery to Rally, you'll need to create a service account in your Google Cloud account and grant it the necessary permissions. This service account acts as a secure intermediary that manages data transfers between Rally and your BigQuery warehouse.

What is a service account?

A Google Cloud Platform (GCP) service account is a specialized account that applications use to access Google APIs on your behalf. When you create this service account, it will serve as a trusted intermediary, safely managing data syncs between BigQuery and Rally. See Google's documentation for more information.

Create a service account

The steps below describe how to create and configure a service account in Google Cloud. Note that these instructions may not always reflect the most recent changes to the GCP console.

1. Enable the BigQuery API

  1. In the Google Cloud Console, navigate to the Library page.

  2. Search for "BigQuery" and select the BigQuery API.

  3. Click Enable to activate the API.

2. Create a service account

  1. Go to IAM & Admin > Service Accounts.

  2. Click Create Service Account.

  3. Enter a Service Account Name and an optional Service Account ID.

  4. Click Create.

3. Grant permissions

  1. On the service account details page, click Grant this service account access to project.

  2. In the New members field, enter the Service Account ID you created.

  3. In the Role field, select BigQuery Job User.

  4. To allow Rally to read datasets from BigQuery for selection during setup, also select BigQuery Data Viewer.

    • If you prefer to grant access at the table level only, skip this step and follow the optional instructions in step 5 below instead.

  5. Click Save.

4. Create a service account key

  1. On the service account details page, go to the Keys tab.

  2. Click Add Key > Create new key.

  3. Select JSON as the key type.

  4. Click Create. A JSON file containing the service account key will be downloaded to your computer.

Store the JSON key file securely. Treat this file like a password — do not share it publicly or commit it to a version control system. When you upload it to Rally, we will encrypt and store it securely in our database.

5. Optional: Grant table-level permissions

If you chose not to grant BigQuery Data Viewer at the project level, you can instead grant access to a specific dataset and table by running the following queries in the BigQuery SQL Editor. Replace the placeholders with your actual values.

GRANT `roles/bigquery.dataViewer`
ON TABLE {{DATASET_NAME}}.{{TABLE_NAME}}
TO "serviceAccount:{{SERVICE_ACCOUNT_NAME}}";

GRANT `roles/bigquery.metadataViewer`
ON TABLE {{DATASET_NAME}}.{{TABLE_NAME}}
TO "serviceAccount:{{SERVICE_ACCOUNT_NAME}}";

Once you've created the service account and downloaded the JSON key file, you're ready to connect BigQuery to Rally.

Set up the BigQuery integration

1. Connect BigQuery to Rally

  1. In the top left corner of Rally, select your workspace name and select Settings.

  2. From the sidebar, select Integrations.

  3. To the right of BigQuery, select Connect.

  4. Enter your Google Cloud Project ID and the service account credentials (JSON key file) for the service account you've configured.

  5. Select Connect to establish the connection.

2. Link a BigQuery table

After connecting, you'll need to specify which BigQuery table Rally should sync from.

  1. Enter the following details about your BigQuery table:

    • Project — your Google Cloud project ID

    • Dataset — the BigQuery dataset name

    • Table — the specific table to sync from

  2. Select Save to link the table.

We recommend creating a dedicated table in BigQuery with only the fields relevant to Rally, rather than syncing directly from a primary warehouse table. This simplifies management and reduces risk.

3. Configure sync preferences

Carefully configure sync preferences to determine how Rally handles records during sync.

Create new people in Rally during sync

  • When enabled, BigQuery records that don't already exist in Rally will be created as new participants. Rally uses the Identifier Field (email) to check for existing records.

  • If enabled, select which Population new participants should be added to.

What should Rally do if a record exists in Rally but is missing in BigQuery?

  • Do nothing — Rally takes no action on the missing record.

  • Opt out missing people in Rally — People missing from BigQuery will be marked as Opted Out in Rally and cannot be contacted.

4. Map fields between BigQuery and Rally

Next, map BigQuery fields to Properties in Rally.

The Identifier Field is how Rally matches BigQuery records to people in Rally. Only Unique Properties (like email address) can be used as an identifier field.

Property Mapping

Rally currently supports mapping the following field types from BigQuery:

BigQuery field type

Rally property type

STRING

text

INTEGER / FLOAT / NUMERIC

number

BOOLEAN

checkbox

DATE / DATETIME / TIMESTAMP

date

RECORD (REPEATED)

multi-select

Field types must be compatible for a mapping to work. For example, you cannot map a BigQuery INTEGER field to a Rally date property. If a mapping isn't working as expected, check that the property type in Rally matches the corresponding field type in BigQuery. You can recreate a Rally property with the correct type and re-import existing data if needed.

We recommend creating the necessary Properties in Rally before running your first sync so that field mappings are ready to go. Properties can also be mapped and added over time; any new mappings will populate values for existing participants on the next sync.

FAQs

Can Rally write data back to BigQuery?

No, the BigQuery integration is one-directional. Data flows from BigQuery into Rally only.

How often does Rally sync with BigQuery?

Rally syncs automatically every 24 hours. You can also trigger a manual sync at any time from Settings > Integrations.

What happens if a participant's email in Rally is different from the one in BigQuery?

Rally matches on any email associated with a participant in Rally — primary or secondary. If a participant's BigQuery email matches a secondary email in Rally, the sync will still find and update that participant's profile. Their preferred primary email in Rally will not be overwritten.

Can I add new field mappings after the integration is already running?

Yes. New property mappings can be added at any time. When a new mapping is added, Rally will populate that property for all matching participants on the next sync using the data already in BigQuery.

Why are some fields showing as blank after a sync?

The most common cause is a field type mismatch between BigQuery and Rally. Check that the BigQuery field type and Rally property type are compatible (see the supported field type mappings table above). Also verify that the fields you're expecting exist in the linked BigQuery table. Fields must live in the same table, not across separate tables or derived from event data.

Why am I not seeing any records updated after a successful sync?

Rally will only update any new or changed property values during a sync. If there were no changes to the properties that are mapped to sync to Rally, then those records will not update within Rally.