Connect to BigQuery
Enable the Database Tools and SQL plugin
This functionality relies on the Database Tools and SQL plugin, which is bundled and enabled in PyCharm by default. If the relevant features aren't available, make sure that you didn't disable the plugin.
Press Control+Alt+S to open the IDE settings and then select .
Open the Installed tab, find the Database Tools and SQL plugin, and select the checkbox next to the plugin name.
Official documentation
For full information about BigQuery, refer to the official documentation.
BigQuery is a REST-based web service that allows you to run complex analytical SQL-based queries under large data sets.
To connect to BigQuery, you can use the following approaches:
Google user account
When you use authorization with the Google user account, you need to receive the authorization code in a web browser.
Connect to BigQuery
To connect to the database, create a data source that will store your connection details. You can do this using one of the following ways:
From the main menu, navigate to and select BigQuery.
In the Database tool window ( ) , click the New icon (
) in the toolbar. Navigate to Data Source and select BigQuery.
Check if there is a Download missing driver files link at the bottom of the connection settings area. Click this link to download drivers that are required to interact with a database. For a direct download link, refer to the JetBrains JDBC drivers page.
You can find the downloaded JDBC drivers in the PyCharm configuration directory.
The IDE does not include bundled drivers in order to have a smaller size of the installation package and to keep driver versions up-to-date for each IDE version.
You can also use your drivers for the database instead of the provided ones. For more information about connecting to a database with your driver, refer to Add a user driver to an existing connection.
If there is no Download missing driver files link, then you already have the required drivers.
From the Authentication list, select Google User Account.
In the Project ID field, type the project ID.
Usually, it is a part of the service account email that goes after the at sign (
@). For example,bigqueryproject-322409. For the project ID's format, refer to the official instructions on creating a service account.From the Authorization Code Required dialog, cut the URL, paste it into the address bar of your web browser and press Enter to follow the URL.
Authorize access to your BigQuery application in your Google account.
Copy the authorization code received from Google, paste it in the Authorization Code Required dialog, and click OK.
Ensure that the connection to the database can be established using the provided details. To do that, click the Test Connection link at the bottom of the connection details area.
In case of any connection issues, refer to the Cannot connect to a database page.
(Optional) By default, only the default project and dataset are introspected and available to work with. If you also want to work with other projects and datasets, in the Schemas tab, select them for the introspection.
Click OK to create the data source.
Find your new data source in the Database tool window (Alt+1) .
For more information about the Database tool window, see the corresponding reference topic.
For more information about working with database objects in PyCharm, refer to Database objects.
To write and run queries, open the default query console by clicking the data source and pressing F4.
To view and edit data of a database object, open Data editor and viewer by double-clicking the object.
Application default credentials
For authorization, the BigQuery driver uses data from a special credentials file — a service account key file. By default, the path to the file is ~/.config/gcloud/application_default_credentials.json.
You can set a custom location for the credentials file by using the GOOGLE_APPLICATION_CREDENTIALS environment variable. Alternatively, set the path to the credentials file in the Service account key file field.
To generate the service account key file for the connection, you need to create a Google Cloud Platform service account.
Step 1. Obtain credentials file
Create a service account by following the official instructions.
On the Grant this service account access to the project step in the wizard, select roles for this service account.
For example, for read-only access, select BigQuery Data Viewer, BigQuery Job User, and BigQuery User from the BigQuery menu. Alternatively, select for access to all resources within the project.
For more information about roles and permissions, refer to Predefined roles and permissions at cloud.google.com.
Generate and download the service account key file by following the official instructions.
Step 2. Connect to BigQuery
To connect to the database, create a data source that will store your connection details. You can do this using one of the following ways:
From the main menu, navigate to and select BigQuery.
In the Database tool window ( ) , click the New icon (
) in the toolbar. Navigate to Data Source and select BigQuery.
Check if there is a Download missing driver files link at the bottom of the connection settings area. Click this link to download drivers that are required to interact with a database. For a direct download link, refer to the JetBrains JDBC drivers page.
You can find the downloaded JDBC drivers in the PyCharm configuration directory.
The IDE does not include bundled drivers in order to have a smaller size of the installation package and to keep driver versions up-to-date for each IDE version.
You can also use your drivers for the database instead of the provided ones. For more information about connecting to a database with your driver, refer to Add a user driver to an existing connection.
If there is no Download missing driver files link, then you already have the required drivers.
From the Authentication list, select Application Default Credentials.
In the Service account key file field, type the path to the JSON service account key file that you obtained on Step 1. Obtain credentials.
Ensure that the connection to the database can be established using the provided details. To do that, click the Test Connection link at the bottom of the connection details area.
In case of any connection issues, refer to the Cannot connect to a database page.
(Optional) By default, only the default project and dataset are introspected and available to work with. If you also want to work with other projects and datasets, in the Schemas tab, select them for the introspection.
Click OK to create the data source.
Find your new data source in the Database tool window (Alt+1) .
For more information about the Database tool window, see the corresponding reference topic.
For more information about working with database objects in PyCharm, refer to Database objects.
To write and run queries, open the default query console by clicking the data source and pressing F4.
To view and edit data of a database object, open Data editor and viewer by double-clicking the object.
Google service account
For authorization, the BigQuery driver uses data from a special credentials file — a service account key file. By default, the path to the file is ~/.config/gcloud/application_default_credentials.json.
You can set a custom location for the credentials file by using the GOOGLE_APPLICATION_CREDENTIALS environment variable. Alternatively, set the path to the credentials file in the Key file field.
To generate the service account key file for the connection, you need to create a Google Cloud Platform service account.
Step 1. Obtain credentials file
Create a service account by following the official instructions.
On the Grant this service account access to the project step in the wizard, select roles for this service account.
For example, for read-only access, select BigQuery Data Viewer, BigQuery Job User, and BigQuery User from the BigQuery menu. Alternatively, select for access to all resources within the project.
For more information about roles and permissions, refer to Predefined roles and permissions at cloud.google.com.
Generate and download the service account key file by following the official instructions.
Step 2. Connect to BigQuery
To connect to the database, create a data source that will store your connection details. You can do this using one of the following ways:
From the main menu, navigate to and select BigQuery.
In the Database tool window ( ) , click the New icon (
) in the toolbar. Navigate to Data Source and select BigQuery.
Check if there is a Download missing driver files link at the bottom of the connection settings area. Click this link to download drivers that are required to interact with a database. For a direct download link, refer to the JetBrains JDBC drivers page.
You can find the downloaded JDBC drivers in the PyCharm configuration directory.
The IDE does not include bundled drivers in order to have a smaller size of the installation package and to keep driver versions up-to-date for each IDE version.
You can also use your drivers for the database instead of the provided ones. For more information about connecting to a database with your driver, refer to Add a user driver to an existing connection.
If there is no Download missing driver files link, then you already have the required drivers.
From the Authentication list, select Google Service Account.
In the Service account email field, type the service account's name.
You can find the service account's name as Email on the Service accounts page () of the Google Cloud Platform. For more information about creating a service account for the name's format, refer to the official instructions. The service account's name should look like this:
intellij-ide@bigqueryproject-322409.iam.gserviceaccount.com.In the Project ID field, type the project ID.
Usually, it is a part of the service account email that goes after the at sign (
@). For example,bigqueryproject-322409. For the project ID's format, refer to the official instructions on creating a service account.In the Key file field, type the path to the JSON service account key file that you obtained on Step 1. Obtain credentials.
Ensure that the connection to the database can be established using the provided details. To do that, click the Test Connection link at the bottom of the connection details area.
In case of any connection issues, refer to the Cannot connect to a database page.
(Optional) By default, only the default project and dataset are introspected and available to work with. If you also want to work with other projects and datasets, in the Schemas tab, select them for the introspection.
Click OK to create the data source.
Find your new data source in the Database tool window (Alt+1) .
For more information about the Database tool window, see the corresponding reference topic.
For more information about working with database objects in PyCharm, refer to Database objects.
To write and run queries, open the default query console by clicking the data source and pressing F4.
To view and edit data of a database object, open Data editor and viewer by double-clicking the object.
Access and refresh tokens (OAuth 2.0)
To connect to the BigQuery database by using the Access and Refresh Tokens authentication method, the following credentials are required: client ID and client secret, refresh and access tokens.
When you use OAuth 2.0 for authentication, your users are authenticated after they agree to terms that are presented to them on a user consent screen. The OAuth consent screen is a dialog that displays a notification about who requests access to the user's data and a type of this data.
Before you generate a client ID and a client secret, you need to configure the OAuth consent screen.
Step 1. Obtain credentials
Create OAuth consent screen by following the official instructions.
For more information about user consent, refer to the User consent section at support.google.com
Get a client ID and a client secret by following the official instructions.
For the URI, use the following one:
https://developers.google.com/oauthplaygroundFind your client ID and client secret in the OAuth client created window. Alternatively, you can click the created OAuth client ID configuration and get your credentials here.
Having a client ID and a client secret, you can generate authorization code and tokens. To do that, perform the following steps:
Open the OAuth 2.0 Playground in a separate browser tab.
Click the OAuth 2.0 Configuration icon, select the Use your own OAuth credentials checkbox.
Paste your client ID and client secret in OAuth Client ID and OAuth Client secret fields.
Click Close in the OAuth 2.0 Configuration window.
In the wizard on Step 1. Select & authorize APIs, select .
Click Authorize APIs.
Follow the wizard in a browser and give permissions to the application.
On Step 2 Exchange authorization code for tokens, click Exchange authorization code for tokens.
Copy received refresh and access tokens.
Step 2. Connect to BigQuery
To connect to the database, create a data source that will store your connection details. You can do this using one of the following ways:
From the main menu, navigate to and select BigQuery.
In the Database tool window ( ) , click the New icon (
) in the toolbar. Navigate to Data Source and select BigQuery.
Check if there is a Download missing driver files link at the bottom of the connection settings area. Click this link to download drivers that are required to interact with a database. For a direct download link, refer to the JetBrains JDBC drivers page.
You can find the downloaded JDBC drivers in the PyCharm configuration directory.
The IDE does not include bundled drivers in order to have a smaller size of the installation package and to keep driver versions up-to-date for each IDE version.
You can also use your drivers for the database instead of the provided ones. For more information about connecting to a database with your driver, refer to Add a user driver to an existing connection.
If there is no Download missing driver files link, then you already have the required drivers.
From the Authentication list, select Access and Refresh Tokens.
In the Project ID field, type the project ID.
Usually, it is a part of the service account email that goes after the at sign (
@). For example,bigqueryproject-322409. For the project ID's format, refer to the official instructions on creating a service account.In the Access token field, paste your access token.
In the Refresh token field, paste your refresh token.
In the Client ID field, paste your client ID.
In the Client secret field, paste your client secret.
Ensure that the connection to the database can be established using the provided details. To do that, click the Test Connection link at the bottom of the connection details area.
In case of any connection issues, refer to the Cannot connect to a database page.
(Optional) By default, only the default project and dataset are introspected and available to work with. If you also want to work with other projects and datasets, in the Schemas tab, select them for the introspection.
Click OK to create the data source.
Find your new data source in the Database tool window (Alt+1) .
For more information about the Database tool window, see the corresponding reference topic.
For more information about working with database objects in PyCharm, refer to Database objects.
To write and run queries, open the default query console by clicking the data source and pressing F4.
To view and edit data of a database object, open Data editor and viewer by double-clicking the object.
