BigQuery

Enable the Database Tools and SQL plugin

This functionality relies on the Database Tools and SQL plugin, which is bundled and enabled in PyCharm by default. If the relevant features are not available, make sure that you did not disable the plugin.

1. Press Ctrl+Alt+S to open settings and then select Plugins.

2. Open the Installed tab, find the Database Tools and SQL plugin, and select the checkbox next to the plugin name.

Official documentation

• For full information about BigQuery, refer to the official documentation.

Connect to a BigQuery database

1. To connect to the database, create a data source that will store your connection details. You can do this using one of the following ways:

   • In the main menu, go to File | New | Data Source and select BigQuery.

   • In the Database tool window ( View | Tool Windows | Database) , click the New icon () in the toolbar. Navigate to Data Source and select BigQuery.

   

2. In the General tab of Data Sources and Drivers dialog right pane, specify the driver and connection type.

   1. In the Driver list, leave the default driver option, unless another driver is required for your connection.

   2. From the Connection type list, select the connection type depending on the connection details that you have:

      • default: connect by using Host, Port, Project ID, Default dataset, and URL.

      • URL only: connect by using only the URL.

      For the URL only connection type, the JDBC URL that you enter is used and as is, including the database credentials.

      For the other connection types, the JDBC URL is broken down into connection details. You can either specify them separately and use the automatically generated URL, or you can enter the URL directly in the corresponding field.

   

3. Check if there is a Download missing driver files link at the bottom of the connection settings area. Click this link to download drivers that are required to interact with a database. For a direct download link, refer to the JetBrains JDBC drivers page.

   Location for the downloaded JDBC drivers is the PyCharm configuration directory.

   

   You can also use your drivers for the database instead of the provided ones. For more information about connecting to a database with your driver, refer to Add a user driver to an existing connection.

   If there is no Download missing driver files link, then you already have the required drivers.

4. Specify the database connection details. Alternatively, paste the JDBC URL in the URL field.

   1. In the Host field, type your server address.

   2. In the Port field, type the port of BigQuery. The default port is 443.

   3. From the Authentication list, select the authentication method that you want to use to authenticate the connection. The following options are available:

      • Google Service Account: by using a Google service account email address and a full path to a private key file for the service account.

        For a data source creation tutorial, refer to BigQuery data source: Google service account.

      • Google User Account: by providing your Google user account credentials to connect to the server. This method requires authentication each time you connect to the database.

        For a data source creation tutorial, refer to BigQuery data source: Google user account.

      • Access and Refresh Tokens: by using access or refresh tokens that have already been generated from the Google Authorization Server. The refresh token also requires you client ID and your client secret. For more information about tokens, refer to Using OAuth 2.0 to Access Google APIs at developers.google.com.

        For a data source creation tutorial, refer to BigQuery data source: Access and refresh tokens (OAuth 2.0).

      • Application Default Credentials: by using credentials obtained through Application Default Credentials (ADC). For more information about ADC and how to obtain them, refer to Authenticating as a service account at cloud.google.com.

        For a data source creation tutorial, refer to BigQuery data source: Application Default Credentials.

      • No auth: authentication is not required.

   4. In the Project ID field, enter a unique identifier for your project.

   5. Depending on the selected authentication method, specify the following connection settings:

      Google Service Account

      • Service account email: your Google service account email address.

      • Key file: the full path to a private key file for the service account.

      Access and Refresh Tokens

      • Access token: your access token.

      • Refresh token: your refresh token.

      • Client ID: your client ID.

      • Client secret: your client secret.

      Application Default Credentials

      • Service account key file: the full path to a private key file for the service account.

   6. In the Default dataset field, specify the name of the default dataset that you want to connect to.

   7. In the URL field, PyCharm generates the JDBC URL automatically using the values of other connection settings.

      If you need to use a JDBC URL with certain additional settings, paste it in the URL field. The general URL to use is as follows:

      • Format: jdbc:bigquery://https://<host_address>:<port_number>;DefaultDataset=<dataset_name>

      • Example: jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;DefaultDataset=myDataset

      For more information about the URL format, refer to the BigQuery official documentation.

   

   1. From the Authentication list, select the authentication method that you want to use to authenticate the connection. The following options are available:

      • Google Service Account: by using a Google service account email address and a full path to a private key file for the service account.

        For a data source creation tutorial, refer to BigQuery data source: Google service account.

      • Google User Account: by providing your Google user account credentials to connect to the server. This method requires authentication each time you connect to the database.

        For a data source creation tutorial, refer to BigQuery data source: Google user account.

      • Access and Refresh Tokens: by using access or refresh tokens that have already been generated from the Google Authorization Server. The refresh token also requires you client ID and your client secret. For more information about tokens, refer to Using OAuth 2.0 to Access Google APIs at developers.google.com.

        For a data source creation tutorial, refer to BigQuery data source: Access and refresh tokens (OAuth 2.0).

      • Application Default Credentials: by using credentials obtained through Application Default Credentials (ADC). For more information about ADC and how to obtain them, refer to Authenticating as a service account at cloud.google.com.

        For a data source creation tutorial, refer to BigQuery data source: Application Default Credentials.

      • No auth: authentication is not required.

   2. In the Project ID field, enter a unique identifier for your project.

   3. Depending on the selected authentication method, specify the following connection settings:

      Google Service Account

      • Service account email: your Google service account email address.

      • Key file: the full path to a private key file for the service account.

      Access and Refresh Tokens

      • Access token: your access token.

      • Refresh token: your refresh token.

      • Client ID: your client ID.

      • Client secret: your client secret.

      Application Default Credentials

      • Service account key file: the full path to a private key file for the service account.

   4. In the URL field, PyCharm generates the JDBC URL automatically using the values of other connection settings.

      If you need to use a JDBC URL with certain additional settings, paste it in the URL field. The general URL to use is as follows:

      • Format: jdbc:bigquery://https://<host_address>:<port_number>;DefaultDataset=<dataset_name>

      • Example: jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;DefaultDataset=myDataset

      For more information about the URL format, refer to the BigQuery official documentation.

   

   For the reference information about connection settings and properties on the General and other tabs of Data Sources and Drivers dialog (Shift+Enter) , see Connection settings and DBMS-specific properties.

5. Ensure that the connection to the database can be established using the provided details. To do that, click the Test Connection link at the bottom of the connection details area.

   

   In case of any connection issues, refer to the Cannot connect to a database page.

6. (Optional) By default, only the default project and dataset are introspected and available to work with. If you also want to work with other projects and datasets, in the Schemas tab, select them for the introspection.

   

7. Click OK to create the data source.

8. Find your new data source in the Database tool window (⌘ 1) .

   • For more information about the Database tool window, see the corresponding reference topic.

   • For more information about working with database objects in PyCharm, refer to Database objects.

   • To write and run queries, open the default query console by clicking the data source and pressing F4.

   • To view and edit data of a database object, open Data editor and viewer by double-clicking the object.