Use cloud storage data sources

Attach an Amazon S3 bucket

1. Open the notebook.

2. Go to Main menu | Tools | Attached data or click the Attached data icon on the left-hand sidebar.

3. Click New connection and select New cloud storage.

4. In the New connection dialog, select Amazon S3.

5. In the New Amazon S3 cloud storage connection dialog, fill in the following fields:

   • Display name: to specify the name for this data source in your system

   • AWS access key and AWS secret access key: to access your AWS account (details here)

   • Region: to specify your AWS region

   • Amazon Bucket name: to specify the name of the bucket you want to mount

   • Custom options: to specify additional parameters. See the example below

   • Endpoint URL: to specify the website of the bucket you want to mount

6. (Optional) Click Test connection to make sure the provided parameters are correct.

7. Click Create data source to finish the procedure.

Attach a Google Cloud Storage bucket

1. Open the notebook.

2. Go to Main menu | Tools | Attached data or click the Attached data icon on the left-hand sidebar.

3. Click New connection and select New cloud storage.

4. In the New connection dialog, select Google cloud storage.

5. In the New Google cloud storage connection dialog, fill in the following fields:

   • Display name: to specify the name for this bucket in your system

   • GCS Bucket name: to specify the name of the bucket you want to mount (details here).

   • GCS key file content: to enter the content of the Google service account key file (.json format).

6. (Optional) Click Test connection to make sure the provided parameters are correct.

7. Click Create data source to finish the procedure.

Attach an SMB/CIFS folder

1. Open the notebook.

2. Go to Main menu | Tools | Attached data or click the Attached data icon on the left-hand sidebar.

3. Click New connection and select New cloud storage.

4. In the New connection dialog, select SMB/CIFS.

5. In the SMB/CIFS cloud storage connection dialog, fill in the following fields:

   • Display name: to specify the name for this data source in your system

   • Host: to specify the SMB server hostname

   • Port: to specify the SMB port

   • Username and Password: to specify the user credentials

   • Domain (Optional): to specify the SMB server domain

   • Custom options: to specify additional parameters

6. (Optional) Click Test connection to make sure the provided parameters are correct.

7. Click Create data source to finish the procedure.

Add a cloud storage data source to a workspace

This procedure adds a cloud storage data source to your workspace resources without attaching it to any notebook automatically. Such data sources are later available for all notebooks of the respective workspace.

1. Select Cloud storage from the menu on the left. This will open the Cloud storages list.

2. Click New connection in the upper-right corner of the list.

3. In the New cloud storage connection, select the cloud storage type.

4. Proceed by following the steps described for the respective cloud storage type.

Attach an existing cloud storage data source

Cloud storage data sources created from the Home page or for a specific notebook are available across the entire workspace and can be attached to any notebook from it.

1. Open the notebook.

2. Select Main menu | Tools | Attached data.

3. In the Attached data tool, click Select data to attach and select the required data source from the list.

Manage attached cloud storage data sources on the notebook level

1. Go to Main menu | Tools | Attached data or click the Attached data icon on the left-hand sidebar. You will see all your data sources including attached buckets.

2. To change the access type, click the pencil icon next to the bucket type and select your option (Read-only access or Read-write access).

3. To add file to the bucket storage, click Upload files, select your option, and complete the procedure accordingly. You csn find more details about uploading or creating file in Attached files.

4. Click the ellipsis to use more options:

   • Open detail view: to view and manage the stored files

   • Edit cloud storage: to edit the connection parameters for the storage

   • Check connection: to test the connection to the storage (for Amazon S3 only)

   • Connect using boto3: to connect to the bucket storage using boto3 in the notebook code (for Amazon S3 only)

   • Detach cloud storage: to detach the storage from the notebook

Manage cloud storage data sources on the workspace level

1. Select Cloud storage from the menu on the left. This will open the list of your bucket data sources.

2. To edit the parameters of a data source, click the respective list item and make your changes in the Edit [data_source_name] connection dialog.

3. To rename a data source, right-click the list item, select Rename from the menu, and provide a new name.

4. To clone the item to other workspaces:

   1. Select Select to other workspaces

   2. In the Clone [data_source_name] to other workspaces dialog, expand the Workspaces dropdown list.

   3. Select the workspaces where you want to clone the data source and click anywhere outside the dropdown.

   4. Click the Clone button. This will close the dialog, followed by a success notification.

5. To delete a data source, right-click the respective list item and select Delete from the menu.

Enable SSE-C for S3 data sources

The Custom_options is a field used for optional parameters when creating an Amazon S3 data source. Below are two example of how it can be used.

• To enable SSE-C for S3 data sources, specify the following in the Custom_options: In the Custom_options field, specify the following:

  use_sse=c:/path/to/keys/file

  where:

  /path/to/keys/file is the file that contain keys. Make sure permissions are 600.

• (For Datalore Enterprise only) To provide access based on a role associated with that of an EC2 instance profile, add public_bucket=0,iam_role into the Custom_options field.