Use bucket data sources
Datalore provides interface for mounting S3 or Google Cloud Storage buckets directly inside your notebooks. Here are the main benefits of using such data sources:
You extend your storage as you do not have to upload to notebooks those files that you store in buckets.
You can grant read or write access to the files stored in buckets.
Attach buckets to your notebook
Open the notebook.
Select . This opens a view to the right of your editor.
Open the New datasource dialog:
Click Add S3 datasource.
Expand the list by clicking the arrow next to and select Add GCS datasource.
In the New datasource dialog, fill in the fields and click Save and close.
Once mounted, the bucket datasource is automatically attached to the notebook. Your further actions:
Copy the datasource path to access it in your code.
Click (rw) next to the datasource path to change the access type to read-only (ro).
Click the vertical ellipsis icon to edit or remove the datasource. For S3 buckets, you can also check the connection or choose connection using boto3.
Unmount the bucket.
Buckets attached to a specific notebook are saved to the file system and can be attached to any notebook.
Attach an existing bucket datasource
Open the notebook.
Select . This opens a view to the right of your editor.
In the Attached datasources view, click Attach existing datasources and select the required datasource from the list.
Enable SSE-C for the Datalore S3 datasource
In the custom_options field, specify the following:
where:
/path/to/keys/file is the file that contain keys. Make sure permissions are 600.