Set up a working environment
Python interpreters in DataSpell
To execute notebooks or Python code in DataSpell, you need to configure at least one virtual environment based on a Python interpreter. You can use a system interpreter that is available with your Python installation. You can also create a Virtualenv, pipenv, Poetry, or conda virtual environment. A virtual environment consists of a base interpreter and the installed packages.
With DataSpell, you can also configure interpreters to execute your notebooks on remote managed Jupyter servers by using SSH, or WSL (only for Windows).
When you attach a directory to the workspace or open a project, the IDE creates a virtual environment for you. In most cases, it is a conda environment based on your Anaconda installation. You can select any other Python environment on your machine or create a new one.
To preview a configured Python environment, modify its settings, and update packages, press Ctrl+Alt+S and go to .
Configuring an existing virtual environment
At any time, you can switch the virtual environment either by using the Python Interpreter selector or in Settings.
Switch the virtual environment using the Python Interpreter selector
The Python Interpreter selector is located on the status bar. It is the most convenient and quickest way to switch the Python interpreter. Just click it and select the target interpreter:
Switch the virtual environment in the IDE settings
Press Ctrl+Alt+S to open the IDE settings and then select .
If you want to change the Python interpreter for a specific project or workspace directory, select it. Otherwise, select workspace.
Click the drop-down and select the desired Python interpreter:
If it's not on the list, click Show All. Then select the desired interpreter in the left pane and click OK.
When DataSpell stops supporting any of the outdated Python versions, the corresponding Python interpreter is marked as unsupported.
Use Anaconda CLI to switch conda environments
In the Terminal window, run the
lscommand in the <CONDA_HOME> /envs directory (for example, /Users/jetbrains/.conda/envs) and select the target environment.Navigate to the bin directory of your anaconda installation (for example, anaconda3/bin).
Execute the
conda activate <env name>command (for example,conda activate my-conda-env).
Modify a virtual environment
Press Ctrl+Alt+S to open the IDE settings and then select .
Expand the list of the available interpreters and click Show All.
You can modify the path to the Python executable in the Interpreter path field.
When the Associate this virtual environment with the current project checkbox is enabled, the interpeter is available only in the current DataSpell project.
To change the interpreter name, select the target interpreter and click
.
The Python interpreter name specified in the Name field, becomes visible in the list of available interpreters. Click OK to apply the changes.
Remove a virtual environment
If you no longer need a Python interpreter for a project, you can remove it from the project settings.
Do one of the following:
Press Ctrl+Alt+S to open the IDE settings and then select .
Click the Python Interpreter selector and choose Interpreter Settings.
Expand the list of the available interpreters and click Show All.
Choose the interpreter that you want to remove and click
.
Creating a new virtual environment
Configuring local virtual environments
To configure a local virtual environment follow one of the procedures below:
Configure a system interpreter
Ensure that you have downloaded and installed Python on your computer.
- Installing Python on Windows from Microsoft Store
If you are on Windows, you can download Python from the Microsoft Store and install it as a Python interpreter. Once the Python application is downloaded from the Microsoft Store, it becomes available in the list of the Python executables.
Note that interpreters added from the Microsoft Store installations come with some limitations. Because of restrictions on Microsoft Store apps, Python scripts may not have full write access to shared locations such as
TEMPand the registry.
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select Add Local Interpreter.
In the left-hand pane of the Add Python Interpreter dialog, select System Interpreter.
In the Interpreter drop-down, select one of the Python interpreters that have been installed in your system, or click
and in the Select Python Interpreter dialog that opens, choose the desired Python executable.
You will need admin privileges to install, remove, and upgrade packages for the system interpreter. When attempting to install an interpreter package through an intention action, you might receive the following error message:
As prompted, consider using a virtual environment for your project.
Click OK to complete the task.
For more information, refer to Configure a system interpreter.
Create a virtualenv environment
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select Add Local Interpreter.
In the left-hand pane of the Add Python Interpreter dialog, select Virtualenv Environment.
The following actions depend on whether you want to create a new virtual environment or to use an existing one.
- New virtual environment
Specify the location of the new virtual environment in the Location field, or click
and browse for the desired location in your file system. The directory for the new virtual environment should be empty.Choose the base interpreter from the list, or click
and find the desired Python executable in your file system.Select the Inherit global site-packages checkbox if you want all packages installed in the global Python on your machine to be added to the virtual environment you're going to create. This checkbox corresponds to the
--system-site-packagesoption of the virtualenv tool.
- Existing virtual environment
Choose the desired interpreter from the list.
If the desired interpreter is not on the list, click
, and then browse for the desired Python executable (for example, venv/bin/python on macOS or venv\Scripts\python.exe on Windows).
The selected virtual environment will be reused for the current project.
Click OK to complete the task.
For more information, refer to Configure a virtual environment.
Create a conda environment
Ensure that Anaconda or Miniconda is downloaded and installed on your computer, and you're aware of a path to its executable file.
For more information, refer to the installation instructions.
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select Add Local Interpreter.
In the left-hand pane of the Add Python Interpreter dialog, select Conda Environment.
The following actions depend on whether you want to create a new conda environment or to use an existing one.
- New conda environment
Select the Python version from the list.
Normally, DataSpell will detect conda installation.
Otherwise, specify the location of the conda executable, or click
to browse for it.Specify the environment name.
- Existing conda environment
Select the environment from the list.
The selected conda environment will be reused for the current project.
Click OK to complete the task.
For more information, refer to Configure a conda virtual environment.
Create a pipenv environment
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select Add Local Interpreter.
In the left-hand pane of the Add Python Interpreter dialog, select Pipenv Environment.
Choose the base interpreter from the list, or click
and find the desired Python executable in your file system.If your project contains Pipfile, you can choose whether you want to install the packages listed in it by enabling or disabling the Install packages from Pipfile checkbox. By default, the checkbox is enabled.
If you have added the base binary directory to your
PATHenvironmental variable, you don't need to set any additional options: the path to the pipenv executable will be autodetected.If the pipenv executable is not found, follow the pipenv installation procedure to discover the executable path, and then specify it in the dialog.
Click OK to complete the task.
When you have set the pipenv virtual environment as a Python interpreter, all available packages are added from the source defined in Pipfile. The packages are installed, removed, and updated in the list of the packages through pipenv rather than through pip.
For more information, refer to Configure a pipenv environment.
Create a Poetry environment
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select Add Local Interpreter.
In the left-hand pane of the Add Python Interpreter dialog, select Poetry Environment.
The following actions depend on whether you want to create a new Poetry environment or to use an existing one.
- New Poetry environment
Select Poetry Environment.
Choose the base interpreter from the list, or click
and find the desired Python executable in your file system.If your project contains pyproject.toml, you can choose whether you want to install the packages listed in it by enabling or disabling the Install packages from pyproject.toml checkbox. By default, the checkbox is enabled.
If DataSpell doesn't detect the poetry executable, specify the following path in the dialog, replacing
jetbrainswith your username:/Users/jetbrains/Library/Application Support/pypoetry/venv/bin/poetryC:\Users\jetbrains\AppData\Roaming\pypoetry\venv\Scripts\poetry.exe/home/jetbrains/.local/bin/poetry
- Existing Poetry environment
Make sure that the project directory contains a pyproject.toml file.
Select Existing environment. Then expand the Interpreter list and choose the desired interpreter.
If the desired interpreter is not on the list, click
, and then browse for the Python executable within the previously configured Poetry environment.
The selected Poetry environment will be reused for the current project.
Click OK to complete the task.
For more information, refer to Configure a Poetry environment.
Configuring remote virtual environments
When a remote Python interpreter is added, at first the DataSpell helpers are copied to the remote host. DataSpell helpers are needed to run remotely the packaging tasks, debugger, tests and other DataSpell features.
Next, the skeletons for binary libraries are generated and copied locally. Also, all the Python library sources are collected from the Python paths on a remote host and copied locally along with the generated skeletons. Storing skeletons and all Python library sources locally is required for resolve and completion to work correctly.
DataSpell checks the remote helpers version on every remote run, so if you update your DataSpell version, the new helpers will be uploaded automatically, and you don't need to recreate remote interpreter.
Configure a WSL interpreter
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select On WSL.
Wait until DataSpell detects Linux on your machine and completes introspection. Click Next to proceed:
In the left-hand pane of the dialog, select the type of the WSL interpreter you want to create: Virtual Environment, Conda Environment, or System Interpreter.
For a system interpreter, just provide the path to the Python executable in the selected Linux distribution.
For virtual and conda environments, you can provide a path to a Python executable of an existing environment in the selected Linux distribution or create a new environment based on the specified Python.
Once done, the new interpreter will be added to your project, and the default mnt mappings will be set.
For more information, refer to Configure an interpreter using WSL.
Configure an interpreter using SSH
Ensure that there is an SSH server running on a remote host, since DataSpell runs remote interpreters via ssh-sessions.
Do one of the following:
Press Ctrl+Alt+S to open Settings and go to . Click the Add Interpreter link next to the list of the available interpreters.
Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.
In the context menu of an attached directory, select Interpreter.
Select On SSH.
Select an option to create a new SSH connection, then specify server information (host, port, and username).
Alternatively, you can select Existing and choose any available SSH configuration from the list. To create a new SSH configuration, follow the steps below:
- Creating an SSH configuration
Click
next to the list of configurations:
Click
, disable the Visible only for this project checkbox, and fill in the required fields:
Once done, the newly created SSH configuration will appear in the list of available configurations. Click Next to proceed:
In the next dialog window, provide the authentication details to connect to the target server.
Select Password or Key pair (OpenSSH or PuTTY) and enter your password or passphrase. If Key pair (OpenSSH or PuTTY) is selected, specify:
Private key: location of the file with a private key
Passphrase: similar to a password, it serves to encrypt the private key.
Click Next to proceed.
Wait until DataSpell completes the introspection of the SSH server.
In the next dialog, select a type of Python environment to configure on the SSH server.
You can create a new virtual environment or сonda environment, select an existing one, or use a system interpreter.
Select the Inherit global site-packages checkbox if you want all packages installed in the global Python on your machine to be added to the virtual environment you're going to create. This checkbox corresponds to the
--system-site-packagesoption of the virtualenv tool.If you need to execute your Python code on the SSH server as a sudo user, enable the Execute code with root privileges via sudo checkbox.
DataSpell can automatically synchronize files with the remote host. To do that, select the Automatically upload files to remote host checkbox, and specify the root directory for synchronization in Root directory. If the directory doesn't exist, it will be created.
When this option is enabled, the attached directories that have been configured to use the SSH interpreter are uploaded to the root directory on the remote host. Then they are synced upon any of the following events:
Applying changes to files
Switching to this interpreter
IDE start-up
You can exclude files that do need to be synchronized. To do that, specify the glob patterns in the Excluded paths field. By default, the field contains some patterns for directories and files that are not required for running a remote interpreter.
Click Create to complete adding the interpreter.
For more information, refer to Configure an interpreter using SSH.
Manage packages
The reason for creating various Conda environments based on the same Anaconda installation is obvious - you can install specific packages for each environment and use them for specific tasks and projects.
You can install packages for an environment using pip and the Conda package manager. You can do it using the IDE UI in the environment settings or in the Terminal window.
Install packages using the environment settings
In the Python interpreter selector, select Interpreter Settings.
Select the target environment from the Python Interpreter list.
Click + to add a new package. Click the Conda package manager button (
) to manage packages from the Conda repository. Otherwise, DataSpell will be using pip.Type a package name in the search field and locate the target package. If needed, specify a package version. Then press Install
Close the window on the task completion. If needed, press the Manage Repositories button and add a custom repository from which you install packages.
Install packages in the Terminal window
Open the Terminal window.
To install a package from pip, execute:
pip install <package name>(for example,pip install plotly).To install packages from the Conda repository:
Navigate to the bin directory of your anaconda installation (for example, anaconda3/bin)
Execute the following command:
conda install <package name>(for example,conda install tensorflow).
Sometimes, you want to upgrade packages in your environment. The quickest way is to use the environment settings.
Upgrade packages
In the Python interpreter selector, select Interpreter Settings.
Mind the
in the Latest version column. Select the corresponding package and click the Upgrade button ().