DataSpell 2023.1 Help

Set up a working environment

Python interpreters in DataSpell

To execute notebooks or Python code in DataSpell, you need to configure at least one virtual environment based on a Python interpreter. You can use a system interpreter that is available with your Python installation. You can also create a Virtualenv, Pipenv, Poetry, or Conda virtual environment. A virtual environment consists of a base interpreter and the installed packages.

With DataSpell, you can also configure interpreters to execute your notebooks on remote managed Jupyter servers by using SSH, or WSL (only for Windows).

Python interpreters

When you attach a directory to the workspace or open a project, the IDE creates a virtual environment for you. In most cases, it is a conda environment based on your Anaconda installation. You can select any other Python environment on your machine or create a new one.

To preview a configured Python environment, modify its settings, and update packages, press Control+Alt+S and go to Project: workspace | Python Interpreter.

Configured interpreters

Configuring an existing virtual environment

At any time, you can switch the virtual environment either by using the Python Interpreter selector or in Settings.

Switch the virtual environment using the Python Interpreter selector

  • The Python Interpreter selector is located on the status bar. It is the most convenient and quickest way to switch the Python interpreter. Just click it and select the target interpreter:

    Select an interpreter

Switch the virtual environment in the IDE settings

  1. Press Control+Alt+S to open the IDE settings and select Project: workspace | Python Interpreter.

  2. If you want to change the Python interpreter for a specific workspace directory, select it. Otherwise, select workspace.

  3. Expand the list of the available interpreters and click Show All.

    Show all available interpreters
  4. Select the target interpreter.

    When DataSpell stops supporting any of the outdated Python versions, the corresponding Python interpreter is marked as unsupported.

Use Anaconda CLI to switch conda environments

  1. In the Terminal window, run the ls command in the <CONDA_HOME> /envs directory (for example, /Users/jetbrains/.conda/envs) and select the target environment.

  2. Navigate to the bin directory of your anaconda installation (for example, anaconda3/bin).

  3. Execute the conda activate <env name> command (for example, conda activate my-conda-env).

Modify a virtual environment

  1. Press Control+Alt+S to open the IDE settings and select Project <project name> | Python Interpreter.

  2. Expand the list of the available interpreters and click Show All.

    Show all available interpreters
  3. You can modify the path to the Python executable in the Interpreter path field.

    To change the interpreter name, select the target interpreter and click App actions edit.

    Edit icon

    The Python interpreter name specified in the Name field, becomes visible in the list of available interpreters. Click OK to apply the changes.

    Edit interpreter settings

Remove a virtual environment

If you no longer need a Python interpreter for a project, you can remove it from the project settings.

  1. Do one of the following:

    • Press Control+Alt+S to open the IDE settings and select Project: workspace | Python Interpreter.

    • Click the Python Interpreter selector and choose Interpreter Settings.

  2. Expand the list of the available interpreters and click Show All.

    Show all available interpreters
  3. Choose the interpreter that you want to remove and click Remove Interpreter.

    Remove a Python interpreter

Creating a new virtual environment

Configuring local virtual environments

To configure a local virtual environment follow one of the procedures below:

Configure a system interpreter

  1. Ensure that you have downloaded and installed Python on your computer.

    Installing Python on Windows from Microsoft Store

    If you are on Windows, you can download Python from the Microsoft Store and install it as a Python interpreter. Once the Python application is downloaded from the Microsoft Store, it becomes available in the list of the Python executables.

    Python interpreter installed from the Windows store
    Note that interpreters added from the Microsoft Store installations come with some limitations. Because of restrictions on Microsoft Store apps, Python scripts may not have full write access to shared locations such as TEMP and the registry.

  2. Do one of the following:

    • Press Control+Alt+S to open Settings and go to Project: workspace | Python Interpreter. Click the Add Interpreter link next to the list of the available interpreters.

    • Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.

    • In the context menu of an attached directory, select Interpreter.

  3. Select Add Local Interpreter.

  4. In the left-hand pane of the Add Python Interpreter dialog, select System Interpreter.

    Adding a system interpreter
  5. In the Interpreter field, type the fully-qualified path to the required interpreter executable, or click the Browse button and in the Select Python Interpreter dialog that opens, choose the desired Python executable.

    Selecting the Python executable

    You will need admin privileges to install, remove, and upgrade packages for the system interpreter. When attempting to install an interpreter package through an intention action, you might receive the following error message:

    System Interpreter warning message
    As prompted, consider using a virtual environment for your project.

  6. Click OK to complete the task.

For more information, see Configure a system interpreter.

Create a virtualenv environment

  1. Do one of the following:

    • Press Control+Alt+S to open Settings and go to Project: workspace | Python Interpreter. Click the Add Interpreter link next to the list of the available interpreters.

    • Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.

    • In the context menu of an attached directory, select Interpreter.

  2. Select Add Local Interpreter.

  3. In the left-hand pane of the Add Python Interpreter dialog, select Virtualenv Environment.

    Creating a virtual environment
  4. The following actions depend on whether you want to create a new virtual environment or to use an existing one.

    New virtual environment
    • Specify the location of the new virtual environment in the Location field, or click Virtual environment location and browse for the desired location in your file system. The directory for the new virtual environment should be empty.

    • Choose the base interpreter from the list, or click Choose the base interpreter and find the desired Python executable in your file system.

    • Select the Inherit global site-packages checkbox if you want all packages installed in the global Python on your machine to be added to the virtual environment you're going to create. This checkbox corresponds to the --system-site-packages option of the virtualenv tool.

    Existing virtual environment
    • Choose the desired interpreter from the list.

    • If the desired interpreter is not on the list, click App general ellipsis, and then browse for the desired Python executable (for example, venv/bin/python on macOS or venv\Scripts\python.exe on Windows).

    The selected virtual environment will be reused for the current project.

  5. Click OK to complete the task.

For more information, see Configure a virtual environment.

Create a conda environment

  1. Ensure that Anaconda or Miniconda is downloaded and installed on your computer, and you're aware of a path to its executable file.

    Refer to the installation instructions for more details.

  2. Do one of the following:

    • Press Control+Alt+S to open Settings and go to Project: workspace | Python Interpreter. Click the Add Interpreter link next to the list of the available interpreters.

    • Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.

    • In the context menu of an attached directory, select Interpreter.

  3. Select Add Local Interpreter.

  4. In the left-hand pane of the Add Python Interpreter dialog, select Conda Environment.

    New Conda environment
  5. The following actions depend on whether you want to create a new conda environment or to use an existing one.

    New conda environment
    • Select the Python version from the list.

    • Normally, DataSpell will detect conda installation.

      Otherwise, specify the location of the conda executable, or click Conda executable location to browse for it.

    • Specify the environment name.

    Existing conda environment
    • Choose the desired environment from the list.

    The selected conda environment will be reused for the current project.

  6. Click OK to complete the task.

For more information, see Configure a conda virtual environment.

Create a pipenv environment

  1. Do one of the following:

    • Press Control+Alt+S to open Settings and go to Project: workspace | Python Interpreter. Click the Add Interpreter link next to the list of the available interpreters.

    • Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.

    • In the context menu of an attached directory, select Interpreter.

  2. Select Add Local Interpreter.

  3. In the left-hand pane of the Add Python Interpreter dialog, select Pipenv Environment.

    Adding a Pipenv environment
  4. Choose the base interpreter from the list, or click Choose the base interpreter and find the desired Python executable in your file system.

  5. If you have added the base binary directory to your PATH environmental variable, you don't need to set any additional options: the path to the pipenv executable will be autodetected.

    If the pipenv executable is not found, follow the pipenv installation procedure to discover the executable path, and then paste it in the Pipenv executable field.

  6. Click OK to complete the task.

Once all the steps are done, the new pipenv environment is set for your project and the packages listed in the Pipfile are installed.

If you open a project with a Pipfile file added but no any interpreter configured, DataSpell offers you to use Pipenv environment.

Pipenv inspection

If you select this option, DataSpell sets pipenv for you automatically. Alternatively, you can click Configure Python interpreter to follow the standard workflow.

Similarly, when you open a project with a Pipfile file in DataSpell for the very first time, for example, by checking it out from the Version Control, the Pipenv virtual environment will be configured automatically.

When you have set the Pipenv virtual environment as a Python interpreter, all available packages are added from the source defined in Pipfile . The packages are installed, removed, and updated in the list of the packages through pipenv rather than through pip.

For more information, see Configure a Pipenv environment.

Create a Poetry environment

  1. Do one of the following:

    • Press Control+Alt+S to open Settings and go to Project: workspace | Python Interpreter. Click the Add Interpreter link next to the list of the available interpreters.

    • Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.

    • In the context menu of an attached directory, select Interpreter.

  2. Select Add Local Interpreter.

  3. In the left-hand pane of the Add Python Interpreter dialog, select Poetry Environment.

    creating a poetry environment
  4. The following actions depend on whether you want to create a new Poetry environment or to use an existing one.

    New Poetry environment
    • Select Poetry Environment.

    • Choose the base interpreter from the list, or click Choose the base interpreter and find the desired Python executable in your file system.

    • If DataSpell doesn't detect the poetry executable, specify the following path in the Poetry executable field, replacing jetbrains with your username:

      /Users/jetbrains/Library/Application Support/pypoetry/venv/bin/poetry
      C:\Users\jetbrains\AppData\Roaming\pypoetry\venv\Scripts\poetry.exe
      /home/jetbrains/.local/bin/poetry
    Existing Poetry environment
    • Make sure that the project directory contains a pyproject.toml file.

    • Select Existing environment. Then expand the Interpreter list and choose the desired interpreter.

    • If the desired interpreter is not on the list, clickApp general ellipsis, and then browse for the Python executable within the previously configured Poetry environment.

    The selected Poetry environment will be reused for the current project.

  5. Click OK to complete the task.

For more information, see Configure a Poetry environment.

Configuring remote virtual environments

When a remote Python interpreter is added, at first the DataSpell helpers are copied to the remote host. DataSpell helpers are needed to run remotely the packaging tasks, debugger, tests and other DataSpell features.

Next, the skeletons for binary libraries are generated and copied locally. Also, all the Python library sources are collected from the Python paths on a remote host and copied locally along with the generated skeletons. Storing skeletons and all Python library sources locally is required for resolve and completion to work correctly.

DataSpell checks remote helpers version on every remote run, so if you update your DataSpell version, the new helpers will be uploaded automatically, and you don't need to recreate remote interpreter. SFTP support is required for copying helpers to the server.

Configure a WSL interpreter

  1. Do one of the following:

    • Press Control+Alt+S to open Settings and go to Project: workspace | Python Interpreter. Click the Add Interpreter link next to the list of the available interpreters.

    • Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.

    • In the context menu of an attached directory, select Interpreter.

  2. Select On WSL.

  3. Wait until DataSpell detects Linux on your machine and completes introspection. Press Next to proceed:

    Detecting Linux
  4. In the left-hand pane of the dialog, select the type of the WSL interpreter you want to create: Virtual Environment, Conda Environment, or System Interpreter.

    New WSL interpreter

    For a system interpreter, just provide the path to the Python executable in the selected Linux distribution.

    For virtual and conda environments, you can provide a path to a Python executable of an existing environment in the selected Linux distribution or create a new environment based on the specified Python.

Once done, the new interpreter will be added to your project, and the default mnt mappings will be set.

For more information, see Configure an interpreter using WSL.

Configure an interpreter using SSH

  1. Ensure that there is an SSH server running on a remote host, since DataSpell runs remote interpreters via ssh-sessions.

  2. Do one of the following:

    • Press Control+Alt+S to open Settings and go to Project: workspace | Python Interpreter. Click the Add Interpreter link next to the list of the available interpreters.

    • Click the Python Interpreter selector and choose Interpreter Settings. Click the Add Interpreter link next to the list of the available interpreters.

    • In the context menu of an attached directory, select Interpreter.

  3. Select On SSH.

  4. Select an option to create a new SSH connection, then specify server information (host, port, and username).

    adding an interpreter via SSH

    Alternatively, you can select Existing and choose any available SSH configuration from the list. To create a new SSH configuration, follow the steps below:

    Creating an SSH configuration
    • Click Browse button next to the list of configurations:

      Add new SSH configuration
    • Clicknew configuration, disable the Visible only for this project checkbox, and fill in the required fields:

      Adding new SSH configuration
    • Once done, the newly created SSH configuration will appear in the list of available configurations. Click Next to proceed:

      Connecting to SSH server
  5. In the next dialog window, provide the authentication details to connect to the target server.

    specifying authentication details

    Select Password or Key pair (OpenSSH or PuTTY) and enter your password or passphrase. If Key pair (OpenSSH or PuTTY) is selected, specify:

    • Private key: location of the file with a private key

    • Passphrase: similar to a password, it serves to encrypt the private key.

    Click Next to proceed.

  6. Wait until DataSpell completes the introspection of the SSH server.

    SSH server introspection
  7. In the next dialog, select a type of Python environment to configure on the SSH server.

    Selecting a Python environment

    You can create a new virtual environment, select an existing one, or use a system interpreter.

    • Select the Inherit global site-packages checkbox if you want all packages installed in the global Python on your machine to be added to the virtual environment you're going to create. This checkbox corresponds to the --system-site-packages option of the virtualenv tool.

    • DataSpell can automatically synchronize files with the remote host. To do that, select the Automatically upload files to remote host checkbox, and specify the root directory for synchronization in Root directory. If the directory doesn't exist, it will be created.

      When this option is enabled, the attached directories that have been configured to use the SSH interpreter are uploaded to the root directory on the remote host. Then they are synced upon any of the following events:

      • Applying changes to files

      • Switching to this interpreter

      • IDE start-up

    • You can exclude files that do need to be synchronized. To do that, specify the glob patterns in the Excluded paths field. By default, the field contains some patterns for directories and files that are not required for running a remote interpreter.

    Click Create to complete adding the interpreter.

For more information, see Configure an interpreter using SSH.

Manage packages

The reason for creating various Conda environments based on the same Anaconda installation is obvious - you can install specific packages for each environment and use them for specific tasks and projects.

You can install packages for an environment using pip and the Conda package manager. You can do it using the IDE UI in the environment settings or in the Terminal window.

Install packages using the environment settings

  1. In the Python interpreter selector, select Interpreter Settings.

  2. Select the target environment from the Python Interpreter list.

  3. Click + to add a new package. Click the Conda package manager button (Conda package manager) to manage packages from the Conda repository. Otherwise, DataSpell will be using pip.

  4. Type a package name in the search field and locate the target package. If needed, specify a package version. Then press Install

    Install a package
  5. Close the window on the task completion. If needed, press the Manage Repositories button and add a custom repository from which you install packages.

Install packages in the Terminal window

  1. Open the Terminal window.

  2. To install a package from pip, execute: pip install <package name> (for example, pip install plotly).

  3. To install packages from the Conda repository:

    • Navigate to the bin directory of your anaconda installation (for example, anaconda3/bin)

    • Execute the following command: conda install <package name> (for example, conda install tensorflow).

Sometimes, you want to upgrade packages in your environment. The quickest way is to use the environment settings.

Upgrade packages

  1. In the Python interpreter selector, select Interpreter Settings.

  2. Mind the latest version is available in the Latest version column. Select the corresponding package and click the Upgrade button (upgrade the package).

    Install a package using the Conda package manager
Last modified: 03 March 2023