Customize or update environment

You can build a custom Docker agent image on top of the default one to customize your environment (install some package from apt to be available in your notebooks, or set up a custom Python environment by pre-installing the required libraries).

Generate indexes to ensure the editor's proper work

After you create a new custom environment or update one of our environments, run the following command in the same file:

RUN /opt/datalore/build_code_insight_data.sh /[environment_path]

where [environment_path] is the path to the environment you created or updated.

If not performed, these indexes will be updated every time the agent starts, which slows down the editor.

Add the created environment file to the main datalore pod

Add the file to following directory:

/opt/datalore/configs/environment_info

Make sure you follow these instructions:

For a conda environment, specify the respective .yml file matching the envrionment name. For example, if you created a conda environment called somename, add an environment_somename.yml file.
For a pip environment, specify the respective .txt file matching the envrionment name. For example, if you created a pip environment called somename, add an requirements_somename.txt file.

Examples

Modify the pip/minimal environment

FROM jetbrains/datalore-agent:2023.5.1
COPY my_additional_packages.txt /tmp/my_additional_packages.txt
RUN /opt/python/envs/minimal/bin/python -m pip install -r my_additional_packages.txt
RUN /opt/datalore/build_code_insight_data.sh /opt/python/envs/minimal

Use the additional_packages.txt file as a list of packages and their versions as shown below:

tensorflow==2.3.1
uvicorn==0.12.2
fastapi==0.63.0

Create a new environment

Assuming the name of the new environment is myenv:

FROM jetbrains/datalore-agent:2023.5.1
RUN /opt/python/bin/python -m venv /opt/python/envs/myenv
RUN /opt/python/envs/myenv/bin/python -m pip install -r /tmp/requirements_myenv.txt
RUN /opt/datalore/build_code_insight_data.sh /opt/python/envs/myenv

Examples using custom environments

Custom Python environment with version 3.10: This custom environment sets Python 3.10 as the default version.
FROM jetbrains/datalore-agent:2023.5.1 USER root ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && apt-get install -y python3.10-venv && rm -rf /var/lib/apt/lists/* && apt-get clean USER datalore RUN /usr/bin/python3.10 -m venv /opt/python/envs/myenv RUN /opt/python/envs/myenv/bin/python -m pip install ipykernel==5.5.3 ipython==7.31.1 ipython_genutils==0.2.0 jedi==0.17.2 aiohttp==3.8.3 lets-plot pandas RUN /opt/datalore/build_code_insight_data.sh /opt/python/envs/myenv
Custom Python environment with version 3.11: This custom environment sets Python 3.11 as the default version.
FROM jetbrains/datalore-agent:2023.5.1 USER root RUN apt-get update && apt-get install -y python3.11 python3.11-venv && rm -rf /var/lib/apt/lists/* && apt-get clean USER datalore RUN /usr/bin/python3.11 -m venv /opt/python/envs/myenv RUN /opt/python/envs/myenv/bin/python -m pip install ipykernel==5.5.3 ipython==7.31.1 ipython_genutils==0.2.0 jedi==0.17.2 aiohttp==3.8.3 lets-plot pandas RUN /opt/datalore/build_code_insight_data.sh /opt/python/envs/myenv
Custom R environment without Anaconda: This custom environment sets a non-conda default environment with the R kernel.
At the moment this solution adds this environment to /opt/anaconda/envs. Therefore, this option will be offered as a conda-based environment when creating a notebook. Such a restriction will be removed in the next release.
FROM jetbrains/datalore-agent:2023.5.1 ENV CUSTOM_ENV_NAME myenv USER root RUN apt-get update && \ DEBIAN_FRONTEND=noninteractive apt-get install -y -q --no-install-recommends libzmq3-dev libcurl4-openssl-dev libssl-dev r-base make g++ libharfbuzz-dev libfribidi-dev libtiff-dev apt-file && \ Rscript -e "install.packages(c('repr', 'IRdisplay', 'IRkernel'), type = 'source')" && \ rm -rf /var/lib/apt/lists/* && apt-get clean RUN sudo chown -R datalore:datalore /home/datalore USER datalore RUN mkdir -p /opt/anaconda3/envs/$CUSTOM_ENV_NAME RUN /opt/python/bin/python -m venv /opt/anaconda3/envs/$CUSTOM_ENV_NAME RUN /opt/anaconda3/envs/$CUSTOM_ENV_NAME/bin/pip install ipykernel==5.5.3 ipython==7.31.1 ipython_genutils==0.2.0 jedi==0.17.2 RUN PATH=/opt/anaconda3/envs/$CUSTOM_ENV_NAME/bin/:$PATH Rscript -e "IRkernel::installspec(sys_prefix=TRUE)" RUN /opt/datalore/build_code_insight_data.sh /opt/anaconda3/envs/$CUSTOM_ENV_NAME

Last modified: 26 October 2023