Add GPU Support
If you want developers to use GPU resources in dev environments, you need to add GPU support to the corresponding computing platform and use this platform in an instance type. All dev environments based on a template with this instance type will have access to GPU resources.
Important: We've tested GPU support in CodeCanvas exclusively with NVIDIA GPUs, so we don't provide instructions for other GPU vendors. However, GPU support for other vendors can be added in a similar way.
How GPU support works
CodeCanvas uses the Docker-in-Docker (DinD) approach to run dev environments: a Kubernetes pod runs a worker container which, in turn, uses DinD to run an internal container. This internal container is an actual dev environment where the IDE runs. The problem is that this container doesn't have direct access to the host machine's hardware, including GPUs. To make GPU resources available in dev environments, you need to explicitly configure GPU support at the worker container level:
Install the NVIDIA Container Toolkit in the worker container.
Enable GPU settings in CodeCanvas (see parameters below).
Add GPU support to computing platform
Follow the instructions of your cloud provider to add GPU support to the Kubernetes cluster that runs dev environments. Typically, this includes adding a GPU node pool to the cluster and installing the GPU drivers to the nodes.
To make GPU resources available in dev environments, corresponding worker containers must have NVIDIA Container Toolkit installed. You can do this by creating a custom Docker image for worker containers. The image must be based on the CodeCanvas worker image and include the NVIDIA Container Toolkit installation. Here is an example of a
Dockerfile:FROM public.registry.jetbrains.space/p/codecanvas/release-charts/codecanvas-worker-k8s:2024.3.1 # Install NVIDIA Container Toolkit using the official instructions # [[[https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html|https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html]]] # You can uncomment and use the following commands to install NVIDIA Container Toolkit # Note that these commands may become outdated, so it's better to refer to the official instructions #RUN mkdir -pm755 /usr/share/keyrings && \ # curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \ # gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && \ # curl -fsSL "https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list" | \ # sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ # tee /etc/apt/sources.list.d/nvidia-container-toolkit.list > /dev/null && \ # apt-get update && \ # apt-get install --no-install-recommends -y nvidia-container-toolkit && \ # rm -rf /var/lib/apt/lists/* && \ # nvidia-ctk runtime configure --runtime=dockerCreate a Docker image from the
Dockerfileand push it to your Docker registry. Say, you've pushed the image tomy-registry.com/my-worker-image:latest.In CodeCanvas, open the Administration | Instance Types page.
Create a new instance type that will have access to GPU resources. When creating the instance type, ensure to specify the following:
Computing platform – the platform where you've added GPU support.
Pod template YAML (a template for worker pods) – specify the required GPU configuration and the custom Docker image created in the previous steps.
For example, to enable a single GPU:
apiVersion: v1 kind: Pod spec: containers: - name: worker-main image: my-registry.com/my-worker-image:latest resources: limits: nvidia.com/gpu: 1 # Allocate one GPU env: - name: WORKER_STEP_CONTAINER_GPUSETTINGS_ENABLED value: true # Enable GPU support - name: WORKER_STEP_CONTAINER_GPUSETTINGS_COUNT value: 1 # Request one GPU- GPU configuration parameters
WORKER_STEP_CONTAINER_GPUSETTINGS_ENABLED(default:false) – enables GPU support in worker containers. Must be set totruefor any other GPU settings to take effect.WORKER_STEP_CONTAINER_GPUSETTINGS_COUNT(default:null, optional) – defines the number of GPUs allocated to the container. This setting makes CodeCanvas run a worker container with the--gpus <count>Docker option. For example, useWORKER_STEP_CONTAINER_GPUSETTINGS_COUNT=1to allocate a single GPU – CodeCanvas will run the worker container withdocker run --gpus 1.WORKER_STEP_CONTAINER_GPUSETTINGS_DEVICEIDS(default:null, optional) Assigns specific GPUs to the container using device IDs. IDs should be separated by semicolons. This setting makes CodeCanvas run a worker container with the--gpus "device=Id1,Id2,..."Docker option. For example, useWORKER_STEP_CONTAINER_GPUSETTINGS_DEVICEIDS="01;02"to assign GPUs01and02– CodeCanvas will run the worker container withdocker run --gpus "device=01,02".WORKER_STEP_CONTAINER_GPUSETTINGS_CAPABILITIES(default:"gpu") Defines GPU capabilities exposed to the container. This setting maps to theNVIDIA_DRIVER_CAPABILITIESenvironment variable.WORKER_STEP_CONTAINER_GPUSETTINGS_DRIVER(default:"nvidia") Specifies the GPU driver to use. Keep"nvidia"unless using a custom driver.
After you create the instance type, CodeCanvas users will be able to select it when creating dev environment templates. All dev environments based on these templates will have access to GPU resources.