Configure agents

Last modified: 28 August 2023

You can change the default agents configuration by mounting a config file into /opt/datalore/configs/agents-config.yaml inside the Datalore container. This config has the following format:

docker:
  network: <name-of-agents-network>
  dataloreHost: <host-of-datalore-server>
  instances:
    - <instance1-definition>
    - <instance2-definition>
      ...
    - <instanceN-definition>

By default, <name-of-agents-network> is datalore-agents-network, and <host-of-datalore-server is datalore.

Each instance definition has the following fields:

Mandatory fields

id: basic-agent

Specifies the unique id for the instance.

default: true

Specifies whether the instance is default. Make sure exactly one instance has this parameter set to true.

label: "docker-base"

Specifies the instance name shown in the UI.

image: docker.io/jetbrains/datalore-agent:2023.4

URI of the agent docker image, changed only to specify a custom agent image

Optional fields

pullPolicy: "IF_ABSENT"

Specifies when the image will be pulled. Acceptable values are: NEVER, ALWAYS, IF_ABSENT (default).

cpus: 1.5

Specifies how much of the available CPU resources a container can use. For example, if the host machine has two CPUs and you set cpus to 1.5, the container is guaranteed at most one and a half of the CPUs.

memory: 4000000000

Specifies the maximum amount of memory the container can use.

note

The following suffixes are supported: M, Mi, G, Gi. Bytes are assumed when no suffix is specified.

env:
  - <env1-definition>
  - <env2-definition>
    ...
  - <envN-definition>

Defines environment variables on the agent. Each env definition contains a name and a value.

Example:

env:
  - name: DATALORE_SKIP_WARMUP
    value: true

In the example, the DATALORE_SKIP_WARMUP environment value is set to true on the agent. In this case, the analysis server warmup is skipped on agent initialization, which leads to faster initialization at the cost of slower code completion for some time.

initialPoolSize: 1

Specifies the number of pre-warmed (pooled) agents. This is the minimum number of agents Datalore will keep started and yet unassigned to any computation. When a notebook is opened, Datalore assigns one of the pooled agents to that notebook and starts another agent to refill the pool. It might be useful to set initialPoolSize to a non-zero value if initialization of your agents takes a lot of time.

note

Options below are only used to adjust UI appearance, not to specify agent properties. To actually set limits for an agent, use the cpus and memory fields.

shortName: "S"

Specifies the short instance name.

note

If it contains more than three letters, only the first three are displayed.

numCPUs: 2

Specifies the number of CPUs to display.

cpuMemoryText: "8 GB"

Specifies the agent RAM description.

numGPUs: 1

Specifies the number of GPUs to display.

gpuMemoryText: "4 GB"

Specifies the GPU memory description.

features:
- "1 vCPU"
- "16 GB RAM"
- "1 NVIDIA GPU"

Specifies specific properties of this agent to display in the UI.

Only properties containing the "RAM", "CPU", or "GPU" substrings will be displayed in the agent selection window.