Upgrade Datalore Enterprise

General information

The instructions below explain how to upgrade Datalore Enterprise. The current latest version is 2022.2.3.

If you are upgrading from 2022.2, 2022.2.1, or 2022.2.2 to 2022.2.3, refer to this section.
If you are upgrading from 2022.2 to 2022.2.1, refer to this section.
If you are upgrading from an earlier version, upgrade to 2022.2 and then to a later version.
When upgrading to version 2022.2 or later, make sure you perform the data migration procedure.

Upgrade from 2022.2-2022.2.2 to 2022.2.3

Choose one of the procedures depending on your installation (Docker or Helm) type.

To upgrade Docker-based Datalore Enterprise, do the following:
1. Update docker-compose.yaml.
2. Run docker compose restart to restart the service.
To upgrade Helm-based Datalore Enterprise, do the following:
1. Execute the helm repo update command.
2. Restart Datalore by running the following commands:
  helm uninstall <release_name> helm upgrade --install -f datalore.values.yaml <release_name> datalore/datalore
  The procedure assumes that you store your custom values in datalore.values.yaml, which is NOT the same as the earlier location (charts/datalore/values.yaml). See the installation guide
  for more details.

Upgrade from 2022.2 to 2022.2.1

Choose one of the procedures depending on your installation (Docker or Helm) type.

To upgrade Docker-based Datalore Enterprise, do the following:
1. Update docker-compose.yaml.
2. Run docker compose restart to restart the service.
To upgrade Helm-based Datalore Enterprise, do the following:
1. Update the datalore-configs repository to the 2022.2.1 branch.
2. Restart Datalore by running the following commands:
  helm uninstall <path_to_datalore-configs>/charts/datalore helm upgrade --install -f datalore.values.yaml <path_to_datalore-configs>/charts/datalore

Upgrade from versions 2021.3.5-2022.1.4 to 2022.2 (Helm installation)

Update the datalore-configs repository to the 2022.2 branch.
Change FRONTEND_URL to DATALORE_PUBLIC_URL.
Change the format of dataloreEnv to a helm map.
Old format:
dataloreEnv: | export FRONTEND_URL="https://datalore.example.com" export HUB_PUBLIC_BASE_URL="https://hub.datalore.example.com/hub" export HUB_DATALORE_SERVICE_ID="XXX" export HUB_DATALORE_SERVICE_SECRET="YYY" export HUB_PERM_TOKEN="perm:ZZZ"
New format:
dataloreEnv: DATALORE_PUBLIC_URL: "https://datalore.example.com" HUB_PUBLIC_BASE_URL: "https://hub.datalore.example.com/hub" HUB_DATALORE_SERVICE_ID: "XXX" HUB_DATALORE_SERVICE_SECRET: "YYY" HUB_PERM_TOKEN: "perm:ZZZ"

Data migration when upgrading to 2022.2 or later versions

Starting from version 2022.2, data directories are accessible under new names. When upgrading to version 2022.2 or later, you need to manually specify the new location for the migrated data.

To access the Datalore container, run the following command:
kubectl exec -it datalore-0 -c datalore -- bash
To rename the data directories in the new version:
mv /opt/data/webdav /opt/data/web-dav mv /opt/data/blobs /opt/data/blob

Earlier versions

Perform the steps in the collapsed section to upgrade from version 0.3.0.

Upgrade from version 0.3.0

Execute the following command to stop traffic to Datalore:
kubectl delete svc datalore-on-premise
Wait for approximately one minute until the running computations are stopped. Then, execute the following command:
kubectl exec -it datalore-on-premise-0 -c postgres -- psql -ddatalore -Uadmin -c 'delete from core."RunningComputationsData";'
Delete the Docker Hub token:
kubectl delete secret regcred
Continue the procedure as described below.

Upgrade to version 2021.3.5

Delete Datalore:
kubectl delete -k ./datalore
If you upgrade from 0.3.0, ignore the following message:
Error from server (NotFound): error when deleting "datalore/": services "datalore-on-premise" not found
Clone the datalore-configs repository.
Check out the 2021.1.3.5 branch.
Update Hub:
1. Copy the spec.template.spec.volumes values from the old hub/statefulSet.yaml file into the new one (in the new branch).
  This is a very important step for preserving your data.
2. Update statefulset:
  kubectl apply -k hub/
3. Get the Hub token and copy it:
  kubectl logs service/hub
4. Open Hub at HUB_ROOT_URL, specify the token, and click login and, in the next window, upgrade.
Update Datalore:
1. Copy datalore/configs/volumes_config.yaml from the old directory to the new one.
  This is a very important step for preserving your data.
2. Copy datalore/configs/plans_config.yaml to the new directory.
3. If the agent configuration was changed, copy the datalore/configs/agents_config.yaml to the new directory and change the image: field to the respective tag.
4. If you used custom environments, copy datalore/configs/envs too.
5. Copy the ROOT_PASSWORD value from datalore/configs/db_config.yaml to datalore/configs/db_secret_env.txt.
  Pay attention to the difference in format:
  - Old file:
    apiVersion: v1 kind: ConfigMap metadata: name: datalore-on-premise-db-env data: ROOT_PASSWORD: "changeme"
  - New file:
    ROOT_PASSWORD=changeme
6. Copy the variables from datalore/configs/user_config.yaml to datalore/configs/user_secret_env.sh.
  Pay attention to the difference in format:
  - Old file:
    apiVersion: v1 kind: ConfigMap metadata: name: datalore-on-premise-env data: FRONTEND_URL: "http://datalore.example.com" HUB_PUBLIC_BASE_URL: "http://hub.datalore.example.com/hub" ...
  - New file:
    export FRONTEND_URL="http://datalore.example.com" export HUB_PUBLIC_BASE_URL="http://hub.datalore.example.com/hub" ...
  - If you have HUB_INTERNAL_BASE_URL="http://hub:8082/hub" in datalore/configs/user_config.yaml, you can remove the parameter. If it has a different value, copy it to user_secret_env.sh too.
  - Remove PASSWORD_SECRET if the file contains it.
Launch Datalore:
kubectl apply -k datalore/

Last modified: 08 September 2022