Installation to Google GKE

This guide describes how to install the CodeCanvas application to a Kubernetes (K8s) cluster hosted in Google Kubernetes Engine (GKE). It implies that the database and object storage services are hosted in Google Cloud, namely, in Cloud SQL and Cloud Storage bucket.

Requirements

CodeCanvas application cluster

Requirement	Description
Kubernetes	Version 1.27 or later
Cluster nodes	The cluster has at least four nodes. OS: Linux, platform: x86_64, recommended min resources: 4 CPU cores and 8GB memory
Namespace	A dedicated Kubernetes namespace for the CodeCanvas installation
User permissions	You have administrator permissions for the Kubernetes namespace
Ingress controller	Any Kubernetes-supported Ingress controller. In our example, we will use an ingress-nginx controller

Dev environment cluster

Requirement	Description
Kubernetes	Version 1.27 or later
Cluster nodes	The cluster has several nodes with enough resources to start dev environments. Recommended min resources: 4 CPU cores and 8GB memory
Cluster node OS	Ubuntu Linux, platform: x86_64. Currently, Container-Optimized OS is not supported
Namespace	A dedicated Kubernetes namespace
User permissions	You have administrator permissions for the Kubernetes namespace
CSI driver	Used by the CodeCanvas application to create Kubernetes volumes and snapshots for dev environments at runtime. By default, the GKE installation uses the `pd.csi.storage.gke.io` CSI driver
Storage class	You should create a storage class before the installation. For ease, you can use our recommended storage class configuration
CSI snapshot controller	An add-on to the CSI driver that implements Kubernetes snapshot manipulation. The snapshot support in the `pd.csi.storage.gke.io` CSI driver is enabled by default in GKE at the moment. By default, the GKE installation has enabled snapshot support in the `pd.csi.storage.gke.io` CSI driver
Volume snapshot class	You should create a volume snapshot class before the installation. For ease, you can use our recommended volume snapshot class configuration
MTU for VPC	The maximum transmission unit is 1500 bytes

Other requirements

Requirement	Description
Helm	Version 3.6.0 or later
DNS	You have a separate domain name for the CodeCanvas instance
CodeCanvas database	PostgreSQL 12.2 – 15.5
Jump server database	PostgreSQL 12.2 – 15.5
Google Storage bucket	You should create a bucket before the installation
SMTP Server	Optional

Prerequisites

DNS zones and TLS

The CodeCanvas GKE installation described in this topic implies that:

DNS zones are managed by ExternalDNS installed in the CodeCanvas application cluster.
TLS certificates issued by Let's Encrypt are managed by cert-manager installed in the CodeCanvas application cluster.

If you're using a different setup for DNS zones and TLS certificates management, you might need to adjust the custom.values.yaml file below accordingly to your setup.

DNS domains

The configuration described in custom.values.yaml below supposes the following DNS domain naming scheme:

EXTERNAL_DOMAIN_PLACEHOLDER – the main domain for the CodeCanvas application that serves the main administrative UI and REST API. For example, codecanvas.example.com
computeservice.EXTERNAL_DOMAIN_PLACEHOLDER – the subdomain that serves the compute-service REST API. It is an internal domain customarily accessed only by dev environment pods.
gateway.EXTERNAL_DOMAIN_PLACEHOLDER – serves the gateway-relay service. It is an external domain accessed by user IDE clients.
jump.EXTERNAL_DOMAIN_PLACEHOLDER – serves the REST API of the jump-server service. It is an internal domain customarily accessed only by dev environment pods and the server.
ssh.EXTERNAL_DOMAIN_PLACEHOLDER – serves the SSH service of the jump-server service. It is an external domain accessed by user IDE clients.

Cloud SQL for PostgreSQL

If you decide to use Cloud SQL for PostgreSQL as a database for the CodeCanvas application and the Jump server, note the following. The CodeCanvas application and the Jump server are set up to connect directly to the Cloud SQL for PostgreSQL databases without using the Cloud SQL Auth Proxy. This means that the CodeCanvas application must be hosted on a VPC-native cluster.

IAM configuration

To set up CodeCanvas access to a storage bucket and a database in Google Cloud, you will need a specific IAM (Identity and Access Management) configuration:

Workload Identity
You should enable the Workload Identity feature for the CodeCanvas application cluster (and its nodes).
Google IAM service account
You should create a separate Google IAM service account specifically for the CodeCanvas application.
Role Binding for Workload Identity
The Google IAM service account should have the roles/iam.workloadIdentityUser role. This role should be bound to both the Kubernetes namespace and the Kubernetes service account that run the CodeCanvas application's pod. This will allow the Kubernetes service account to use the identity of the Google IAM service account to access Google Cloud resources.
Access to Storage Bucket
The same Google IAM service account should have the roles/storage.objectUser role (or custom role with the similar permissions). This role grants the service account access to the Google Storage bucket.

Sysbox configuration

By default, CodeCanvas runs worker containers in the --privileged mode (the containers have root privileges on the host node). If you want to avoid this due to security reasons, use Sysbox Container Runtime as described here.

I. Install CodeCanvas

1. Create a Kubernetes namespace for the CodeCanvas application

Create a namespace for the CodeCanvas application (replace NAMESPACE_PLACEHOLDER with an actual namespace name):

kubectl create namespace NAMESPACE_PLACEHOLDER

2. Create custom.values.yaml

Create a custom.values.yaml and copy the snippet below to it.

application: replicaCount: 2 serviceAccount: name: "CODECANVAS_KSA_NAME" annotations: "iam.gke.io/gcp-service-account": "CODECANVAS_GSA_EMAIL" ingress: annotations: "cert-manager.io/cluster-issuer": "CERT_MANAGER_NAME_PLACEHOLDER" ingressClassName: "INGRESS_CLASS_PLACEHOLDER" hostname: "EXTERNAL_DOMAIN_PLACEHOLDER" computeHostname: "computeservice.EXTERNAL_DOMAIN_PLACEHOLDER" tls: - hosts: - "EXTERNAL_DOMAIN_PLACEHOLDER" - "computeservice.EXTERNAL_DOMAIN_PLACEHOLDER" secretName: codecanvas-and-computeservice-tls config: codecanvas: frontend: enableWebdriverTestIds: true relay: url: wss://gateway.EXTERNAL_DOMAIN_PLACEHOLDER jump: enabled: true sshHost: ssh.EXTERNAL_DOMAIN_PLACEHOLDER sshExternalHost: ssh.EXTERNAL_DOMAIN_PLACEHOLDER hotPool: enabled: true execution: k8s: worker: storageClassName: "WORKER_STORAGE_CLASS_NAME_PLACEHOLDER" volumeSnapshotClassName: "WORKER_VOLUME_SNAPSHOT_CLASS_NAME_PLACEHOLDER" secret: objectStorage: existingSecretName: "codecanvas-objectstorage-secret-ext" postgresql: existingSecretName: "codecanvas-db-secret-ext" codecanvas: jump: jwtPrivateKey: | JUMP_PRIVATE_KEY_PLACEHOLDER jumpHostPrivateKey: | JUMP_HOST_PRIVATE_KEY_PLACEHOLDER localAdministrator: firstName: "Admin" lastName: "Admin" username: "ADMIN_USERNAME_PLACEHOLDER" password: "ADMIN_PASSWORD_PLACEHOLDER" email: "ADMIN_EMAIL_PLACEHOLDER" masterSecret: "MASTER_SECRET_PLACEHOLDER" relay: jwtPrivateKey: | GATEWAY_PRIVATE_KEY_PLACEHOLDER jump: enabled: true application: sshService: type: LoadBalancer annotations: "external-dns.alpha.kubernetes.io/hostname": "ssh.EXTERNAL_DOMAIN_PLACEHOLDER" ingress: enabled: true ingressClassName: "INGRESS_CLASS_PLACEHOLDER" hostname: "jump.EXTERNAL_DOMAIN_PLACEHOLDER" annotations: "cert-manager.io/cluster-issuer": "CERT_MANAGER_NAME_PLACEHOLDER" tls: - hosts: - "jump.EXTERNAL_DOMAIN_PLACEHOLDER" secretName: jump-tls secret: postgresql: existingSecretName: "jump-db-secret-ext" jump: sshHostKey: | JUMP_HOST_PRIVATE_KEY_PLACEHOLDER ecdsaPublicKey: | JUMP_PUBLIC_KEY_PLACEHOLDER relay: application: ingress: ingressClassName: "INGRESS_CLASS_PLACEHOLDER" hostname: "gateway.EXTERNAL_DOMAIN_PLACEHOLDER" annotations: "cert-manager.io/cluster-issuer": "CERT_MANAGER_NAME_PLACEHOLDER" tls: - hosts: - "gateway.EXTERNAL_DOMAIN_PLACEHOLDER" secretName: gateway-tls secret: relayJwtPublicKey: | GATEWAY_PUBLIC_KEY_PLACEHOLDER

3. Assign a Google IAM service account to the CodeCanvas application

In custom.values.yaml, replace CODECANVAS_GSA_EMAIL with the email of the Google IAM service account for CodeCanvas application.

4. Set up an external domain

In custom.values.yaml, replace EXTERNAL_DOMAIN_PLACEHOLDER with the domain name you've registered for your CodeCanvas instance.

5. Set up cert-manager

In custom.values.yaml, replace CERT_MANAGER_NAME_PLACEHOLDER with the name of the cert-manager's ClusterIssuer used in your cluster.

6. Specify CodeCanvas database settings

The CodeCanvas installation implies that you use an external PostgreSQL database.

6.1 Specify credentials

In the CodeCanvas PostgreSQL database, create a user account that will be used for accessing the database. The user should have permissions to create, read, update, and delete all entities in the schema.

6.2 Create a database secret

A database secret is used to secure access to the PostgreSQL database.

Create a codecanvas-db-secret.yaml file and copy the snippet below to it.
apiVersion: v1 kind: Secret type: Opaque metadata: name: codecanvas-db-secret-ext namespace: NAMESPACE_PLACEHOLDER stringData: DB_HOST: "CODECANVAS_DB_HOST_PLACEHOLDER" DB_NAME: "CODECANVAS_DB_NAME_PLACEHOLDER" DB_PASSWORD: "CODECANVAS_DB_PASSWORD_PLACEHOLDER" DB_PORT: "CODECANVAS_DB_PORT_PLACEHOLDER" DB_USERNAME: "CODECANVAS_DB_USERNAME_PLACEHOLDER"
Replace the following placeholders:
- NAMESPACE_PLACEHOLDER with your Kubernetes namespace
- CODECANVAS_DB_PASSWORD_PLACEHOLDER with a password for the CODECANVAS_DB_USERNAME_PLACEHOLDER user
- CODECANVAS_DB_HOST_PLACEHOLDER with the PostgreSQL hostname
- CODECANVAS_DB_PORT_PLACEHOLDER with the PostgreSQL port
Run:
kubectl apply -f codecanvas-db-secret.yaml
Delete the codecanvas-db-secret.yaml file.

7. Specify Jump server database settings

The Jump server also uses an external PostgreSQL database.

7.1 Specify credentials

In the Jump server PostgreSQL database, create a user account that will be used for accessing the database.

7.2 Create a database secret

A database secret is used to secure access to the PostgreSQL database.

Create a jump-db-secret.yaml file and copy the snippet below to it.
apiVersion: v1 kind: Secret type: Opaque metadata: name: jump-db-secret-ext namespace: NAMESPACE_PLACEHOLDER stringData: DB_HOST: "JUMPSERVER_DB_HOST_PLACEHOLDER" DB_NAME: "JUMPSERVER_DB_NAME_PLACEHOLDER" DB_PASSWORD: "JUMPSERVER_DB_PASSWORD_PLACEHOLDER" DB_PORT: "JUMPSERVER_DB_PORT_PLACEHOLDER" DB_USERNAME: "JUMPSERVER_DB_USERNAME_PLACEHOLDER"
Replace the following placeholders:
- NAMESPACE_PLACEHOLDER with your Kubernetes namespace
- JUMPSERVER_DB_PASSWORD_PLACEHOLDER with a password for the JUMPSERVER_DB_USERNAME_PLACEHOLDER user
- JUMPSERVER_DB_HOST_PLACEHOLDER with the PostgreSQL hostname
- JUMPSERVER_DB_PORT_PLACEHOLDER with the PostgreSQL port
Run:
kubectl apply -f jump-db-secret.yaml
Delete the jump-db-secret.yaml file.

8. Specify object storage settings

CodeCanvas installation implies that you use an external object storage. The instructions below apply to Google Storage.

8.1 Create an object-storage secret

The object-storage secret is used to secure access to the Google Storage database.

Create an object-storage-secret.yaml file and copy the snippet below to it.
apiVersion: v1 kind: Secret type: Opaque metadata: name: codecanvas-objectstorage-secret-ext namespace: NAMESPACE_PLACEHOLDER stringData: CODECANVAS_OBJECT_STORAGE_ACCESS_KEY: "CODECANVAS_OBJECT_STORAGE_ACCESS_KEY_PLACEHOLDER" CODECANVAS_OBJECT_STORAGE_SECRET_KEY: "CODECANVAS_OBJECT_STORAGE_SECRET_KEY_PLACEHOLDER" CODECANVAS_OBJECT_STORAGE_BUCKET: "CODECANVAS_OBJECT_STORAGE_BUCKET_PLACEHOLDER" CODECANVAS_OBJECT_STORAGE_ENDPOINT: "https://storage.googleapis.com" CODECANVAS_OBJECT_STORAGE_REGION: "CODECANVAS_OBJECT_STORAGE_REGION_PLACEHOLDER"
Replace the following placeholders:
- NAMESPACE_PLACEHOLDER with your Kubernetes namespace
- CODECANVAS_OBJECT_STORAGE_BUCKET_PLACEHOLDER with the name of the Google Storage bucket created earlier
- CODECANVAS_OBJECT_STORAGE_REGION_PLACEHOLDER with the region of the Google Storage bucket (like europe-west1)
For access with AWS-like credentials, replace the following placeholders:
- CODECANVAS_OBJECT_STORAGE_ACCESS_KEY_PLACEHOLDER with the access key
- CODECANVAS_OBJECT_STORAGE_SECRET_KEY_PLACEHOLDER with the secret key
Run:
kubectl apply -f object-storage-secret.yaml
Delete the object-storage-secret.yaml file.

9. Create a master secret

The CodeCanvas application keeps user secrets (e.g., credentials to external services) in the database in an encrypted form. The master secret is used to encrypt and decrypt these data.

Generate the master secret by running
openssl rand -base64 32
In custom.values.yaml, replace MASTER_SECRET_PLACEHOLDER with the generated value.

10. Configure the system administrator account

The system administrator account will be used for logging in to and configuring CodeCanvas after the installation.

In custom.values.yaml, replace ADMIN_USERNAME_PLACEHOLDER and ADMIN_PASSWORD_PLACEHOLDER with desired administrator credentials.
Replace ADMIN_EMAIL_PLACEHOLDER with an email address for receiving administrator notifications from CodeCanvas.

11. Specify the storage class

In custom.values.yaml, replace WORKER_STORAGE_CLASS_NAME_PLACEHOLDER with the storage class name you've created.

12. Specify the volume snapshot class name

In custom.values.yaml, replace WORKER_VOLUME_SNAPSHOT_CLASS_NAME_PLACEHOLDER with the volume snapshot class name you've created.

13. Create gateway-relay keys

The Relay server acts as an intermediary between JetBrains Gateway on a user machine and the dev environment. The communication between them is secured with SSL/TLS. To establish a secure connection, the gateway and the Relay server must have a pair of keys which you need to generate. The public key is shared with the Relay server. The private key is then used by JetBrains Gateway to authenticate dev environments in the Relay server.

Generate a private key:
openssl ecparam -name prime256v1 -genkey -noout -out gateway-ec-prime256v1-priv-key.pem
cat gateway-ec-prime256v1-priv-key.pem | base64
In custom.values.yaml, replace GATEWAY_PRIVATE_KEY_PLACEHOLDER with the generated key value.
Generate a public key:
openssl ec -in gateway-ec-prime256v1-priv-key.pem -pubout > gateway-ec-prime256v1-pub-key.pem
cat gateway-ec-prime256v1-pub-key.pem | base64
In custom.values.yaml, replace GATEWAY_PUBLIC_KEY_PLACEHOLDER with the generated key value.
Delete the gateway-ec-prime256v1-priv-key.pem and gateway-ec-prime256v1-pub-key.pem files.

14. Create Jump server keys

The Jump server acts as an intermediate server that provides indirect SSH connections between a user machine (SSH client, VS Code in the remote mode, etc.) and the SSH daemon in a dev environment. The SSH connection requires a pair of keys which you need to generate. The public key is shared with the Jump server. The private key is then used by the SSH client to authenticate in the Jump server.

Generate a private key:
openssl ecparam -name prime256v1 -genkey -noout -out jump-ec-prime256v1-priv-key.pem
cat jump-ec-prime256v1-priv-key.pem | base64
In custom.values.yaml, replace JUMP_PRIVATE_KEY_PLACEHOLDER with the generated key value.
Generate a public key:
openssl ec -in jump-ec-prime256v1-priv-key.pem -pubout > jump-ec-prime256v1-pub-key.pem
cat jump-ec-prime256v1-pub-key.pem | base64
In custom.values.yaml, replace JUMP_PUBLIC_KEY_PLACEHOLDER with the generated key value.
Delete the jump-ec-prime256v1-priv-key.pem and jump-ec-prime256v1-pub-key.pem files.
Generate a private host key:
openssl genrsa -traditional 2048 2>/dev/null
In custom.values.yaml, replace JUMP_HOST_PRIVATE_KEY_PLACEHOLDER with the generated key value.

15. Specify the Ingress class

In custom.values.yaml, replace INGRESS_CLASS_PLACEHOLDER with the Ingress class used for the CodeCanvas Kubernetes cluster.

16. Specify the Kubernetes service account for the CodeCanvas pod

Suppose you've set up Workload Identity in the application cluster and prefer the service account name to be independent of the Helm release name. In that case, you may want to specify a particular name for the Kubernetes service account that the CodeCanvas Helm chart will create. To do this, in custom.values.yaml, replace CODECANVAS_KSA_NAME with the desired name.

17. Install the CodeCanvas chart

Run:

                helm upgrade -n NAMESPACE_PLACEHOLDER --wait --install \
                -f custom.values.yaml \
                codecanvas \
                oci://public.registry.jetbrains.space/p/codecanvas/releases/codecanvas \
                --version 2024.1.1
            

Here:

NAMESPACE_PLACEHOLDER is your Kubernetes namespace
codecanvas is the Helm release name. You can change it if needed

II. Verify the installation

After you install your CodeCanvas instance, verify the installation.

1. Verify the state of CodeCanvas pods

Run:

kubectl -n NAMESPACE_PLACEHOLDER get pods

All pods must be in the Running state. On average, it takes about 2 minutes after deployment for a pod to become active.

If the pods are not Running, try finding the cause by running:

kubectl -n NAMESPACE_PLACEHOLDER get event

and

kubectl -n NAMESPACE_PLACEHOLDER describe pod POD_NAME_PLACEHOLDER

2. Verify domain name resolution

The domain name must resolve to the Ingress load balancer. You can check this by running:

nslookup EXTERNAL_DOMAIN_PLACEHOLDER

nslookup gateway.EXTERNAL_DOMAIN_PLACEHOLDER

nslookup jump.EXTERNAL_DOMAIN_PLACEHOLDER

nslookup ssh.EXTERNAL_DOMAIN_PLACEHOLDER

The output must not contain any errors.

3. Check the CodeCanvas application

Open your CodeCanvas instance in a browser. When logging in to CodeCanvas, use the administrator credentials provided during the installation.

III. Connect the dev environment cluster

To connect the dev environment cluster to CodeCanvas, you should create a connection in the CodeCanvas application.

In CodeCanvas, on the sidebar menu, select Administration, then Computing Platform.
Click New connection.
Give this connection a Name, specify the cluster's K8s Namespace, and click Save. The connection will be added to the list. Here you can also modify the pod YAML template according to your needs. Learn more
Click the connection in the list to open its details.
The connection details page provides the snippet of the helm upgrade command that you should use to install the CodeCanvas operator in the dev environment cluster. This operator will communicate with the CodeCanvas application and start/stop dev environments in the cluster.
To connect to the CodeCanvas application, it will need an access token:
1. Click Generate token.
2. Copy the snippet to the clipboard.
Install the CodeCanvas operator Helm chart on the dev environment cluster using the snippet.
Click Test connection and ensure that all checks are successful.

IV. Post-installation steps

After successfully verifying the installation and connecting the dev environment cluster, you can proceed to configure your CodeCanvas instance: creating dev environment instance types, adding users, namespaces, and so on.

Last modified: 02 July 2024