Remote development overview

Remote development lets you use a remote server as a development environment. The SSH connection allows you to quickly connect to the remote server and take full advantage of PyCharm features.

Architecture and definitions

To better understand bits and pieces of this architecture, you need to get familiar with the following definitions:

Server

This is a physical or virtual machine to host the source code and run the headless PyCharm. It performs most of the IDE features.

Requirements: Linux (macOS and Windows are coming later)

Client

This is a physical machine used by you to connect to the server and provide a thin client for working. For example, this could be your laptop. However, there are no restrictions for clients specifications.

Requirements: macOS, Linux, Windows

IDE Backend

This is a compatible JetBrains IDE running on the server. It is run as a server process, and therefore has no visible user interface - it’s headless. This is the same executable as you might currently be running for your JetBrains IDE.

JetBrains Client

This is a thin client you see after you launch JetBrains Gateway and establish a connection to a remote server. It gets connected to the IDE backend and gives you the ability to do remote development as if it is local.

In contrast to remote-desktop solutions, JetBrains Client has an advanced editor based on the JetBrains Rider’s RD protocol, which makes typing feel instant. Additionally, the advanced code intelligence implemented on the PyCharm server is presented in a way that looks as if it’s running locally.

JetBrains Client is based on the IntelliJ platform, therefore it imports any local IDE settings found on the local machine (Client). For example, if there is the PyCharm 2021.3 version installed on Client, the downloaded and just launched JetBrains Client will import its settings, so the remote development experience will be personalized for you.

JetBrains Gateway

This is an entry point for remote development. It runs on Client, and lets you create new remote environments or connect to the existing ones. After you request an environment in JetBrains Gateway, it launches JetBrains Client that is specific to that environment.

JetBrains Gateway is responsible for the following tasks:

Allowing you to set up a remote connection to any kind of remote backend you have
Exploring existing recent remote projects
Connecting to either the new or recent projects
Downloading JetBrains Client of the appropriate version from the indicated storage
Launching JetBrains Client
Maintaining the connection. It is the JetBrains Gateway process that configures and establishes a connection to Server (SSH or port forwarding or any other custom connection) and launches JetBrains Client against the opened connection or a port. Moreover, JetBrains Gateway is responsible for holding the connection, handling reconnects (with or without UI consent), and indicating the status information about the current connections.

Workflows

There are two types of main workflows available in remote development:

Server to client flow: when an organization first sets up a remote headless server, obtains a link (customized or not) which then can be used by you on Client (for example, your laptop) to connect to the available server.
Client to server flow: when you set up and install the remote server from the Client UI using the SSH or other connection.

Server to client workflow

You need to run the IDE backend on your server with the following command:

remote-dev-server.sh run <path/to/project> --ssh-link-host <host>

You can specify other ssh link connection options for the browser flow:

--ssh-link-user
--ssh-link-port

Check the following examples:

remote-dev-server.sh run <path/to/project> --ssh-link-host remote.server.com
--ssh-link-user adam --ssh-link-port 42

In the output you should receive the following connection link:

Http link: https://code-with-me...

You can use this link in the different connection scenarios such as connection via SSH, via custom connection, or via browser.

For the detailed information on how to start your Remote Development session, refer to Getting started with remote development.

Connect using SSH

Start JetBrains Gateway.
Select Connect via SSH.
Invoke the SSH settings and configure options for the already running backend.
JetBrains Gateway connects by SSH, gets the connection link from the existing backend. JetBrains Gateway forwards a port from the connection link to the local machine and prepares a new connection link with that local-to-client port.
It also downloads and launches JetBrains Client with this link. The JetBrains Client version is specified in the link. JetBrains Gateway operates in the background and forwards the port while JetBrains Client is still active.

Configure the custom connection

Start JetBrains Gateway.
Select some custom connection UI provided by the JetBrains Gateway plugin.
A custom plugin may connect to some cloud service, fetch a list of backends from that service, create backends, and so on.
JetBrains Gateway gets the connection link (in most cases it points to the local host and port) from the custom plugin, and with this link, downloads and launches JetBrains Client. The JetBrains Client version is specified in the link. JetBrains Gateway may stay in the background in case it should maintain connection between Client and Server.

Connect through browser

In the browser, click a specifically created link that describes a backend location.
In the case of SSH, it contains a username, a server name, the SSH port, project location on the server, and so on.
This link leads to a landing page that suggests opening JetBrains Gateway or downloading it if JetBrains Gateway is missing on Client. JetBrains Gateway handles link opening from the browser and connects to a specified backend.

Client-to-server workflow

The client-to-server workflow means that you don't have any IDE installed on your remote server. In this case, you first need to download and install JetBrains Gateway. For more detailed information, refer to Launch JetBrains Gateway and connect to a remote server.

Connect via SSH

Download and install JetBrains Gateway.
In the JetBrains Gateway wizard, select Connect via SSH to connect to a remote server.
On the next page of the wizard, select the server to which you want to connect. If there is no IDE on the remote server, JetBrains Gateway will download it.
Depending on the user preference, the IDE may be downloaded from JetBrains servers, uploaded from the Client machine, or downloaded from a custom location.
The RD protocol is specific to each version of the IDE backend. The matching version of JetBrains Client needs to be downloaded on the server.

After this, JetBrains Gateway starts the PyCharm server on the remote machine, establishes the TCP tunnel, and starts JetBrains Client with the appropriate arguments to have it connected.

The TCP connection link format is as follows:

                tcp://127.0.0.1:PORT#jt=ONE_TIME_CONNECTION_TOKEN fp=SERVER_FINGERPRINT cb=THIN_CLIENT_BUILD jb=THIN_CLIENT_RUNTIME_VERSION
            

Check the following example:

                tcp://127.0.0.1:5990#jt=3f2f2471-84f2-4d8b-ac67-c1698f5a1be7 fp=ABC88C383B0A62654C4EF75052C60D4D475885075DCA5B85733BD8D4B9E28CC
                0 cb=213.2667 jb=11_0_11b1620.1
            

Extensibility

The IDE backend

The IDE backend is the full-blown JetBrains IDE, which is just being launched in a special headless unattended mode.

The backend can be extended with all the diversity of PyCharm plugins in the following ways:

By unpacking required plugins to plugins/ folder of the distribution
By running the following code (requires network connection to JetBrains Plugin Marketplace):
./bin/remote-dev-server.sh installPlugins <pathtoproject> <PLUGIN_ID1> <PLUGIN_ID2> ...

If a plugin provides the new set of inspections and features, all of those will be shown on JetBrains Client.

The only plugins that will not have any effect on Remote Development are the ones that aggressively modify the UI . Such plugins should be installed on the JetBrains Client side.

JetBrains Gateway SDK

JetBrains Gateway can be extended like any other IntelliJ platform-based product.

You can use one of the following ways:

Set up a new project with https://github.com/JetBrains/gradle-intellij-plugin/ (gradle-intellij-plugin should be >= 1.1.4)
Use the following settings to build your plugin against JetBrains Gateway:
intellij { version.set("213.2667-CUSTOM-SNAPSHOT") type.set("GW") instrumentCode.set(false) }
You may see available versions at https://www.jetbrains.com/intellij-repository/snapshots (see group com.jetbrains.gateway)

JetBrains Gateway / Thick PyCharm on a Client

Apart from the basic SSH and Code With Me connections, a vendor can customize JetBrains Gateway for its own orchestration service. This can be done within the custom deal between the JetBrains and the vendor.

JetBrains Gateway is based on the IntelliJ platform, it has APIs for connections and interactions with JetBrains Client.

Check the following example:

A big organization wants to write its own orchestration. The basic SSH flow is not enough due to security reasons. The organization writes an internal plugin and delivers it to its developers. Developers can install this plugin in JetBrains Gateway or in PyCharm on their laptops.

This is a very brief introduction of the APIs (they are not yet final and indeed this is not the full scope, but they explain the overall idea).

JetBrains Client

JetBrains Client is not designed to be extensible for the connection part. However, you can develop and install all the variety of PyCharm plugins, which modify the UI, keyboard shortcuts, themes, and other parts that touch the IDE UI interaction, but not its functionality.

Last modified: 12 April 2022