ProfessionalRemote development overview
Last modified: 06 July 2022Remote development lets you use a remote server as a development environment. The SSH connection allows you to quickly connect to the remote server and take full advantage of PyCharm features.
Architecture and definitions
To better understand bits and pieces of this architecture, you need to get familiar with the following definitions:
- Server
This is a physical or virtual machine to host the source code and run the headless PyCharm. It performs most of the IDE features.
Requirements: Linux (macOS and Windows are coming later)
- Client
This is a physical machine used by you to connect to the server and provide a thin client for working. For example, this could be your laptop. However, there are no restrictions for clients specifications.
Requirements: macOS, Linux, Windows
- IDE Backend
This is a compatible JetBrains IDE running on the server. It is run as a server process, and therefore has no visible user interface - it’s headless. This is the same executable as you might currently be running for your JetBrains IDE.
- JetBrains Client
This is a thin client you see after you launch JetBrains Gateway and establish a connection to a remote server. It gets connected to the IDE backend and gives you the ability to do remote development as if it is local.
In contrast to remote-desktop solutions, JetBrains Client has an advanced editor based on the JetBrains Rider’s RD protocol, which makes typing feel instant. Additionally, the advanced code intelligence implemented on the PyCharm server is presented in a way that looks as if it is running locally.
JetBrains Client is based on the IntelliJ platform, therefore it imports any local IDE settings found on the local machine (Client). For example, if there is the PyCharm 2021.3 version installed on Client, the downloaded and just launched JetBrains Client will import its settings, so the remote development experience will be personalized for you.
JetBrains Client's version is always equal to the version of the backen IDE.
- JetBrains Gateway
This is an entry point for remote development. It runs on Client, and lets you create new remote environments or connect to the existing ones. After you request an environment in JetBrains Gateway, it launches JetBrains Client that is specific to that environment.
JetBrains Gateway is responsible for the following tasks:
Allowing you to set up a remote connection to any kind of remote backend you have
Exploring existing recent remote projects
Connecting to either the new or recent projects
Downloading JetBrains Client of the appropriate version from the indicated storage
Launching JetBrains Client
Maintaining the connection. It is the JetBrains Gateway process that configures and establishes a connection to Server (SSH or port forwarding or any other custom connection) and launches JetBrains Client against the opened connection or a port. Moreover, JetBrains Gateway is responsible for holding the connection, handling reconnects (with or without UI consent), and indicating the status information about the current connections.
Workflows
There are two types of main workflows available in remote development:
Server-to-client flow: when an organization first sets up a remote headless server, obtains a link (customized or not) which then can be used by you on Client (for example, your laptop) to connect to the available server.
Client-to-server flow: when you set up and install the remote server from the Client UI using the SSH or other connection.
Connect to a remote project at manually launched remote IDE (Server-to-client flow)
If you already have PyCharm installed on your remote server, you can launch it manually and connect to the remote project started in that IDE. It works the same way as from JetBrains Gateway.
Use this approach if your company has a custom orchestration or in case your remote IDE starts automatically on its side.
The process can be described as follows:
Start a backend project in the remote IDE.
Select one of the connection links generated by the backend.
Open the link on your local machine.
The main script to run a remote IDE is remote-dev-server.sh, located in the bin subdirectory of your unpacked IDE.
note
In case of backend previously installed by JetBrains Gateway, the default path is
/home/your_system_user/.cache/JetBrains/RemoteDev/dist/.
note
For more information and available parameters, run
remote-dev-server.sh --help
note
Keep in mind that the connection is bound to a specific project. To connect to another remote project, you need to configure a new connection.
Connect to remote IDE
Ensure you have downloaded and installed JetBrains Gateway or an IDE with this bundled plugin on your local machine.
Ensure you have an SSH connection to the remote machine and the PyCharm IDE on it.
In the remote server's terminal, run the following command:
remote-dev-server.sh run /path_to_project/ --ssh-link-host host_server_address --ssh-link-user remote_side_user --ssh-link-port ssh_connection_portSee example:
If you don't pass the parameters, the script will use the default ones: 22 for port, system user's username, and host from the
hostnamecommand.~/PY-213.5744.248/bin/remote-dev-server.sh run ~/home/jetbrains/PythonProjects/SampleApp/ --ssh-link-host ec2-13-50-136-85.eu-north-1.compute.amazonaws.comIf the project starts successfully, as an output, you should receive the following 3 links inside terminal:
Join link: tcp://127.0.0.1:5990...[ ]: contains the local address and port where the remote IDE is listening now.
To use it, ensure the remote machine should be accessible by this local address. For example, for inside-Docker IDE with forwarded/open ports.
tip
Check "ssh -L" for detailed information.
Http link: https://code-with-me.jetbrains.com/remoteDev...[ ]: contains information about your host-port-user, the IDE and its version.
When opening in your local browser, it displays a welcome page and tries to call the local Gateway application with pre-filled connection settings values.
If no JetBrains Gateway application is found on the local machine, you'll be able to download it from the welcome page.
Gateway link: jetbrains-gateway://connect#idePath... [ ]: also contains information about your host-port-user, IDE and its version.
When opening in your local browser, it launches the local JetBrains Gateway application directly without a welcome page.
Copy the generated link and paste it into your local browser and allow it to launch Open JetBrains Gateway.
All these links can be also opened in the already launched JetBrains Gateway.
For this, on the JetBrains Gateway welcome screen, paste a link in the Connect to a Running IDE field, click Connect.
PyCharm downloads the required version of JetBrains Client and opens the remote project inside it.
Registering previously installed remote IDE
Since version 221.5481, you can manually register an existing backend IDE on the remote server and make it visible for the Gateway.
To register the installed IDE and make it appear in the list of available builds follow these steps:
Enter remote server by SSH
Locate the folder with unpacked IDE, enter
binfolderIn the command line, execute the following command:
remote-dev-server.sh registerBackendLocationForGatewayFor example,
sh WebStorm-221.5591.52/bin/remote-dev-server.sh registerBackendLocationForGateway
You can use those links in the different connection scenarios such as connection via SSH, via custom connection, or via browser.
For the detailed information on how to start your Remote Development session, refer to Connect and work with JetBrains Gateway.
Connect using SSH
Start JetBrains Gateway.
Select Connect via SSH.
Invoke the SSH settings and configure options for the already running backend.
JetBrains Gateway connects by SSH, gets the connection link from the existing backend. JetBrains Gateway forwards a port from the connection link to the local machine and prepares a new connection link with that local-to-client port.
It also downloads and launches JetBrains Client with this link. The JetBrains Client version is specified in the link. JetBrains Gateway operates in the background and forwards the port while JetBrains Client is still active.
Configure the custom connection
Start JetBrains Gateway.
Select some custom connection UI provided by the JetBrains Gateway plugin.
A custom plugin may connect to some cloud service, fetch a list of backends from that service, create backends, and so on.
JetBrains Gateway gets the connection link (in most cases it points to the local host and port) from the custom plugin, and with this link, downloads and launches JetBrains Client. The JetBrains Client version is specified in the link. JetBrains Gateway may stay in the background in case it should maintain connection between Client and Server.
Connect through browser
In the browser, click a specifically created link that describes a backend location.
In the case of SSH, it contains a username, a server name, the SSH port, project location on the server, and so on.
This link leads to a landing page that suggests opening JetBrains Gateway or downloading it if JetBrains Gateway is missing on Client. JetBrains Gateway handles link opening from the browser and connects to a specified backend.
Client-to-server workflow
The client-to-server workflow means that you don't have any IDE installed on your remote server. In this case, you first need to download and install JetBrains Gateway. For more detailed information, refer to Launch JetBrains Gateway and connect to a remote server.
Connect via SSH
Download and install JetBrains Gateway.
In the JetBrains Gateway wizard, select Connect via SSH to connect to a remote server.
On the next page of the wizard, select the server to which you want to connect. If there is no IDE on the remote server, JetBrains Gateway will download it.
Depending on the user preference, the IDE may be downloaded from JetBrains servers, uploaded from the Client machine, or downloaded from a custom location.
The RD protocol is specific to each version of the IDE backend. The matching version of JetBrains Client needs to be downloaded on the server.
After this, JetBrains Gateway starts the PyCharm server on the remote machine, establishes the TCP tunnel, and starts JetBrains Client with the appropriate arguments to have it connected.
The TCP connection link format is as follows:
tcp://127.0.0.1:PORT#jt=ONE_TIME_CONNECTION_TOKEN fp=SERVER_FINGERPRINT cb=THIN_CLIENT_BUILD jb=THIN_CLIENT_RUNTIME_VERSIONCheck the following example:
tcp://127.0.0.1:5990#jt=3f2f2471-84f2-4d8b-ac67-c1698f5a1be7 fp=ABC88C383B0A62654C4EF75052C60D4D475885075DCA5B85733BD8D4B9E28CC
0 cb=213.2667 jb=11_0_11b1620.1Extensibility
The IDE backend
The IDE backend is the full-blown JetBrains IDE, which is just being launched in a special headless unattended mode.
The backend can be extended with all the diversity of PyCharm plugins in the following ways:
By unpacking required plugins to
plugins/folder of the distributionBy running the following code (requires network connection to JetBrains Plugin Marketplace):
./bin/remote-dev-server.sh installPlugins <pathtoproject> <PLUGIN_ID1> <PLUGIN_ID2> ...
If a plugin provides the new set of inspections and features, all of those will be shown on JetBrains Client.
The only plugins that will not have any effect on Remote Development are the ones that aggressively modify the UI . Such plugins should be installed on the JetBrains Client side.
JetBrains Gateway SDK
JetBrains Gateway can be extended like any other IntelliJ platform-based product.
You can use one of the following ways:
Set up a new project with https://github.com/JetBrains/gradle-intellij-plugin/ (
gradle-intellij-pluginshould be >= 1.1.4)Use the following settings to build your plugin against JetBrains Gateway:
intellij { version.set("213.2667-CUSTOM-SNAPSHOT") type.set("GW") instrumentCode.set(false) }You may see available versions at https://www.jetbrains.com/intellij-repository/snapshots (see group com.jetbrains.gateway)
JetBrains Gateway / Thick PyCharm on a Client
Apart from the basic SSH and Code With Me connections, a vendor can customize JetBrains Gateway for its own orchestration service. This can be done within the custom deal between the JetBrains and the vendor.
JetBrains Gateway is based on the IntelliJ platform, it has APIs for connections and interactions with JetBrains Client.
Check the following example:
A big organization wants to write its own orchestration. The basic SSH flow is not enough due to security reasons. The organization writes an internal plugin and delivers it to its developers. Developers can install this plugin in JetBrains Gateway or in PyCharm on their laptops.
This is a very brief introduction of the APIs (they are not yet final and indeed this is not the full scope, but they explain the overall idea).
JetBrains Client
JetBrains Client is not designed to be extensible for the connection part. However, you can develop and install all the variety of PyCharm plugins, which modify the UI, keyboard shortcuts, themes, and other parts that touch the IDE UI interaction, but not its functionality.
Thanks for your feedback!