Shared indexes
One of the possible ways of reducing the indexing time is by using shared indexes. Unlike the regular indexes that are built locally, shared indexes are generated once and are later reused on another computer whenever they are needed.
tip
For more information about indexing and other ways of reducing the indexing time, refer to Indexing.
PyCharm can build and use shared indexes for your project's code. Whenever PyCharm needs to reindex your application, it will use the available shared indexes and will build local indexes for the rest of the project. Normally, this is faster than building local indexes for the entire application from scratch.
Shared project indexes are built for project sources and their dependencies. By generating indexes on a single computer and distributing them to other computers, you can avoid duplicating the indexing process on each machine. This way, each computer can benefit from the precomputed indexes without having to perform the expensive indexing operations individually. This is the main advantage of shared indexes over ordinary indexes.
Using shared indexes is reasonable for large projects, where indexing might take a lot of time, creating inconveniences for the teams involved. For smaller projects, we recommend other ways of reducing the indexing time.
To be able to use shared project indexes, the Shared Project Indexes bundled plugin must be enabled in the settings:
Press CtrlAlt0S to open settings and then select Plugins.
Switch to the Installed tab, type
Shared Project Indexes
, and make sure that the checkbox next to it is selected.Otherwise, select the checkbox to enable the plugin.
Apply the changes and close the dialog. Restart the IDE if prompted.
note
The Shared Project Indexes plugin is available in PyCharm Community Edition for 30 days. If you want to continue using the plugin, you can activate a license for PyCharm Professional, GoLand, PhpStorm, IntelliJ IDEA Ultimate, RubyMine, or WebStorm in your PyCharm Community Edition.
To be able to use shared project indexes, install the Shared Project Indexes plugin:
Press CtrlAlt0S to open settings and then select Plugins.
Switch to the Marketplace tab and type
Shared Project Indexes
in the search field.Once the plugin is found in the list of available plugins, click Install next to it.
Apply the changes and close the dialog. Restart the IDE if prompted.
The computer that builds project shared indexes and any computers that use the project shared indexes:
Should use the same IDE version to ensure index compatibility.
Can have different operating systems.
However, in previous PyCharm versions, shared project indexes were OS-specific. Refer to the documentation that corresponds to your IDE version by using the version switcher in the top-left corner of this page.
Before incorporating shared project indexes into your infrastructure, you can evaluate how much time they can save on indexing by running the command-line tool.
If you are satisfied with the results, you can keep using shared indexes. In more complex scenarios, you can customize sample scripts from the IntelliJ Shared Indexes Tool Example repository.
Before using shared project indexes in your workflow, you can see how much time they are able to save you. During evaluation, the tool will generate some files in the directory that you have specified.
Download the latest version of the command-line tool and unzip the archive.
In the command line, navigate to the directory of the command line tool and execute the binary file by running:
WindowsmacOSLinux.\bin\ij-shared-indexes-tool-cli.bat boost
./bin/ij-shared-indexes-tool-cli boost
./bin/ij-shared-indexes-tool-cli boost
with the following options:
--ij
: (mandatory) path to a locally installed IDE.note
The path to the locally installed IDE may differ depending on how it was installed.
--project
: (mandatory) path to the project for which you want to build shared indexes.--port
: port for running a local server. The default port is 25561.--data-directory
: directory for storing generated indexes and server data. The default directory is ./ij-shared-indexes-tool-data.
For example:
./bin/ij-shared-indexes-tool-cli boost --ij "/Users/jetbrains/Library/Application Support/JetBrains/Toolbox/apps/PyCharm-P/ch-1/232.7754.76/PyCharm 2023.2 EAP.app" --project "/Users/jetbrains/PycharmProjects/mySampleProject"
Wait for some time. After that, you will see the indexing time with shared indexes and without them.
By looking at the difference in time, you can decide whether you want to use shared project indexes in your workflow or not.
Use these steps to generate index files. After the indexes have been generated, you will need to upload the files to a file server or S3-compatible storage.
Download the latest version of the command-line tool and unzip the archive.
In the command line, navigate to the directory of the command line tool and execute the binary file via the command line by running:
WindowsmacOSLinux.\bin\ij-shared-indexes-tool-cli.bat indexes
./bin/ij-shared-indexes-tool-cli indexes
./bin/ij-shared-indexes-tool-cli indexes
with the following options:
--ij
: (mandatory) path to a locally installed IDE.--project
: (mandatory) path to the project for which you want to build shared indexes.--base-url
: (mandatory) Indexes server URL. The base URL that will be used to define the full URL where shared indexes are hosted.--data-directory
: directory for storing generated indexes and server data. The default directory is ./ij-shared-indexes-tool-data. You should upload files from this directory to the server specified in thebase-url
parameter.
Once the project indexes have been generated, you will see a message with the information required to configure shared indexes in your project.
After you have generated indexes, two folders appear in the directory you have specified when executing the indexes
command: server and temp. The index files are stored within the server directory.
After you have generated indexes, two folders appear in the directory you have specified when executing the indexes
command: server and temp. The index files are stored within the server directory.
Upload the data and the project folders from the server directory to your file server or S3-compatible storage. The storage location should have the same base URL as the URL that was defined using the
base-url
parameter when generating the indexes.
When testing shared indexes, you can use the server
command to start a server on the generated files.
In the command line, navigate to the directory of the command line tool and execute the binary file by running:
WindowsmacOSLinux.\bin\ij-shared-indexes-tool-cli.bat server
./bin/ij-shared-indexes-tool-cli server
./bin/ij-shared-indexes-tool-cli server
with the following options:
--port
: port for running a local server.--server-directory
: directory for server data. The default directory is ./ij-shared-indexes-tool-data ./server
For example:
WindowsmacOSLinux.\bin\ij-shared-indexes-tool-cli.bat server --port 8080 --server-directory "C:\Users\jetbrains\ij-shared-indexes-tool-data\server"
./bin/ij-shared-indexes-tool-cli server --port 8080 --server-directory "/Users/jetbrains/ij-shared-indexes-tool-data/server"
./bin/ij-shared-indexes-tool-cli server --port 8080 --server-directory "/home/jetbrains/ij-shared-indexes-tool-data/server"
Once the project indexes are uploaded to the file storage, they can be downloaded and applied on another computer.
In the project directory, create a new file intellij.yaml and copy the information from the terminal to the new file, for example:
If the project hasn't been indexed yet or you have cleared the system cache, the IDE will download the shared indexes and use them.
In the default configuration, the IDE will download the shared indexes automatically. You may choose to change this and confirm every download manually.
Project indexes will be downloaded to index/shared_indexes in the IDE system directory.
tip
Place the <project home>/intellij.yaml file under VCS so that other developers in your team can get access to the project shared indexes.
When the indexes are being downloaded, the progress is displayed in the status bar.
When everything is ready, you will see a corresponding notification in the Notifications tool window.
In the default configuration, the IDE will be downloading shared indexes automatically. You can confirm every download manually.
From the intellij.yaml file, remove the
consents
section and restart your project.In the Settings dialog (CtrlAlt0S) , select Tools | Shared Indexes and in the Project Shared Indexes area, select Ask before download.
Apply the changes and close the dialog.
- boost
Measures indexing speed up with project shared indexes. Options:
--project
: (mandatory) Path to a project.--ij
: (mandatory) Path to a locally installed IDE.--port
: Port for running a local server. The default port is 25561.--data-directory
: Directory to use for generation and server data.
- indexes
Generates project shared indexes and makes them ready to be uploaded. Options:
--project
: (mandatory) Path to a project.--ij
: (mandatory) Path to a locally installed IDE.--base-url
: (mandatory) Indexes server URL.--data-directory
: Directory to use for generation and server data.
- server
Starts a server on local indexes. Options:
--port
: port for running a local server. The default port is 25561.--server-directory
: directory for server data. The default directory is ./ij-shared-indexes-tool-data ./server
- version
Displays shared indexes version information. Options:
--ij
: (mandatory) path to a locally installed IDE..
You can keep using shared indexes from the command line. In more complex scenarios, you can customize sample scripts from the IntelliJ Shared Indexes Tool Example repository.
Thanks for your feedback!