Shared indexes

Prepare a file storage

Sharing indexes from the command line requires that you have a file storage that will keep the exported project indexes.

• Make sure that you have a file storage ready, and all members of your team have access to this storage.

Install cdn-layout-tool

To be able to export and share project indexes, install cdn-layout-tool – a JetBrains tool for working with shared project indexes:

1. Download the latest version of the tool from the Space repository.

2. Once the .zip file is downloaded, extract its contents to a meaningful location.

   Among the .zip file's content, there's a bin directory. There, you will find an OS-specific startup script that you will need later for generating metadata.

   

Prepare the IDE for sharing project indexes

1. Create a folder where you want to store your shared indexes.

   For example, it can be the indexes directory in the root directory of your user (for example, /Users/jetbrains/).

2. Create the following directories or remove the contents from the existing ones:

   • ide-system

   • ide-config

   • ide-log

3. Copy the <IDE home>/bin/idea.properties file to the directory where you want to store your shared indexes (in our case, it is /Users/jetbrains/indexes). For more information on idea.properties, refer to Common properties.

   To find the <IDE home> directory, use the following default paths.

   • Windows: %APPDATA%\JetBrains\<product><version> (for example, C:\Users\JohnS\AppData\Roaming\JetBrains\GoLand2022.3).

   • macOS: ~/Library/Application Support/JetBrains/<product><version> (for example, ~/Library/Application Support/JetBrains/GoLand2022.3).

   • Linux: ~/.config/JetBrains/<product><version> (for example, ~/.config/JetBrains/GoLand2022.3).

4. Rename idea.properties to ide.properties.

5. Change the following properties in the ide.properties file Point these properties to directories that you created on Step 2. Consider the following example for macOS.

   • idea.system.path=/Users/jetbrains/indexes/ide-system

   • idea.config.path=/Users/jetbrains/indexes/ide-config

   • idea.log.path=/Users/jetbrains/indexes/ide-log

   If you don't have these properties in the file, add them at the end.

6. Set a temporary environment variable to specify the custom IDE properties file. To do so, run the following command:

   Windows

   Linux/macOS

   set GOLAND_PROPERTIES=<temp>=C:\indexes\ide.properties

   export GOLAND_PROPERTIES=<temp>=/Users/jetbrains/indexes/ide.properties

   The following screenshot shows the configuration process on macOS.

   

Export project indexes

1. In the directory with shared indexes, ensure that the generate-output directory is empty. If there's no such directory, do not create it as this will be done automatically during the export.

2. Run the IDE with the following command line to export the project indexes to generate-output.

   Consider the following example of the script for macOS. You can run this script in the Terminal tool window (View | Tool Windows | Terminal) in the IDE.

   goland dump-shared-index project --output=/Users/jetbrains/indexes/generate-output --tmp=/Users/jetbrains/indexes/temp --project-dir=/Users/jetbrains/git/go-metrics/ --project-id=go_metrics --commit=cf1acfcdf4751e0554ffa765d03e479ec491cad6

   Otherwise, consider the following approaches to run the IDE from the command line.

   Windows

   macOS

   Linux

   You can find the executable for running GoLand in the installation directory under bin. To use this executable as the command-line launcher, add it to your system PATH as described in Command-line interface.

   Syntax

   goland.bat dump-shared-index project --output=<temp>/generate-output --tmp=<temp>/temp --project-dir=<path> --project-id=<project-name> --commit=<Git HEAD>

   Options

   --output=<temp>/generate-output	Output location for the generated shared indexes in the <temp> folder.
   --tmp=<temp>/temp	The temp directory used for the internal needs of the IDE.
   --project-dir=<path>	The path to the project for which you want to export indexes.
   --project-id=<id>	The name of the project.
   --commit=<vcs revision>	The VCS revision identifier (for example, Git commit hash) representing the state of the project for which the shared index is being generated. This is needed to lay out the shared indexes of several VCS revisions in the file storage. This identifier is optional. If it is not specified, it will be set to -.
   --compression=<compression type>	Compress shared indexes. Possible options: xz, gzip, and plain.

   By default, GoLand does not provide a command-line launcher. For information about creating a launcher script for GoLand, see Command-line interface.

   Syntax

   goland dump-shared-index project --output=<temp>/generate-output --tmp=<temp>/temp --project-dir=<path> --project-id=<project-name> --commit=<Git HEAD>

   Options

   --output=<temp>/generate-output	Output location for the generated shared indexes in the <temp> folder.
   --tmp=<temp>/temp	The temp directory used for the internal needs of the IDE.
   --project-dir=<path>	The path to the project for which you want to export indexes.
   --project-id=<id>	The name of the project.
   --commit=<vcs revision>	The VCS revision identifier (for example, Git commit hash) representing the state of the project for which the shared index is being generated. This is needed to lay out the shared indexes of several VCS revisions in the file storage. This identifier is optional. If it is not specified, it will be set to -.
   --compression=<compression type>	Compress shared indexes. Possible options: xz, gzip, and plain.

   You can find the script for running GoLand in the installation directory under bin. To use this script as the command-line launcher, add it to your system PATH as described in Command-line interface.

   Syntax

   goland.sh dump-shared-index project --output=<temp>/generate-output --tmp=<temp>/temp --project-dir=<project> --project-id=<project-name> --commit=<Git HEAD>

   Options

   --output=<temp>/generate-output	Output location for the generated shared indexes in the <temp> folder.
   --tmp=<temp>/temp	The temp directory used for the internal needs of the IDE.
   --project-dir=<path>	The path to the project for which you want to export indexes.
   --project-id=<id>	The name of the project.
   --commit=<vcs revision>	The VCS revision identifier (for example, Git commit hash) representing the state of the project for which the shared index is being generated. This is needed to lay out the shared indexes of several VCS revisions in the file storage. This identifier is optional. If it is not specified, it will be set to -.
   --compression=<compression type>	Compress shared indexes. Possible options: xz, gzip, and plain.

   The export might take some time. When the process is finished, the following files are placed to <temp>/generate-output:

   • shared-index-project-<name>-<hash>.ijx.xz

   • shared-index-project-<name>-<hash>.metadata.json

   • shared-index-project-<name>-<hash>.sha256

   

Copy the generated files to a new folder

1. Create a new directory and download there all the contents of your remote file storage.

   You can create the new directory inside the directory that has generate-output subdirectory. This can be any directory, for example ~/indexes/project/<project name>/<VCS hash>/share.

2. Copy the generated files (.ijx.xz, .metadata.json, .sha256) from generate-output to the created subdirectory (in our case, share).

Generate metadata

• Generate auxiliary files used by the IDE to look up for and download shared indexes. To do so, run cdn-layout-tool with the following command line.

  Consider the following example command for macOS.

  /Users/jetbrains/Downloads/cdn-layout-tool--0.7.52/bin/cdn-layout-tool --indexes-dir=/Users/jetbrains/indexes/goProject/cf1acfcdf4751e0554ffa765d03e479ec491cad6/share --url=127.0.0.1

  This command uses the following format.

  <cdn-layout-tool> --indexes-dir=<dir> --url=<file storage root URL>

  Options

  <cdn-layout-tool>	The path to cdn-layout-tool startup script located in bin in the tool directory.
  --indexes-dir=<dir>	The directory containing all the files available in the file storage, as well as the newly generated shared indexes residing in a suitable location beneath this directory. The tool will generate the missing files and replace existing ones if necessary. After the tool has finished running, all the contents of this directory must be uploaded to the file storage. This will set the new state of the storage.
  --url=<file storage root URL>	The base URL of the file storage root directory. cdn-layout-tool will substitute the download URL in all the corresponding .json index lists.

  The following screenshot shows the output of the example command.

  

Upload the index files to your file storage

• Upload all the files from share to the file storage as is.

  If there are already uploaded index files in the storage, update them.

1. In the project directory, create a new file intellij.yaml and add the following code:

   sharedIndex:
       project:
           - url: <feed URL>

   Replace <feed URL> with the URL of your file storage in the following format: <file storage root URL>/project/<project-id>.

2. In the Settings/Preferences dialog (Ctrl+Alt+S), select Tools | Shared Indexes.

3. In the Shared Project Indexes area, select the way you want to download shared indexes from the storage.

   Select Download automatically to allow GoLand to download the JDK indexes silently whenever they are needed or select Ask before download if you prefer to confirm every download manually.

   

4. Apply the changes and close the dialog.

Project indexes will be downloaded to index/shared_indexes in the IDE system directory.