Scaling Upsource services in the cluster
You should consider scaling Upsource services if you experience server performance issues because of increasing number of projects or users, or you want your distributed system to be fault-tolerant 24/7.
Scaling PSI and frontend
-
To scale PSI, run:
./cluster.sh -H tcp://<swarm.master.host>:<swarm.master.port> scale psi=N -
To scale frontend, run:
./cluster.sh -H tcp://<swarm.master.host>:<swarm.master.port> scale frontend=N
Scaling analyzer
The default setup provides for a single analyzer that covers all Upsource projects. Let's say you have ten projects, nine of which are small to medium-scale projects that see several commits a day, while the last one is a huge project with hundreds of commits every day. Overall indexing performance might be improved by moving that resource-intensive project to a separate analyzer, relieving the original analyzer from stress.
When you decide to add more analyzer instances, you split all of your projects in groups (subsets) consisting of one or more projects each. For each subset there should be an analyzer defined as a separate service in docker-compose.xml and started with its own unique configuration. Different analyzers cannot share the same project — i.e. the project can only be included in one subset.
The subset of projects the analyzer works on is defined by the environment variable ANALYZER_PROJECTS. Its value is specified with a mask using the following format:
+: include
-: exclude
.+ all projects
For example:
- ANALYZER_PROJECTS=+:.+ include all projects (process all projects)
- ANALYZER_PROJECTS=-:.+,+:Project-A exclude all projects, include Project-A (process Project-A only)
- ANALYZER_PROJECTS=+:.+,-:Project-A include all projects, exclude Project-A (process everything except for Project-A)
- ANALYZER_PROJECTS=-:.+,+:Project-A,+:Project-B exclude all projects, include Project-A, include Project-B (process Project-A and Project-B only)
The default singleton analyzer service in docker-compose.yml has no definition for ANALYZER_PROJECTS and therefore processes all projects registered in Upsource).
To add an analyzer node:
-
Stop the analyzer service:
./cluster.sh -H tcp://<swarm.master.host>:<swarm.master.port> stop analyzer ./cluster.sh -H tcp://<swarm.master.host>:<swarm.master.port> rm analyzer -
Add a new service to docker-compose.yml and redefine scopes of the projects each analyzer would work on. Two analyzers can't work on the same project!
For instance, if you have a large project A and would like to supply a separate analyzer instance for that project, then the following two analyzer services should be defined instead of one service defined by default.
Please note that these two services have different values for their properties:
analyzer-A: image: jetbrains/upsource-analyzer:${UPSOURCE_VERSION} container_name: analyzer_A restart: unless-stopped env_file: - ./upsource.env networks: up_net: aliases: - analyzer-A expose: - 10061 environment: - ANALYZER_PROJECTS=-:.+,+:A - UPSOURCE_SUBNET=${UPSOURCE_SUBNET} volumes: - /var/log/upsource/analyzer-A:/opt/upsource-analyzer/logs analyzer-other: image: jetbrains/upsource-analyzer:${UPSOURCE_VERSION} container_name: analyzer_other restart: unless-stopped env_file: - ./upsource.env networks: up_net: aliases: - analyzer-other expose: - 10061 environment: - ANALYZER_PROJECTS=+:.+,-:A - UPSOURCE_SUBNET=${UPSOURCE_SUBNET} volumes: - /var/log/upsource/analyzer-other:/opt/upsource-analyzer/logsNote that newly mapped directories on the host machine must have a read-write access granted to the user jetbrains with ID 13001.
For the above example, the following command should be executed on every swarm cluster node to grant the access:
mkdir -p -m 750 /var/log/upsource/analyzer-A chown 13001:13001 /var/log/upsource/analyzer-A mkdir -p -m 750 /var/log/upsource/analyzer-other chown 13001:13001 /var/log/upsource/analyzer-other -
Stop and delete the old analyzer service(s).
./cluster.sh -H tcp://<swarm.master.host>:<swarm.master.port> stop analyzer./cluster.sh -H tcp://<swarm.master.host>:<swarm.master.port> rm analyzer
You can stop and delete several services at once.
-
Start your new analyser service(s)
./cluster.sh -H tcp://<swarm.master.host>:<swarm.master.port> upThis command will not affect the services which configurations were not changed. It will deploy only new services.
Services that cannot be scaled
opscenter and haproxy are singleton services and therefore can't be scaled.