Troubleshooting Upsource cluster

Upsource Cluster doesn't start/restart with the following error:

This is a known integration issue between swarm and docker-compose. The error occurs when the swarm manager tries to recreate a service container that defines port mapping to the host machine port. See this article for details.

Solution:

Remove haproxy container before starting the cluster:

Swarm cluster issues

All nodes marked as Unknown in the docker output

Solution:

The problem may be caused by closed ports on some of the machines. Ensure that all ports used by the swarm cluster are opened. For instance:

• Docker engine of the swarm cluster listens to some tcp ports (2375, 2376 by default)
• Swarm master listens to some port (4000 by default)
• Key-value storage listens to some port (8500 is the default for consul)

To check if the ports are open, run:

Upsource services don't see each other

Solution:

To check whether services see each other or not, you can issue a GET request to http://<address.of.node.where.opscenter.is.running>:10080/monitoring/frontends. It should return a non-empty list if the cluster is running.

If the link is not available it could mean that the haproxy/opscenter services are not finished starting yet.

If the link returns an empty list, then most likely a connection between services has not been established (note that the list might be empty for a short period of time on startup while the frontend service is not yet initialized).