How to Debug problems

To fix a problem, we may need versatile information about your system and various logs. Below you may find hints on how to collect this information from you system when trouble comes.

Logging



TeamCity (both server and build agents) uses Log4j as a logger.

Logging rules and Log4j configuration files placement for TeamCity 2 are described in Logs of Internal Events.

By default, Log4J configuration file is TeamCity/conf/teamcity-server-log4j.xml for the server, and buildAgent/conf/teamcity-agent-log4j.xml for build agents.

Before reproducing the problem it makes sense to enable 'DEBUG' log level for TeamCity classes. To do it, change the following section in teamcity-(server|agent)-log4j.xml files:

<category name="jetbrains.buildServer">
  <priority value="DEBUG"/>
  <appender-ref ref="ROLL"/>
</category>

After that, DEBUG messages will go to teamcity-*.log files.

Hangs and thread dumps



In case you experience problems with TeamCity server (e.g. no responding or working too slow) we would appreciate a thread dump of the server process. On Windows, press Ctrl-Break in console window (Or use http://www.adaptj.com/root/main/download). On Linux, run 'jstack <pid_of_java_process>' or 'kill -3 <pid_of_java_process>'. Output will appear in TeamCity/logs/catalina.out.

OutOfMemory problems



In case you experience problems with TeamCity "eating" too much memory (OutOfMemory errors), please do the following:

• determine what process encounters the error (the actual building process, TeamCity server, or TeamCity agent)

• try to increase the memory for the process via -Xmx JVM option, like -Xmx512m. The option needs to be passed to the process with problems:

  • if it is the building process itself, use "JVM Command Line Parameters" settings in the build runner. e.g. Inspections builds may specifically need to increase the parameter;

  • if it is TeamCity server, modify runAll.* script or (if server is run as Windows service), use TeamCity/bin/tomcat5w.exe //ES//TeamCity (see more);

  • if it is TeamCity build agent, modify agent.* script

• if increasing memory does not help, please get the memory dump and send it to us for further analysis:

  • to get a memory dump (hprof file) when OOM error occurs, please add the following JVM option (works for JDK 1.5.0_07+): -XX:+HeapDumpOnOutOfMemoryError;

  • when OOM error occurs next time, java_xxx.hprof file will be created in the process startup directory (TeamCity/bin or buildAgent/bin);

  • please archive and upload this file via https://uploads.jetbrains.com;

  • let us know about the problem via creating an issue in Jira. Don't forget to provide name of file and version of TeamCity.

SVN Debug Logging



To turn on debug logging of SVNkit library, uncomment the following lines in the <TeamCity_server_or_agent_home>\conf\teamcity-(server|agent)-log4j.xml file:

<category name="javasvn.output">
  <priority value="DEBUG"/>
  <appender-ref ref="SVN.LOG"/>
</category>

Do it on server for server-side issues (e.g. when "VCS checkout mode" is set to "Automatically on server"), or on agent (when "Automatically on agent" checkout mode is used) for the build configuration. Then, restart the server or agent, correspondingly.

The log will be saved to the logs\teamcity-svn.log file.

CVS Debug Logging



To turn on debug logging for CVS commands, run server with -Dcvs.log.commands=true JVM parameter.

Remote Run problems



How to retrieve the "patch" file with the changes of a remote run:

• add property agent.save.patch=true to the build configuration properties: Administration->Edit configuration->Properties and requirement variables->Add new property...

• try personal build again.

Agent log will contain line "Patch is saved to file ${file.name}" Get the file and provide it with the problem description.