How to Identify the cause of a Cisco Tomcat memory leak using the heap dump file.

If Tomcat has used up all its allocated memory (aka heap) and is unable to free up space then the process will generate a heap dump and crash out.

Heapdumps are useful as they are a snapshot of all the objects in the processes heap when it died. The heapdump can be identified as a .hprof file and will typically be ~500MB.

You can download the dump file by using RTMT. Simply goto:

Trace & Log Central -> Remote Browse -> Trace Files -> Cisco CallManager -> Cisco Tomcat, then select the CCM server on which Cisco Tomcat crashed, drill down into System -> Cisco Tomcat -> Logs, there you should see the .hprof file in the file list.

 

 

 

 

 

 

Download the file to your PC ensuring you choose the option to zip the file before downloading.

Once you have the file you need to analyse it, for this you need a program called “Eclipse Memory Analyser Tool (MAT)” this is a free program available here.

NOTE: Ensure you have the correct java JRE version installed, i.e. if you downloaded the 64bit version of MAT make sure you have the 64bit JRE installed (64bit JRE can be downloaded here if you need it!). 

Once you have MAT running choose the option to “Open a Heap Dump”:

Select the .hprof file you downloaded earlier. It will take some time to import. Once it’s ready it will present you with the “Getting Started Wizard”, from choose the “Leak Suspects Report” and click finish. Once the report is generated it will identify the likely source objects of the leak as well as their heap utilisation, it should look something like this:

You can see in this case the issue was with com.rsa.sslj.x.cu, this corresponds with a known bug CSCty36110 due to an issue with AXL’s interaction with Tomcat. AXL was being used by their PhoneX One system.

Leave a Reply

Your email address will not be published. Required fields are marked *