A JVM will be greyed out in Introscope when there is a communication issue between the JVM and Introscope. In this example, JVM2 on Tomcat server2 is greyed out.
Is the JVM running?
The most likely cause of this issue is that the JVM has been stopped. Use the ps command can be used to determine if the JVM is running.
ps -ef | grep <jvm>
Is the JVM "shut off"
Right-clicking on a JVM in Introscope will produce the following dialog box. If "Shut Off jvm" is selected and check marked, the JVM will remain greyed out in Introscope. Simply removing the check mark may be all that is need to return the JVM to the blue status.
If the JVM is running on Tomcat, the JVMs catalina.out log may catch the communication issue between the JVM and Introscope.
9/25/18 11:34:18 PM CDT [ERROR] [IntroscopeAgent.IncomdingRouteConnector] Closing incoming route connector due to unrecoverable error.
A communication issue between the JVM and Introscope can be due to the JVM being misconfigured. In this example, the JVM could not be initialized due to the JVM using an invalid certificate alias for SSL.
28-Sep-2018 01:01:59.124 SEVERE [main] org.apache.catalina.core.StandardService.initInternal Failed to initialize connector [Connector[HTTP/1.1-12345]] Caused by: java.lang.IllegalArgumentException: java.io.IOException: Alias name [*.example.com] does not identify a key entry
A communication issue between the JVM and Introscope can be the result of a JVM running out of memory. Out of memory events should be caught in the JVMs catalina.out log (Tomcat).
java.lang.OutOfMemoryError: PermGen space java.lang.OutOfMemoryError: Java heap space java.lang.OutOfMemoryError: GC overhead limit exceeded
If the cause of the communication issue is that the JVM ran out of memory, restarting the JVM should resolve the communication issue. However, this should only be considered a stop gap measure - the cause of the out of memory situation should be analyzed.
When a JVM runs out of memory, refer to the following:
- Analyze and respond to a Mule heap dump
- Analyze and respond to a Tomcat heap dump
- Analyze and respond to a WebSphere heap dump