NRPE sudo check_jvm not working on RHEL 8 or Debian 10

[drakedts]

Here's output from the RHEL 8 machine (command adjusted slightly for system-specific differences):

Code: Select all

# /usr/bin/sudo -u tomcat8 bash -x /usr/lib64/nagios/plugins/check_jvm -n org.apache.catalina.startup.Bootstrap -p heap -w 90 -c 101
+ JVMINSPECTOR=/usr/local/libexec/JvmInspector.jar
+ export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ export ENV=
+ ENV=
+ export CDPATH=
+ CDPATH=
+ STATE_OK=0
+ STATE_WARNING=1
+ STATE_CRITICAL=2
+ STATE_UNKNOWN=3
+ MSG_OK=OK
+ MSG_WARNING=WARNING
+ MSG_CRITICAL=CRITICAL
+ MSG_UNKNOWN=UNKNOWN
++ basename /usr/lib64/nagios/plugins/check_jvm
+ SCRIPT_NAME=check_jvm
+ '[' 8 -eq 0 ']'
+ '[' '!' -z -n ']'
+ case $1 in
+ shift
+ NAME=org.apache.catalina.startup.Bootstrap
+ shift
+ '[' '!' -z -p ']'
+ case $1 in
+ shift
+ PROPERTY=heap
+ shift
+ '[' '!' -z -w ']'
+ case $1 in
+ shift
+ WARNING=90
+ shift
+ '[' '!' -z -c ']'
+ case $1 in
+ shift
+ CRITICAL=101
+ shift
+ '[' '!' -z '' ']'
+ '[' -z org.apache.catalina.startup.Bootstrap ']'
+ '[' -z heap ']'
+ '[' -z 90 ']'
+ '[' -z 101 ']'
+ expr 90 : '[0-9]\+$'
+ expr 101 : '[0-9]\+$'
+ '[' -f /usr/local/libexec/JvmInspector.jar ']'
++ grep org.apache.catalina.startup.Bootstrap
++ head -1
+++ tr ' ' ,
+++ head -1
++++ pgrep java
+++ echo 963385
++ ps axo pid,uid,command -q 963385
+ PSLINE=' 963385   982 /usr/lib/jvm/jre/bin/java -Xms14336M -Xmx14336M -Dbanner.logging.dir=/var/log/tomcat8 -XX:MaxMetaspaceSize=2048m -classpath /usr/share/tomcat8/bin/bootstrap.jar:/usr/share/tomcat8/bin/tomcat-juli.jar: -Dcatalina.base=/usr/share/tomcat8 -Dcatalina.home=/usr/share/tomcat8 -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat8/temp -Djava.util.logging.config.file=/usr/share/tomcat8/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start'
++ echo ' 963385   982 /usr/lib/jvm/jre/bin/java -Xms14336M -Xmx14336M -Dbanner.logging.dir=/var/log/tomcat8 -XX:MaxMetaspaceSize=2048m -classpath /usr/share/tomcat8/bin/bootstrap.jar:/usr/share/tomcat8/bin/tomcat-juli.jar: -Dcatalina.base=/usr/share/tomcat8 -Dcatalina.home=/usr/share/tomcat8 -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat8/temp -Djava.util.logging.config.file=/usr/share/tomcat8/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start'
++ awk '{print $1}'
+ PID=963385
++ echo ' 963385   982 /usr/lib/jvm/jre/bin/java -Xms14336M -Xmx14336M -Dbanner.logging.dir=/var/log/tomcat8 -XX:MaxMetaspaceSize=2048m -classpath /usr/share/tomcat8/bin/bootstrap.jar:/usr/share/tomcat8/bin/tomcat-juli.jar: -Dcatalina.base=/usr/share/tomcat8 -Dcatalina.home=/usr/share/tomcat8 -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat8/temp -Djava.util.logging.config.file=/usr/share/tomcat8/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager org.apache.catalina.startup.Bootstrap start'
++ awk '{print $2}'
+ PUID=982
+ '[' -z 963385 ']'
+ expr 963385 : '[0-9]\+$'
+ '[' 982 = 982 ']'
+ TIMEOUT=
+ timeout --version
+ TIMEOUT='timeout 7'
++ timeout 7 java -jar /usr/local/libexec/JvmInspector.jar 963385
+ JVMDATA='
JVM pid: 963385
  name: org.apache.catalina.startup.Bootstrap start
  thread count: 157 (peak: 165)
  class count: 140503
  heap memory: max=14903934976|commited=14903934976|used=7686867520
  non-heap memory: max=3472883712|commited=1766739968|used=1670093696
  java version: 1.8.0_272 (Red Hat, Inc.)
  java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-3.el8_3.x86_64/jre
  jvm arguments: [-Xms14336M, -Xmx14336M, -Dbanner.logging.dir=/var/log/tomcat8, -XX:MaxMetaspaceSize=2048m, -Dcatalina.base=/usr/share/tomcat8, -Dcatalina.home=/usr/share/tomcat8, -Djava.endorsed.dirs=, -Djava.io.tmpdir=/var/cache/tomcat8/temp, -Djava.util.logging.config.file=/usr/share/tomcat8/conf/logging.properties, -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager]
  class path: /usr/share/tomcat8/bin/bootstrap.jar:/usr/share/tomcat8/bin/tomcat-juli.jar:
  server container: Apache Tomcat/8.5.61
  active sessions: total=0|/FinanceSelfService=0|/=0|/StudentSelfService=0|/FacultySelfService=0|/BannerExtensibility=0|/BannerGeneralSsb=0|/EmployeeSelfService=0'
+ '[' 0 -ne 0 ']'
+ echo '
JVM pid: 963385
  name: org.apache.catalina.startup.Bootstrap start
  thread count: 157 (peak: 165)
  class count: 140503
  heap memory: max=14903934976|commited=14903934976|used=7686867520
  non-heap memory: max=3472883712|commited=1766739968|used=1670093696
  java version: 1.8.0_272 (Red Hat, Inc.)
  java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-3.el8_3.x86_64/jre
  jvm arguments: [-Xms14336M, -Xmx14336M, -Dbanner.logging.dir=/var/log/tomcat8, -XX:MaxMetaspaceSize=2048m, -Dcatalina.base=/usr/share/tomcat8, -Dcatalina.home=/usr/share/tomcat8, -Djava.endorsed.dirs=, -Djava.io.tmpdir=/var/cache/tomcat8/temp, -Djava.util.logging.config.file=/usr/share/tomcat8/conf/logging.properties, -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager]
  class path: /usr/share/tomcat8/bin/bootstrap.jar:/usr/share/tomcat8/bin/tomcat-juli.jar:
  server container: Apache Tomcat/8.5.61
  active sessions: total=0|/FinanceSelfService=0|/=0|/StudentSelfService=0|/FacultySelfService=0|/BannerExtensibility=0|/BannerGeneralSsb=0|/EmployeeSelfService=0'
+ grep 'class count'
+ '[' heap = threads ']'
+ '[' heap = classes ']'
+ '[' heap = heap ']'
++ printf %s '
JVM pid: 963385
  name: org.apache.catalina.startup.Bootstrap start
  thread count: 157 (peak: 165)
  class count: 140503
  heap memory: max=14903934976|commited=14903934976|used=7686867520
  non-heap memory: max=3472883712|commited=1766739968|used=1670093696
  java version: 1.8.0_272 (Red Hat, Inc.)
  java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-3.el8_3.x86_64/jre
  jvm arguments: [-Xms14336M, -Xmx14336M, -Dbanner.logging.dir=/var/log/tomcat8, -XX:MaxMetaspaceSize=2048m, -Dcatalina.base=/usr/share/tomcat8, -Dcatalina.home=/usr/share/tomcat8, -Djava.endorsed.dirs=, -Djava.io.tmpdir=/var/cache/tomcat8/temp, -Djava.util.logging.config.file=/usr/share/tomcat8/conf/logging.properties, -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager]
  class path: /usr/share/tomcat8/bin/bootstrap.jar:/usr/share/tomcat8/bin/tomcat-juli.jar:
  server container: Apache Tomcat/8.5.61
  active sessions: total=0|/FinanceSelfService=0|/=0|/StudentSelfService=0|/FacultySelfService=0|/BannerExtensibility=0|/BannerGeneralSsb=0|/EmployeeSelfService=0'
++ awk 'BEGIN { FS = ": " } ;/^  heap memory/{print $2}'
+ TEMPDATA='max=14903934976|commited=14903934976|used=7686867520'
++ printf %s 'max=14903934976|commited=14903934976|used=7686867520'
++ awk 'BEGIN { FS="|" } {print $1}'
++ awk 'BEGIN { FS="=" } {print $2}'
+ MAX=14903934976
++ printf %s 'max=14903934976|commited=14903934976|used=7686867520'
++ awk 'BEGIN { FS="|" } {print $2}'
++ awk 'BEGIN { FS="=" } {print $2}'
+ COMMITED=14903934976
++ printf %s 'max=14903934976|commited=14903934976|used=7686867520'
++ awk 'BEGIN { FS="|" } {print $3}'
++ awk 'BEGIN { FS="=" } {print $2}'
+ USED=7686867520
+ RESULT=7686867520
++ echo 7686867520
++ numfmt --to=iec
+ FRESULT=7.2G
+ PERFDATA='max=14903934976;;; commited=14903934976;;; used=7686867520;;;'
+ '[' -z 7686867520 ']'
+ expr 7686867520 : '-\?[0-9]\+$'
+ '[' 7686867520 -ge 101 ']'
+ p_critical '7.2G |max=14903934976;;; commited=14903934976;;; used=7686867520;;;'
+ echo 'CRITICAL 7.2G |max=14903934976;;; commited=14903934976;;; used=7686867520;;;'
CRITICAL 7.2G |max=14903934976;;; commited=14903934976;;; used=7686867520;;;
+ exit 2

And here's what i get on Debian:

Code: Select all

# /usr/bin/sudo -u tomcat bash -x /usr/lib/nagios/plugins/check_jvm -n org.apache.catalina.startup.Bootstrap -p heap -w 90 -c 101
+ JVMINSPECTOR=/usr/local/libexec/JvmInspector.jar
+ export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ export ENV=
+ ENV=
+ export CDPATH=
+ CDPATH=
+ STATE_OK=0
+ STATE_WARNING=1
+ STATE_CRITICAL=2
+ STATE_UNKNOWN=3
+ MSG_OK=OK
+ MSG_WARNING=WARNING
+ MSG_CRITICAL=CRITICAL
+ MSG_UNKNOWN=UNKNOWN
++ basename /usr/lib/nagios/plugins/check_jvm
+ SCRIPT_NAME=check_jvm
+ '[' 8 -eq 0 ']'
+ '[' '!' -z -n ']'
+ case $1 in
+ shift
+ NAME=org.apache.catalina.startup.Bootstrap
+ shift
+ '[' '!' -z -p ']'
+ case $1 in
+ shift
+ PROPERTY=heap
+ shift
+ '[' '!' -z -w ']'
+ case $1 in
+ shift
+ WARNING=90
+ shift
+ '[' '!' -z -c ']'
+ case $1 in
+ shift
+ CRITICAL=101
+ shift
+ '[' '!' -z '' ']'
+ '[' -z org.apache.catalina.startup.Bootstrap ']'
+ '[' -z heap ']'
+ '[' -z 90 ']'
+ '[' -z 101 ']'
+ expr 90 : '[0-9]\+$'
+ expr 101 : '[0-9]\+$'
+ '[' -f /usr/local/libexec/JvmInspector.jar ']'
++ head -1
++++ pgrep java
+++ tr ' ' ,
+++ head -1
++ grep org.apache.catalina.startup.Bootstrap
+++ echo 645
++ ps axo pid,uid,command -q 645
+ PSLINE='  645   995 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Djava.util.logging.config.file=/var/lib/tomcat9/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xms4096M -Xmx4096M -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dignore.endorsed.dirs= -classpath /usr/share/tomcat9/bin/bootstrap.jar:/usr/share/tomcat9/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat9 -Dcatalina.home=/usr/share/tomcat9 -Djava.io.tmpdir=/tmp org.apache.catalina.startup.Bootstrap start'
++ echo '  645   995 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Djava.util.logging.config.file=/var/lib/tomcat9/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xms4096M -Xmx4096M -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dignore.endorsed.dirs= -classpath /usr/share/tomcat9/bin/bootstrap.jar:/usr/share/tomcat9/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat9 -Dcatalina.home=/usr/share/tomcat9 -Djava.io.tmpdir=/tmp org.apache.catalina.startup.Bootstrap start'
++ awk '{print $1}'
+ PID=645
++ echo '  645   995 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Djava.util.logging.config.file=/var/lib/tomcat9/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xms4096M -Xmx4096M -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dignore.endorsed.dirs= -classpath /usr/share/tomcat9/bin/bootstrap.jar:/usr/share/tomcat9/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat9 -Dcatalina.home=/usr/share/tomcat9 -Djava.io.tmpdir=/tmp org.apache.catalina.startup.Bootstrap start'
++ awk '{print $2}'
+ PUID=995
+ '[' -z 645 ']'
+ expr 645 : '[0-9]\+$'
+ '[' 995 = 995 ']'
+ TIMEOUT=
+ timeout --version
+ TIMEOUT='timeout 7'
++ timeout 7 java -jar /usr/local/libexec/JvmInspector.jar 645
+ JVMDATA=
+ '[' 0 -ne 0 ']'
+ grep 'class count'
+ echo ''
+ p_unknown 'Can'\''t connect to the JVM: '
+ echo 'UNKNOWN Can'\''t connect to the JVM: '
UNKNOWN Can't connect to the JVM: 
+ exit 3

[dchurch]

Seems to be working on RHEL8, just not Debian 10.

Evidently java is having some trouble running the JvmInspector JAR file. Make sure your version of Java installed can run the class file.

The JvmInspector JAR I noticed hasn't been updated in a long time. I had trouble running it on the latest version of the JRE available in public repos.

Please log in to the Debian 10 machine and run the following command:

Code: Select all

java -version

[drakedts]

Yes, on RHEL 8 the command works except when run from NRPE.

The Debian machine runs Java 11 and has the Java 11 version of JvmInspector that was just released a month ago:

Code: Select all

# java -version
openjdk version "11.0.9.1" 2020-11-04
OpenJDK Runtime Environment (build 11.0.9.1+1-post-Debian-1deb10u2)
OpenJDK 64-Bit Server VM (build 11.0.9.1+1-post-Debian-1deb10u2, mixed mode, sharing)
# ls -l /usr/local/libexec/JvmInspector.jar
-r--r--r-- 1 root root 7224 2020-12-15 12:23:18 /usr/local/libexec/JvmInspector.jar

[dchurch]

Since Debian 10 fails consistently from the command line, let's check if it's failing due to some environment issue:

Code: Select all

su - tomcat8
env -i timeout 7 java -jar /usr/local/libexec/JvmInspector.jar 645

[drakedts]

OK, i tried that. JvmInspector doesn't return any output. I tried trimming the command down to just the java part and it still returns no output. Then i used ps to show that the PID 645 was correct; it is the Tomcat process, and is the only Java process running. So would it be fair to say that the JvmInspector is broken?

Code: Select all

[root@lnx-cms-test ~]# su - tomcat
[tomcat@lnx-cms-test /]$ env -i timeout 7 java -jar /usr/local/libexec/JvmInspector.jar 645
[tomcat@lnx-cms-test /]$ echo $?
0
[tomcat@lnx-cms-test /]$ timeout 7 java -jar /usr/local/libexec/JvmInspector.jar 645
[tomcat@lnx-cms-test /]$ echo $?
0
[tomcat@lnx-cms-test /]$ java -jar /usr/local/libexec/JvmInspector.jar 645
[tomcat@lnx-cms-test /]$ echo $?
0
[tomcat@lnx-cms-test /]$ ps ax | grep 645
  645 ?        Ssl  4168:16 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Djava.util.logging.config.file=/var/lib/tomcat9/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xms4096M -Xmx4096M -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dignore.endorsed.dirs= -classpath /usr/share/tomcat9/bin/bootstrap.jar:/usr/share/tomcat9/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat9 -Dcatalina.home=/usr/share/tomcat9 -Djava.io.tmpdir=/tmp org.apache.catalina.startup.Bootstrap start
 6600 pts/0    S+     0:00 grep --color=auto 645
[tomcat@lnx-cms-test /]$ ps ax | grep java
  645 ?        Ssl  4168:53 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Djava.util.logging.config.file=/var/lib/tomcat9/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xms4096M -Xmx4096M -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dignore.endorsed.dirs= -classpath /usr/share/tomcat9/bin/bootstrap.jar:/usr/share/tomcat9/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat9 -Dcatalina.home=/usr/share/tomcat9 -Djava.io.tmpdir=/tmp org.apache.catalina.startup.Bootstrap start
 6620 pts/0    S+     0:00 grep --color=auto java

[dchurch]

It might be an environment variable problem.

Can you run this on the Debian 10 machine?

Code: Select all

su -s /bin/bash tomcat8
env -i timeout 7 java -jar /usr/local/libexec/JvmInspector.jar 645

[drakedts]

No luck on that one, JvmInspector still returns no output.

Code: Select all

[root@lnx-cms-test /]# su -s /bin/bash tomcat
tomcat@lnx-cms-test:/$ env -i timeout 7 java -jar /usr/local/libexec/JvmInspector.jar 645
tomcat@lnx-cms-test:/$ echo $?
0

[ssax]

This is likely why it's failing:

Code: Select all

[root@lnx-cms-test /]# su -s /bin/bash tomcat
tomcat@lnx-cms-test:/$ env -i timeout 7 java -jar /usr/local/libexec/JvmInspector.jar 645
tomcat@lnx-cms-test:/$ echo $?
0

So that env -i is saying to ignore all envars so this must be envar/path related. The problem is that the developer of the JvmInspector.jar doesn't have the source code listed so your guess is as good as mine to what it's missing.

You can either reach out to the developer of the plugin to determine what environmental variables it requires or you can try the envars that this shows:

Code: Select all

su - tomcat8
env

Then add them to the check_jvm plugin (see the other exports in there) until it works:

Code: Select all

export XXXX="XXXX"

[drakedts]

I'm not sure how adding variables from Tomcat's account will help, since JvmInspector doesn't work under the Tomcat account either (note that the PID changed from before because i rebooted the system recently):

Code: Select all

[root@lnx-cms-test ~]# su - tomcat
[tomcat@lnx-cms-test /]$ ps ax | grep java
  639 ?        Ssl  384:21 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Djava.util.logging.config.file=/var/lib/tomcat9/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xms4096M -Xmx4096M -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dignore.endorsed.dirs= -classpath /usr/share/tomcat9/bin/bootstrap.jar:/usr/share/tomcat9/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat9 -Dcatalina.home=/usr/share/tomcat9 -Djava.io.tmpdir=/tmp org.apache.catalina.startup.Bootstrap start
28128 pts/0    S+     0:00 grep --color=auto java
[tomcat@lnx-cms-test /]$ java -jar /usr/local/libexec/JvmInspector.jar 639
[tomcat@lnx-cms-test /]$ 

I've reached out to the developer to see if i can get more help with it on Debian though.

Back on RHEL 8, i took the entirety of the Tomcat user's environment and converted it into exports like so:

Code: Select all

export LS_COLORS='rs=0:di=38;5;33:ln=38;5;51:mh=00:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=01;05;37;41:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;40:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.zst=38;5;9:*.tzst=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.wim=38;5;9:*.swm=38;5;9:*.dwm=38;5;9:*.esd=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.mjpg=38;5;13:*.mjpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.m4a=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.oga=38;5;45:*.opus=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:'
export LANG=en_US.UTF-8
export HISTCONTROL=ignoredups
export LESS=MRX
export HOSTNAME=lnx-b9ssb-devl.test.drake.edu
export NO_AT_BRIDGE=1
export S_COLORS=auto
export USER=tomcat8
export LC_COLLATE=C
export PWD=/usr/share/tomcat8
export HOME=/usr/share/tomcat8
export MAIL=/var/spool/mail/tomcat8
export SHELL=/bin/bash
export TERM=xterm-256color
export SHLVL=1
export LOGNAME=tomcat8
export PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
export PS1=\[\e[1m\][\u@\h \W]\$\[\e[0m\] 
export HISTSIZE=1000
export LESSOPEN=||/usr/bin/lesspipe.sh %s
export LC_TIME=en_XX.UTF-8
export _=/usr/bin/env

I pasted that into check_jvm and it still doesn't work from NRPE:

Code: Select all

# /usr/local/nagios/libexec/check_nrpe -H lnx-b9ssb-devl -u -t 30 -c tomcat_heap
UNKNOWN Can't connect to the JVM: 

[ssax]

What I was trying to have you do is to run the command when it works but prepend it with env -i so that it would not carry over the environmental variables (like exactly how it runs through NRPE). I thought it worked for tomcat8, that's why I was having you run it that way.

Unfortunately, we don't know what that jar file does so we are unable to figure out what it's looking for.

You can see if strace will give you any more info:

Code: Select all

sudo apt-get install strace

Then edit /usr/lib64/nagios/plugins/check_jvm and change this:

Code: Select all

JVMDATA=$(${TIMEOUT} java -jar "${JVMINSPECTOR}" "${PID}" 2>&1)

To this (adding a line above it):

Code: Select all

$(strace -f java -jar "${JVMINSPECTOR}" "${PID}" 2>&1 >> /tmp/NAGIOSTEST.txt)
JVMDATA=$(${TIMEOUT} java -jar "${JVMINSPECTOR}" "${PID}" 2>&1)

Then run the check, wait a minute or so, and then send the /tmp/NAGIOSTEST.txt file for review.

You can remove that line after it's done.