NRPE: UNKNOWN Can't connect to the JVM

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
pmoradiya
Posts: 26
Joined: Fri Oct 06, 2017 9:48 am

NRPE: UNKNOWN Can't connect to the JVM

Post by pmoradiya »

Hi,

I have found seemingly inconsistent behavior of NRPE (v.3.2.1). I am running check_jvm plugin on the monitored hosts. All hosts have NRPE installed and configured exactly the same way. However, NRPE command on couple of hosts is not able to connect to JVM. Here is the info on config:

nrpe.cfg

Code: Select all

command[check_tomcat_threads]=/usr/bin/sudo -u tomcat8 /usr/local/nagios/libexec/check_jvm -n org.apache.catalina.startup.Bootstrap -p threads -w 195 -c 225
sudoers file

Code: Select all

nagios ALL=(ALL) NOPASSWD: /usr/local/nagios/libexec/
check_nrpe works as expected for other commands but not for check_jvm plugin

Code: Select all

root@server1:/usr/local/nagios/libexec# ./check_nrpe -H localhost -c check_load
OK - load average: 0.00, 0.00, 0.00|load1=0.000;0.150;0.300;0; load5=0.000;0.100;0.250;0; load15=0.000;0.050;0.200;0;
root@server1:/usr/local/nagios/libexec# ./check_nrpe -H localhost -c check_tomcat8
PROCS OK: 1 process with UID = 112 (tomcat8) | procs=1;;1:5;0;
root@server1:/usr/local/nagios/libexec# ./check_nrpe -H localhost -c check_tomcat_threads
UNKNOWN Can't connect to the JVM:
If the command in nrpe.cfg is run on the shell, it works fine both as root user and nagios user. Hence, no issue with check_jvm plugin too.

Code: Select all

root@server1:/usr/local/nagios/libexec# /usr/bin/sudo -u tomcat8 /usr/local/nagios/libexec/check_jvm -n org.apache.catalina.startup.Bootstrap -p threads -w 195 -c 225
OK 37 |threads=37;;;
I don't know what I am missing. I really appreciate your assistance with this.

Thanks.
pmoradiya
Posts: 26
Joined: Fri Oct 06, 2017 9:48 am

Re: NRPE: UNKNOWN Can't connect to the JVM

Post by pmoradiya »

Additional information from the debug log at /usr/local/nagios/var/nrpe.log
Showing WARNING as my_system() seteuid(0):Operation not permitted.

Code: Select all

[1510171606] Connection from 127.0.0.1 port 14494
[1510171606] is_an_allowed_host (AF_INET): is host >127.0.0.1< an allowed host >127.0.0.1<
[1510171606] is_an_allowed_host (AF_INET): host is in allowed host list!
[1510171606] Host address is in allowed_hosts
[1510171606] Host 127.0.0.1 is asking for command 'check_tomcat_threads' to be run...
[1510171606] Running command: /usr/bin/sudo -u tomcat8 /usr/local/nagios/libexec/check_jvm -n org.apache.catalina.startup.Bootstrap -p threads -w 195 -c 225
[1510171606] WARNING: my_system() seteuid(0): Operation not permitted
[1510171606] Command completed with return code 3 and output: UNKNOWN Can't connect to the JVM:
[1510171606] Return Code: 3, Output: UNKNOWN Can't connect to the JVM:
[1510171606] Connection from 127.0.0.1 closed.
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: NRPE: UNKNOWN Can't connect to the JVM

Post by mcapra »

Have you checked the permissions of JvmInspector.jar and validated that the tomcat8 user is able to execute it? That would be my first thought.
Former Nagios employee
https://www.mcapra.com/
pmoradiya
Posts: 26
Joined: Fri Oct 06, 2017 9:48 am

Re: NRPE: UNKNOWN Can't connect to the JVM

Post by pmoradiya »

mcapra wrote:Have you checked the permissions of JvmInspector.jar and validated that the tomcat8 user is able to execute it? That would be my first thought.
Yes. It has the same permission as on the other server where it is working.

Code: Select all

root@server2:/usr/local/bin# ls -l
-rw-r--r-- 1 root root 4714063 Nov  3 17:02 JvmInspector.jar

Code: Select all

root@server1:/usr/local/bin# ls -lart
-rw-r--r--  1 root root 4714063 Nov  9 15:32 JvmInspector.jar
npolovenko
Support Tech
Posts: 3457
Joined: Mon May 15, 2017 5:00 pm

Re: NRPE: UNKNOWN Can't connect to the JVM

Post by npolovenko »

Hello, @pmoradiya.
Please make the following change in your sudoers file:
Change:

Code: Select all

nagios ALL=(ALL) NOPASSWD: /usr/local/nagios/libexec/
To:

Code: Select all

nagios ALL=(ALL) NOPASSWD: /usr/local/nagios/libexec/check_jvm
Let us know if that fixes your issue.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
pmoradiya
Posts: 26
Joined: Fri Oct 06, 2017 9:48 am

Re: NRPE: UNKNOWN Can't connect to the JVM

Post by pmoradiya »

npolovenko wrote:Hello, @pmoradiya.
Please make the following change in your sudoers file:
Change:

Code: Select all

nagios ALL=(ALL) NOPASSWD: /usr/local/nagios/libexec/
To:

Code: Select all

nagios ALL=(ALL) NOPASSWD: /usr/local/nagios/libexec/check_jvm
Let us know if that fixes your issue.

I updated the sudoers file as suggested but the same result.

Code: Select all

nagios ALL=(ALL) NOPASSWD: /usr/local/nagios/libexec/check_jvm

root@server1:/usr/local/nagios/libexec# ./check_nrpe -H localhost -c check_tomcat_threads
UNKNOWN Can't connect to the JVM:
pmoradiya
Posts: 26
Joined: Fri Oct 06, 2017 9:48 am

Re: NRPE: UNKNOWN Can't connect to the JVM

Post by pmoradiya »

Appreciate if anyone can point to any small or big part of configuration I might have missed.

Thanks.
npolovenko
Support Tech
Posts: 3457
Joined: Mon May 15, 2017 5:00 pm

Re: NRPE: UNKNOWN Can't connect to the JVM

Post by npolovenko »

@pmoradiya, Can you upload your nrpe.cfg file? Also, since you're running the check from the tomcat8 user, I'd add another line in the sudoers file:

Code: Select all

tomcat8 ALL=(ALL) NOPASSWD: /usr/local/nagios/libexec/check_jvm
Keep us updated on this.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
pmoradiya
Posts: 26
Joined: Fri Oct 06, 2017 9:48 am

Re: NRPE: UNKNOWN Can't connect to the JVM

Post by pmoradiya »

npolovenko wrote:@pmoradiya, Can you upload your nrpe.cfg file? Also, since you're running the check from the tomcat8 user, I'd add another line in the sudoers file:

Code: Select all

tomcat8 ALL=(ALL) NOPASSWD: /usr/local/nagios/libexec/check_jvm
Keep us updated on this.
I have also added this line to sudoers but same result.

I have attached nrpe.cfg
Attachments
nrpe.cfg
(13.91 KiB) Downloaded 311 times
npolovenko
Support Tech
Posts: 3457
Joined: Mon May 15, 2017 5:00 pm

Re: NRPE: UNKNOWN Can't connect to the JVM

Post by npolovenko »

@pmoradiya, Can you manually switch to tomcat8 user and attempt to run the plugin?

Code: Select all

 su - tomcat8
And then:

Code: Select all

/usr/bin/sudo -u tomcat8 /usr/local/nagios/libexec/check_jvm -n org.apache.catalina.startup.Bootstrap -p threads -w 195 -c 225
Does that work?
I'd also modify permissions for JvmInspector.jar to allow for tomcat8 user.

Code: Select all

chown tomcat8 JvmInspector.jar
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Locked