yum updates unknown - CHECK_NRPE: Receive header underflow -

[bruce_clegg]

Brand New User here.

I am setting up nagios for the first time here - I'm still very much in the testing phase.

I have Nagios running and have set up just a few checks so far. I'm monitoring a web page and a CentOS 7 test server running elk. I went through the configuration wizard and during setup I asked it to monitor mysql and httpd. both of these services are up on the server, but Nagios is reporting them down. Other services such as Disk Usage, Swap Usage, Ping, etc. are working fine.

what do I need to do to get nagios to recognize that mysql and httpd are up? Or are 'Apache Web Server' and 'MySQL Server' something other than what I can see by doing a ps -ef | grep httpd or ps -ef | grep mysql ?

Also, I'm getting a warning on Yum Updates - not sure why. Can that be explained too?

[lmiltchev]

I went through the configuration wizard and during setup I asked it to monitor mysql and httpd. both of these services are up on the server, but Nagios is reporting them down. Other services such as Disk Usage, Swap Usage, Ping, etc. are working fine.
what do I need to do to get nagios to recognize that mysql and httpd are up? Or are 'Apache Web Server' and 'MySQL Server' something other than what I can see by doing a ps -ef | grep httpd or ps -ef | grep mysql ?

Can you elaborate on that? What configuration wizard did you run? Can you show us the actual check (that is failing) run from the command line along with the output of it? What is the OS/distro of the client (remote machine)?

Also, I'm getting a warning on Yum Updates - not sure why. Can that be explained too?

Most likely, you have packages that could be upgraded. What happens if you run the following command on the client (remote machine):

Code: Select all

yum update -y

then schedule an immediate "yum" check in Nagios XI? Did the "warning" status changed to "OK"?

Note: The above command is to be used on CentOS/RHEL machines. If your remote machine's OS is Ubuntu or Debian, you could run the following command to update it:

Code: Select all

apt-get update -y && apt-get upgrade -y && apt-get dist-upgrade -y

[bruce_clegg]

Thank you for your quick response.

Can you elaborate on that? What configuration wizard did you run? Can you show us the actual check (that is failing) run from the command line along with the output of it? What is the OS/distro of the client (remote machine)?

The client is also a CentOS 7 server. I added the machine using the 'Linux Server' configuration wizard. I went with the defaults, but checked and saw that httpd and mysql were running on the server, so I added those checks.

Running as root for all of these.

For httpd:

# /usr/local/nagios/libexec/check_nrpe -H 10.X.X.X -t 30 -c check_init_service -a 'httpd' -n
unknown

For mysql:

# /usr/local/nagios/libexec/check_nrpe -H 10.X.X.X -t 30 -c check_init_service -a 'mysqld' -n
unknown

ps -ef shows me that both httpd and mysqld are running on the client machine.

I just ran yum update -y and it completed successfully.

# /usr/local/nagios/libexec/check_nrpe -H 10.X.X.X -t 30 -c 'check_yum' -n
CHECK_NRPE: Receive header underflow - only -1 bytes received (4 expected).

[lmiltchev]

What is the NRPE version that you have on both machines - the server and the client?

Can you show us how the "check_init_service" command is configured on the remote machine (client)? It should be defined either in /usr/local/nagios/etc/nrpe/common.cfg or /usr/local/nagios/etc/nrpe.cfg (depending on how you installed NRPE).

Run the following commands on the remote machine and show the output:

This is run as "root" user:

Code: Select all

/usr/local/nagios/libexec/check_init_service httpd
/usr/local/nagios/libexec/check_init_service 'mysqld'

This is run as "nagios" user:
su - nagios

Code: Select all

/usr/local/nagios/libexec/check_init_service httpd
/usr/local/nagios/libexec/check_init_service 'mysqld'

Also show the output of the following commands run on the client:

Code: Select all

ps -ef | grep 'httpd' | grep -v grep
ps -ef | grep 'mysqld' | grep -v grep

[bruce_clegg]

Again - thank you for the quick response

I have config files at both /usr/local/nagios/etc/nrpe/common.cfg and /usr/local/nagios/etc/nrpe.cfg. How can I tell which one is being used? Normally I'd run a ps -ef and see information about the process, but I can't see a nrpe service in ps


What is the NRPE version that you have on both machines - the server and the client?


Master
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1
connect to address 127.0.0.1 port 5666: Connection refused
connect to host 127.0.0.1 port 5666: Connection refused

Is this my problem!?

client:
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1
NRPE v3.2.1

Here's where I go all red in the face - httpd and mysql are not running on the client. Somehow I fat-fingered the IP multiple times. I was mistaken. So that part of my question is resolved.

Still curious about yum

[lmiltchev]

I have config files at both /usr/local/nagios/etc/nrpe/common.cfg and /usr/local/nagios/etc/nrpe.cfg. How can I tell which one is being used?

You shouldn't have the same commands defined twice (in two different files). If you do, one of them should be commented out. NRPE would use both files, and most likely would use the "last" command as each line is read in order (top - down).

Normally I'd run a ps -ef and see information about the process, but I can't see a nrpe service in ps

Depending on how you installed NRPE, it could run under xinetd or as a "standalone daemon". Most probably, NRPE is running under xinetd on your system. That's why you don't see it when you use the "ps" command.

Example of NRPE running as a "standalone" daemon

Code: Select all

netstat -anp | grep 5666
tcp        0      0 0.0.0.0:5666                0.0.0.0:*                   LISTEN      1139/nrpe
tcp        0      0 :::5666                     :::*                        LISTEN      1139/nrpe

Example of NRPE running under xinetd

Code: Select all

netstat -anp | grep 5666
tcp6       0      0 :::5666                 :::*                    LISTEN      24236/xinetd

What is the NRPE version that you have on both machines - the server and the client?


Master
# /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1
connect to address 127.0.0.1 port 5666: Connection refused
connect to host 127.0.0.1 port 5666: Connection refused

Is this my problem!?

No, not really. It only shows that 127.0.0.1 is not allowed IP address in the configs. You can check the NRPE version on the master by running:

Code: Select all

/usr/local/nagios/bin/nrpe -V

or/and

Code: Select all

/usr/local/nagios/libexec/check_nrpe -V

Here's where I go all red in the face - httpd and mysql are not running on the client. Somehow I fat-fingered the IP multiple times. I was mistaken. So that part of my question is resolved.

No worries - this happens more often than you think. I am glad this issue is resolved!

Still curious about yum

Can you run the following commands on the client, and show the output?

Code: Select all

yum clean all
yum check-updates
/usr/local/nagios/libexec/check_yum

Show us the "check_yum" definition on the client machine (from common.cfg and/or nrpe.cfg file).

[bruce_clegg]

here are the results you asked for:

[root@ip-10-X-X-X ~]# yum clean all
Loaded plugins: fastestmirror
Cleaning repos: base epel extras kibana-7.x logstash-2.2 nagios-base nagiosxi-deps updates
Other repos take up 2.6 M of disk space (use --verbose for details)
[root@ip-10-X-X-X ~]# yum check-updates
Loaded plugins: fastestmirror
Determining fastest mirrors
epel/x86_64/metalink | 18 kB 00:00:00
* base: d36uatko69830t.cloudfront.net
* epel: mirrors.syringanetworks.net
* extras: d36uatko69830t.cloudfront.net
* updates: d36uatko69830t.cloudfront.net
base | 3.6 kB 00:00:00
epel | 5.3 kB 00:00:00
extras | 2.9 kB 00:00:00
kibana-7.x | 1.3 kB 00:00:00
logstash-2.2 | 951 B 00:00:00
nagios-base | 1.5 kB 00:00:00
nagiosxi-deps | 1.5 kB 00:00:00
updates | 2.9 kB 00:00:00
(1/10): base/7/x86_64/group_gz | 165 kB 00:00:00
(2/10): base/7/x86_64/primary_db | 6.0 MB 00:00:00
(3/10): epel/x86_64/group_gz | 90 kB 00:00:00
(4/10): epel/x86_64/updateinfo | 1.0 MB 00:00:01
(5/10): extras/7/x86_64/primary_db | 159 kB 00:00:00
(6/10): updates/7/x86_64/primary_db | 6.7 MB 00:00:00
(7/10): kibana-7.x/primary | 101 kB 00:00:00
(8/10): nagios-base/primary | 1.3 kB 00:00:00
(9/10): nagiosxi-deps/primary | 4.3 kB 00:00:00
(10/10): epel/x86_64/primary_db | 6.9 MB 00:00:01
logstash-2.2/primary | 2.6 kB 00:00:00
kibana-7.x 279/279
logstash-2.2 5/5
nagios-base 3/3
nagiosxi-deps 9/9
[root@ip-10-X-X-X ~]# /usr/local/nagios/libexec/check_yum
YUM OK: O/S is up to date.

[lmiltchev]

Show us the "check_yum" definition on the client machine (from common.cfg and/or nrpe.cfg file).

Did you forget to show us the "check_yum" command?

[root@ip-10-X-X-X ~]# /usr/local/nagios/libexec/check_yum
YUM OK: O/S is up to date.

So, the "check_yum" plugin is showing that the OS is up to date on the client. You should get the same output when testing the check from the command line on the Nagios XI server. Try:

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H 10.X.X.X -t 30 -c 'check_yum'

or

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H 10.X.X.X -t 30 -c 'check_yum' -n

FYI, you showed us commands with "-n" in previous posts, but I am not sure if you are using SSL or not.

[bruce_clegg]

Here is the latest. Again, I very much appreciate your help.

[root@ip-10-X-X-X ~]# /usr/local/nagios/libexec/check_yum
YUM OK: O/S is up to date.
[root@ip-10-X-X-X ~]# /usr/local/nagios/libexec/check_nrpe -H 10.X.X.X -t 30 -c 'check_yum'
CHECK_NRPE: Error - Could not connect to 10.X.X.X: Connection reset by peer
[root@ip-10-X-X-X ~]# /usr/local/nagios/libexec/check_nrpe -H 10.X.X.X -t 30 -c 'check_yum' -n
CHECK_NRPE: Receive header underflow - only -1 bytes received (4 expected).

cat /usr/local/nagios/etc/nrpe/common.cfg

### GENERIC SERVICES ###
command[check_init_service]=sudo /usr/local/nagios/libexec/check_init_service $ARG1$
command[check_services]=/usr/local/nagios/libexec/check_services -p $ARG1$

### MISC SYSTEM METRICS ###
#command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_users]=/usr/local/nagios/libexec/check_users $ARG1$
command[check_load]=/usr/local/nagios/libexec/check_load $ARG1$
command[check_swap]=/usr/local/nagios/libexec/check_swap $ARG1$
command[check_cpu_stats]=/usr/local/nagios/libexec/check_cpu_stats.sh $ARG1$
command[check_mem]=/usr/local/nagios/libexec/custom_check_mem -n $ARG1$

### SYSTEM UPDATES ###
command[check_yum]=/usr/local/nagios/libexec/check_yum
command[check_apt]=/usr/local/nagios/libexec/check_apt

### DISK ###
command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$
command[check_ide_smart]=/usr/local/nagios/libexec/check_ide_smart $ARG1$

### PROCESSES ###
command[check_all_procs]=/usr/local/nagios/libexec/custom_check_procs
command[check_procs]=/usr/local/nagios/libexec/check_procs $ARG1$

### OPEN FILES ###
command[check_open_files]=/usr/local/nagios/libexec/check_open_files.pl $ARG1$

### NETWORK CONNECTIONS ###
command[check_netstat]=/usr/local/nagios/libexec/check_netstat.pl -p $ARG1$ $ARG2$

I did not see anything referencing check_yum in /usr/local/nagios/etc/nrpe.cfg. Please let me know if you'd like me to include a copy here.


Here is the check yum file:

# cat /usr/local/nagios/libexec/check_yum
#!/usr/bin/perl
# Predefined NAGIOS exit codes.
%exit_codes = ('UNKNOWN' ,-1,
'OK' , 0,
'WARNING' , 1,
'CRITICAL', 2,);



sub usage()
{
print "\ncheck_yum (Nagios Plugin v1.0)";
print "\n Check that an O/S is up to date.";
print "\nUsage:";
print "\n check_yum\n\n";
exit $exit_codes{'OK'};
}



if ($#ARGV ge 0)
{ &usage; }

$yum = system("yum check-update >/dev/null");
#chomp($yum = `yum check-update | grep -e i386 -e noarch -e x86_64 | grep -i -v excluding | wc -l`);

if ($yum != 0)
{
print "YUM WARNING: O/S requires an update.\n";
exit $exit_codes{'WARNING'};
}
else
{
print "YUM OK: O/S is up to date.\n";
exit $exit_codes{'OK'};
}

[lmiltchev]

Please review the KB article below in order to troubleshoot this "Connection reset by peer" error:

https://support.nagios.com/kb/article/c ... r-616.html

It is possible that you haven't added your Nagios XI server's IP address to the "only_from" line in the "/etc/xinetd.d/nrpe" file, and restarted xinetd. Run the following command on the client and show the output:

Code: Select all

cat /etc/xinetd.d/nrpe

You said that you are using NRPE v3.2.1 on the client but what is the version of NRPE on the Nagios XI server that you are using? Can you run the following command from the command line on the XI box, and show the output?

Code: Select all

/usr/local/nagios/libexec/check_nrpe -V
nmap <client IP> -p 5666