Problem recieving passive checks

jmasquelet · Post by **jmasquelet** » Mon Feb 15, 2016 9:35 am

Problem receiving passive checks from NCPA.

Hi, i'm running nagios core 4 with Centos 7

A few days ago we were trying to activate passive checks. When finally decided to use NCPA (on windows machine) and NRDP on the server side.

The problem is that NCPA sends the check but Nagios is not receiving it.

Here are my config files and last logs entries.

ncpa.cfg

Code: Select all

[listener]
uid = nagios
certificate = adhoc
loglevel = info
ip = 0.0.0.0
gid = nagcmd
logfile = var/ncpa_listener.log
port = 5693
pidfile = var/ncpa_listener.pid
# Available versions: PROTOCOL SSLv2, SSLv3, TLSv1
ssl_version = TLSv1

[passive]
uid = nagios
handlers = nrds,nrdp
loglevel = info
gid = nagcmd
sleep = 300
logfile = var/ncpa_passive.log
pidfile = var/ncpa_passive.pid

[nrdp]
token = tokentest
hostname = 192.168.56.101
parent = http://192.168.56.101/nrdp/

[nrds]
URL =
CONFIG_VERSION =0
TOKEN =
CONFIG_NAME =
CONFIG_OS = None
PLUGIN_DIR=plugins/
UPDATE_CONFIG=1
UPDATE_PLUGINS=1

[api]
community_string =tokentest

[plugin directives]
plugin_path = plugins/
.sh = /bin/sh $plugin_name $plugin_args
.ps1 = powershell -ExecutionPolicy Bypass -File $plugin_name $plugin_args
.vbs = cscript $plugin_name $plugin_args //NoLogo

[passive checks]
%HOSTNAME%|memory usage = /memory/virtual/percent --warning 10 --critical 20

ncpa_passive.log

Code: Select all

2016-02-15 11:22:29,000:INFO:__init__:Establishing passive handler: Handler
2016-02-15 11:22:29,003:ERROR:__init__:Invalid URL u'': No schema supplied. Perhaps you meant http://?
Traceback (most recent call last):
  File "C:\ncpa\agent\ncpa_windows.py", line 192, in run_all_handlers
  File "C:\Program Files (x86)\Nagios\NCPA\passive\nrds.py", line 37, in run
    if self.config_update_is_required(nrds_url, nrds_token, nrds_config, nrds_config_version):
  File "C:\Program Files (x86)\Nagios\NCPA\passive\nrds.py", line 135, in config_update_is_required
    url_request = utils.send_request(nrds_url, **get_args)
  File "C:\Program Files (x86)\Nagios\NCPA\passive\utils.py", line 13, in send_request
    r = requests.post(url, data=kwargs, verify=False, allow_redirects=True)
  File "C:\Python27\lib\site-packages\requests\api.py", line 88, in post
  File "C:\Python27\lib\site-packages\requests\api.py", line 44, in request
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 349, in request
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 287, in prepare_request
  File "C:\Python27\lib\site-packages\requests\models.py", line 287, in prepare
  File "C:\Python27\lib\site-packages\requests\models.py", line 338, in prepare_url
MissingSchema: Invalid URL u'': No schema supplied. Perhaps you meant http://?
2016-02-15 11:22:29,005:INFO:__init__:Establishing passive handler: Handler
2016-02-15 11:22:29,005:INFO:__init__:Running check: /memory/virtual/percent --warning 10 --critical 20
2016-02-15 11:22:29,010:INFO:connectionpool:Starting new HTTP connection (1): 192.168.56.101
2016-02-15 11:22:29,019:INFO:__init__:Message from NRDP server: OK
2016-02-15 11:22:29,019:INFO:__init__:Meta output from NRDP server: 1 checks processed.

nagios.log

[root@localhost var]# tail nagios.log
[1455544409] SERVICE NOTIFICATION: nagiosadmin;192.168.1.6;CHECK_PASIVO;CRITICAL;notify-service-by-email;CRITICAL: CRITICAL: No se han recibido resultados en tiempo
[1455544409] wproc: NOTIFY job 38 from worker Core Worker 1385 is a non-check helper but exited with return code 127
[1455544409] wproc: host=192.168.1.6; service=CHECK_PASIVO; contact=nagiosadmin
[1455544409] wproc: early_timeout=0; exited_ok=1; wait_status=32512; error_code=0;
[1455544409] wproc: stderr line 01: /bin/sh: /bin/mail: No existe el fichero o el directorio
[1455544769] Warning: The results of service 'CHECK_PASIVO' on host '192.168.1.6' are stale by 0d 0h 1m 0s (threshold=0d 0h 5m 0s). I'm forcing an immediate check of the service.
[1455545129] Warning: The results of service 'CHECK_PASIVO' on host '192.168.1.6' are stale by 0d 0h 1m 0s (threshold=0d 0h 5m 0s). I'm forcing an immediate check of the service.
[1455545489] Warning: The results of service 'CHECK_PASIVO' on host '192.168.1.6' are stale by 0d 0h 1m 0s (threshold=0d 0h 5m 0s). I'm forcing an immediate check of the service.
[1455545849] Warning: The results of service 'CHECK_PASIVO' on host '192.168.1.6' are stale by 0d 0h 1m 0s (threshold=0d 0h 5m 0s). I'm forcing an immediate check of the service.
[1455546209] Warning: The results of service 'CHECK_PASIVO' on host '192.168.1.6' are stale by 0d 0h 1m 0s (threshold=0d 0h 5m 0s). I'm forcing an immediate check of the service.
[root@localhost var]#

ls -lrt /usr/local/nagios/var/spool/checkresults

-rw-r--r-- 1 apache nagcmd 0 feb 15 03:07 cKLxdGJ.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 09:57 cw3aase.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:02 cvlfPsV.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:07 cSNOJcC.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:12 cXfYRas.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:17 cOxwIEZ.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:22 cCzenoG.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:27 crEGG7m.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:32 cILat0X.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:37 c5OuLBf.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:42 cC3j2jr.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:47 cXjoy37.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:52 csTReTt.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:57 cGdztci.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:02 cknjUCa.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:07 ciZRfEA.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:12 cLPgqIz.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:17 csRW7qg.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:22 cVl0YCK.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:27 cmrgQLE.ok
[root@localhost checkresults]# pwd
/usr/local/nagios/var/spool/checkresults
[root@localhost checkresults]#

object file

[root@localhost objects]# cat mipc.cfg

define host{
use linux-server ; Name of host template to use
host_name 192.168.1.6
address 192.168.1.6
icon_image critico.png
}

define service{
use local-service ; Name of service template to use
host_name 192.168.1.6
service_description PING
check_command check_ping!100.0,20%!500.0,60%
}
define service{
use generic-service
host_name 192.168.1.6
service_description CHECK_PASIVO
check_command sin_resultados
passive_checks_enabled 1
active_checks_enabled 0
check_freshness 1
freshness_threshold 300 ; Umbral de 10 minutos, si no recibe información ejecutara el check_command
}

As you can see ncpa logs shows that they deliver the check and nrdp responses OK but no files are loaded on checkresults.

I think the problem might be in one of this files.

in /usr/local/nagios/var/spool/checkresults i can see that last night i've received some results:

[root@localhost checkresults]# cat cVZUhGI
### NRDP Check ###
start_time=1455496296.0
# Time: Mon, 15 Feb 2016 00:31:36 +0000
host_name=192.168.56.101
service_description=memory usage
check_type=1
early_timeout=1
exited_ok=1
return_code=1
output=WARNING: percent was 75% | 'percent_0'=75%;60;80;\n
[root@localhost checkresults]#

but for some reason that's not happening now and i don't know why.

rkennedy · Post by **rkennedy** » Mon Feb 15, 2016 3:16 pm

This first part is a side note -

Code: Select all

[1455544409] wproc: stderr line 01: /bin/sh: /bin/mail: No existe el fichero o el directorio

Your notifications won't fire because your command definitions are not set properly. You'll need to update the commands.cfg file, and change the path in the command_line for the command_name 'notify-host-by-email', and 'notify-service-by-email'.

This also leads me to believe that others components didn't install properly. I believe the permissions on that checkresults folder may not be correct.

What is the output of getenforce, and ls -l /usr/local/nagios/var/spool/?

jmasquelet · Post by **jmasquelet** » Mon Feb 15, 2016 4:06 pm

Hi! thanks for helping me

[root@localhost spool]# getenforce
Disabled
[root@localhost spool]# ls -l
total 16
drwxrwsr-x. 2 nagios nagcmd 12288 feb 15 17:47 checkresults

Post by **Box293** » Mon Feb 15, 2016 6:19 pm

Did you fix the commands as per what @rkennedy suggested?

Did it fix your problem?

jmasquelet · Post by **jmasquelet** » Mon Feb 15, 2016 8:28 pm

Well not really. He Was talking about email notification. I still don't configure that (its a new instalation )because my Server wont send mails.

rkennedy · Post by **rkennedy** » Tue Feb 16, 2016 6:00 pm

Can you provide a link to the instructions you used to install Core?

Can you verify a few permissions for me with these commands?

Code: Select all

ls -l /usr/local/nagios
ls -l /usr/local/nagios/var
ls -l /usr/local/nagios/var/spool

Post by **Box293** » Tue Feb 16, 2016 6:03 pm

jmasquelet wrote:ls -lrt /usr/local/nagios/var/spool/checkresults
-rw-r--r-- 1 apache nagcmd 0 feb 15 03:07 cKLxdGJ.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 09:57 cw3aase.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:02 cvlfPsV.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:07 cSNOJcC.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:12 cXfYRas.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:17 cOxwIEZ.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:22 cCzenoG.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:27 crEGG7m.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:32 cILat0X.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:37 c5OuLBf.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:42 cC3j2jr.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:47 cXjoy37.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:52 csTReTt.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:57 cGdztci.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:02 cknjUCa.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:07 ciZRfEA.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:12 cLPgqIz.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:17 csRW7qg.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:22 cVl0YCK.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:27 cmrgQLE.ok
[root@localhost checkresults]# pwd
/usr/local/nagios/var/spool/checkresults
[root@localhost checkresults]#

jmasquelet wrote:[root@localhost spool]# ls -l
total 16
drwxrwsr-x. 2 nagios nagcmd 12288 feb 15 17:47 checkresults

Can you try:

Code: Select all

chmod -R g+w /usr/local/nagios/var/spool/checkresults

Also,

jmasquelet wrote:[root@localhost checkresults]# cat cVZUhGI
### NRDP Check ###
start_time=1455496296.0
# Time: Mon, 15 Feb 2016 00:31:36 +0000
host_name=192.168.56.101
service_description=memory usage
check_type=1
early_timeout=1
exited_ok=1
return_code=1
output=WARNING: percent was 75% | 'percent_0'=75%;60;80;\n
[root@localhost checkresults]#

jmasquelet wrote:define service{
use generic-service
host_name 192.168.1.6
service_description CHECK_PASIVO
check_command sin_resultados
passive_checks_enabled 1
active_checks_enabled 0
check_freshness 1
freshness_threshold 300 ; Umbral de 10 minutos, si no recibe información ejecutara el check_command
}

Do you have a passive service for "memory usage"

jmasquelet · Post by **jmasquelet** » Wed Feb 17, 2016 10:26 am

rkennedy wrote:Can you provide a link to the instructions you used to install Core?

Can you verify a few permissions for me with these commands?
Code: Select all
ls -l /usr/local/nagios
ls -l /usr/local/nagios/var
ls -l /usr/local/nagios/var/spool

This is the link:
http://www.aboutmonitoring.com/installi ... -redhat-7/

ls -l /usr/local/nagios
total 20
drwxrwxr-x. 2 nagios nagios 36 ene 29 00:07 bin
drwxrwxr-x. 5 nagios nagios 4096 feb 15 17:46 etc
drwxr-xr-x. 2 root root 6 ene 29 00:14 include
drwxrwxr-x. 3 nagios nagios 4096 feb 16 01:58 libexec
drwxrwxr-x. 2 nagios nagios 4096 ene 29 00:07 sbin
drwxrwxr-x. 15 nagios nagios 4096 ene 29 00:14 share
drwxrwxr-x. 5 nagios nagios 4096 feb 17 11:52 var

ls -l /usr/local/nagios/var
total 304
drwxrwxr-x. 2 nagios nagios 6 ene 29 00:07 archives
-rw-r--r-- 1 nagios nagios 7174 feb 17 11:42 livestatus.log
-rw-r--r--. 1 nagios nagios 34 feb 17 11:48 nagios.configtest
-rw-r--r-- 1 nagios nagios 5 feb 17 11:48 nagios.lock
-rw-r--r--. 1 nagios nagios 178474 feb 17 11:50 nagios.log
-rw-rw-r-- 1 nagios nagios 19328 feb 17 11:44 nagios.tmpFADhN0
-rw-r--r--. 1 nagios nagios 22045 feb 17 11:48 objects.cache
-rw-r--r--. 1 nagios nagios 22045 feb 17 11:48 objects.precache
-rw------- 1 nagios nagios 18889 feb 17 11:48 retention.dat
drwxrwsr-x. 3 nagios nagcmd 65 feb 17 11:48 rw
drwxr-xr-x. 3 root root 25 ene 29 00:07 spool
-rw-rw-r-- 1 nagios nagios 19450 feb 17 11:52 status.dat

ls -l /usr/local/nagios/var/spool
total 36
drwxrwsr-x. 2 nagios nagcmd 16384 feb 17 11:48 checkresults

Box293 wrote:
jmasquelet wrote:ls -lrt /usr/local/nagios/var/spool/checkresults
-rw-r--r-- 1 apache nagcmd 0 feb 15 03:07 cKLxdGJ.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 09:57 cw3aase.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:02 cvlfPsV.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:07 cSNOJcC.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:12 cXfYRas.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:17 cOxwIEZ.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:22 cCzenoG.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:27 crEGG7m.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:32 cILat0X.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:37 c5OuLBf.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:42 cC3j2jr.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:47 cXjoy37.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:52 csTReTt.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 10:57 cGdztci.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:02 cknjUCa.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:07 ciZRfEA.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:12 cLPgqIz.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:17 csRW7qg.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:22 cVl0YCK.ok
-rw-r--r-- 1 apache nagcmd 0 feb 15 11:27 cmrgQLE.ok
[root@localhost checkresults]# pwd
/usr/local/nagios/var/spool/checkresults
[root@localhost checkresults]#

jmasquelet wrote:[root@localhost spool]# ls -l
total 16
drwxrwsr-x. 2 nagios nagcmd 12288 feb 15 17:47 checkresults
Can you try:
Code: Select all
chmod -R g+w /usr/local/nagios/var/spool/checkresults 

i'm soure there is no problem with permissions in /usr/local/nagios/var/spool/checkresults because of this. The directory recieve a file every 5 minutes and if i execute ls -lrt on that directory exactly in the minute 5 i can see this:

ls -lrt
..
-rw-r--r-- 1 apache nagcmd 0 feb 17 09:25 csUIegz.ok
-rw-r--r-- 1 apache nagcmd 0 feb 17 09:30 cGUfCZf.ok
-rw-r--r-- 1 apache nagcmd 256 feb 17 11:54 cZYQNPv
[root@localhost checkresults]#

right after try to read it but it's gone and the next ls-lrt shows the following output

-rw-r--r-- 1 apache nagcmd 0 feb 16 09:07 cyYiaeg.ok
-rw-r--r-- 1 apache nagcmd 0 feb 17 09:25 csUIegz.ok
-rw-r--r-- 1 apache nagcmd 0 feb 17 09:30 cGUfCZf.ok
-rw-r--r-- 1 apache nagcmd 0 feb 17 11:54 cZYQNPv.ok
[root@localhost checkresults]#

the name of the file change with an .ok at the end and it's empty

Box293 wrote:
Also,

jmasquelet wrote:[root@localhost checkresults]# cat cVZUhGI
### NRDP Check ###
start_time=1455496296.0
# Time: Mon, 15 Feb 2016 00:31:36 +0000
host_name=192.168.56.101
service_description=memory usage
check_type=1
early_timeout=1
exited_ok=1
return_code=1
output=WARNING: percent was 75% | 'percent_0'=75%;60;80;\n
[root@localhost checkresults]#

jmasquelet wrote:define service{
use generic-service
host_name 192.168.1.6
service_description CHECK_PASIVO
check_command sin_resultados
passive_checks_enabled 1
active_checks_enabled 0
check_freshness 1
freshness_threshold 300 ; Umbral de 10 minutos, si no recibe información ejecutara el check_command
}

Do you have a passive service for "memory usage"

yes, this is the service

define service{
use generic-service
host_name 192.168.1.6
service_description memory usage
check_command sin_resultados
passive_checks_enabled 1
active_checks_enabled 0
check_freshness 1
freshness_threshold 300 ; Umbral de 10 minutos, si no recibe información ejecutara el check_command
}

rkennedy · Post by **rkennedy** » Wed Feb 17, 2016 2:26 pm

I believe the permissions need to be changed for this folder -

drwxr-xr-x. 3 root root 25 ene 29 00:07 spool

Can you try running the following?

Code: Select all

chown nagios:nagios /usr/local/nagios/var/spool

jmasquelet · Post by **jmasquelet** » Wed Feb 17, 2016 7:47 pm

rkennedy wrote:I believe the permissions need to be changed for this folder -
drwxr-xr-x. 3 root root 25 ene 29 00:07 spool
Can you try running the following?
Code: Select all
chown nagios:nagios /usr/local/nagios/var/spool

Done, but the problem persist.

Finally able to read that file that was automatically deleted:

-rw-r--r-- 1 apache nagcmd 0 feb 17 18:54 cqTJxfO.ok
-rw-r--r-- 1 apache nagcmd 0 feb 17 18:54 c6aiUZj.ok
-rwxrwx--- 1 apache nagcmd 244 feb 17 18:54 c6aiUZj
[root@localhost checkresults]# cat c6aiUZj
### NRDP Check ###
start_time=1455746046.0
# Time: Wed, 17 Feb 2016 21:54:06 +0000
host_name=192.168.56.101
service_description=memory usage
check_type=1
early_timeout=1
exited_ok=1
return_code=0
output=OK: percent was 59% | 'percent_0'=59%;60;80;\n

which leads me to believe that there may be some directive in a configuration file that is causing these files are deleted??

Nagios Support Forum

Problem recieving passive checks

Problem recieving passive checks

Re: Problem recieving passive checks

Re: Problem recieving passive checks

Re: Problem recieving passive checks

Re: Problem recieving passive checks

Re: Problem recieving passive checks

Re: Problem recieving passive checks

Re: Problem recieving passive checks

Re: Problem recieving passive checks

Re: Problem recieving passive checks