Inboundnrdp passive - Some services host seldom get updated

[flashmannag]

Hi

--- Our setup ---

We have several Nagios that actively (ActiveNagios from now on) monitor hosts and stuff. Those Nagios propagate all this data as an "Outbound transfer>NRDP" to a central Nagios (NagiosCentral).
The only purpouse of this Nagios is to give our IT Operations team a centralized view.

The NagiosCentral has checked "Unconfigured Objects>Auto Configure Settings>Enable Auto Import". "xwizard_passive_host" and "xwizard_passive_service" templates are applied.

The current HOST and SERVICE count is hosts=1126, services=24974

--- Versions ---
NagiosCentral:
- Nagios V5.6.13
- OS RH7
Other Active Nagios:
- Nagios V5.6.8
- OS Centos6 and Centos7

--- The issue ---
Some services seldom get updated, they can go for hours without updating in the NagiosCentral while they work just fine in the ActiveNagios. There are no performance or communication issues. I really don't now how to debug this.
I see no errors in the logs. None issue related ones at least.

--- Request of enhacement ---
Also, and I think of this as a request for enhacement more than an issue as is working as intended (I think). The Active Nagios are alive and changing so we have the "Deadpool Settings" for host and service deletion.
The thing is that our hosts and services are passive only in NagiosCentral and "deadpool" for some reason skips them. Plenty of "Passive only -> skipping host" and "Passive only -> skipping service" in deadpool.log.

As step 1 We are going to upgrade to latest version all the involved Nagios. But after that I really don't know what look for.

Thanks and best regards.

[benjaminsmith]

Hi,

Can you verify if the incoming check results are spooling up on the server. Check the following directory.

Code: Select all

/usr/local/nagios/var/spool

See: NRDP - Passive Check Results Not Being Submitted To Nagios

Let's also enable debugging on NRDP and watch the log for any errors. Edit the following file:

Code: Select all

/usr/local/nrdp/server/config.inc.ph

Then set $cfg["debug"] = true; in that file and then run these commands:

Code: Select all

touch /usr/local/nrdp/server/debug.log
chown apache.nagios /usr/local/nrdp/server/debug.log
chmod 664 /usr/local/nrdp/server/debug.log

Then watch the following file for errors:

Code: Select all

tail -Fn0 /usr/local/nrdp/server/debug.log

Also, please post the /usr/local/nagios/var/nagios.log so we can review any errors.

Thanks,
Benjamin

[flashmannag]

Hi Benjamin

Thanks for the fast response.

Can you verify if the incoming check results are spooling up on the server. Check the following directory.

Code: Select all

/usr/local/nagios/var/spool

The files in the spool directory are being processed fine.

See: NRDP - Passive Check Results Not Being Submitted To Nagios

Both nagios user and apache user are members of nagcmd group.

Code: Select all

[root@V99SCE-004649 checkresults]# lid -g nagcmd
 nagios(uid=55556)
 apache(uid=48)
 snmptt(uid=990)

Also I have activated the debug and let it run for a while. I attach both log files.

I have upgraded the nagios central and some of the active nagios to the latest version.

Thanks and best regards.

[ssax]

I think you're hitting this bug that was fixed in 2.0.2 of NRDP:

2.0.2 - 07/18/2019

Fixed issue with passive check timestamp not being applied properly -SW

Because the time says 0 on a lot of them.

Edit this file on NagiosCentral:

Code: Select all

/usr/local/nrdp/server/plugins/nagioscorepassivecheck/nagioscorepassivecheck.inc.php

Change line 216 from this:

Code: Select all

    $check_result_contents .= "start_time=" . time() . "\n";

To this:

Code: Select all

    $check_result_contents .= "start_time=" . time() . ".0\n";

See if that alleviates the issue.

[flashmannag]

Hi ssax

The 216 line already is this one

Code: Select all

 $check_result_contents .= "start_time=" . time() . ".0\n";

The first thing I did was upgrade the Nagios to version 5.7.4. Still the same issue, some services can go for hours without new data.

Thanks and best regards.

[ssax]

Please send the FULL output of these commands (as root):
- NOTE: You may need to adjust the -h 127.0.0.1, the -uroot, and -pnagiosxi in the first command if your DB is contained/stored on another server and/or you've changed the root mysql password

Code: Select all

mysql -h 127.0.0.1 -uroot -pnagiosxi -e 'SELECT NOW(); SELECT @@GLOBAL.time_zone, @@SESSION.time_zone;'
date
ls -l /etc/localtime
php -r 'echo date("D M j G:i:s T Y")."\n";'
grep "date.timezone =" /etc/php.ini

[flashmannag]

Hi

Code: Select all

+---------------------+
| NOW()               |
+---------------------+
| 2020-10-27 07:34:41 |
+---------------------+
+--------------------+---------------------+
| @@GLOBAL.time_zone | @@SESSION.time_zone |
+--------------------+---------------------+
| SYSTEM             | SYSTEM              |
+--------------------+---------------------+
Tue Oct 27 07:34:41 CET 2020
lrwxrwxrwx 1 root root 33 Jul 29  2019 /etc/localtime -> /usr/share/zoneinfo/Europe/Madrid
Tue Oct 27 7:34:41 CET 2020
date.timezone = Europe/Madrid

Thanks.

[BanditBBS]

Looks like there are two similar issues going on and being worked on...
https://support.nagios.com/forum/viewto ... 16&t=60426

[ssax]

@flashmannag, please PM me a copy of your profile, you can download it from Admin > System Profile by clicking the Download Profile button. Additionally, is your OS LDAP/AD integrated? (meaning when you SSH into the XI server you're using LDAP or AD credentials?)

@BanditBBS, I noticed that yesterday when working them, still not sure what is causing it yet. Please PM me a copy of your profile as well (or attach it to the ticket you opened). I'm not seeing the chgroup error in flashmannag's output so I'm not sure if that's related or not.

[flashmannag]

Hi ssax

PM with the profile sent.

Additionally, is your OS LDAP/AD integrated? (meaning when you SSH into the XI server you're using LDAP or AD credentials?)

I'm not totally sure. I use a local user that's for sure, but the VM was deployed through our cloud system and I know that other VMs are integrated with a central login system (AD). And I don't know if all of them use the same template. I'll ask our unix team.

Thanks and best regards.