4 nagios process at the same time

[zahik]

hi
we are having this issue a lot that after apply we see 4 nagios process not 2
usually after apply config( but not always most of the time it OK)

i see this alert every week or so..


i have to kill them manually and restart nagios

we are using nagios 5.4.2 but it append in older version also

any advice?


[root@nagios_xi_prod ~]# ps -ef | grep nagios.cfg | grep -v grep
nagios 1269 1 4 Feb24 ? 01:04:20 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 1313 1269 0 Feb24 ? 00:00:03 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 12084 1 4 Feb24 ? 00:32:58 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 12135 12084 0 Feb24 ? 00:00:02 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
[root@nagios_xi_prod ~]# service nagios stop
Stopping nagios:. done.
[root@nagios_xi_prod ~]# ps -ef | grep nagios.cfg | grep -v grep
nagios 1269 1 4 Feb24 ? 01:04:20 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 1313 1269 0 Feb24 ? 00:00:03 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
[root@nagios_xi_prod ~]# kill -9 1269
[root@nagios_xi_prod ~]# ps -ef | grep nagios.cfg | grep -v grep
[root@nagios_xi_prod ~]# service nagios start
Starting nagios: done.
[root@nagios_xi_prod ~]#

[Fred Kroeger]

Yes - I've noticed this happening now too on the odd occasion - maybe monthly. It appears to happen more freqently on busy Nagios servers (weekly) .
It's almost like the old Nagios process doesn't stop (or wait long enough) when a new config gets applied.
I've had to create a new monitor to alert me when we have more than 2 Nagios processes running (grepping for nagios.cfg).
On instances where I'm running Mod-Gearman this is an absolute disaster - because there are now 2 Results queues.

regards.... Fred

[zahik]

hi fred ,
yes u can say in the newer versions this append more (alomost weekly for us system run about 15000 checks at 15 min)
i did add a monitor for this

/usr/local/nagios/libexec/check_procs -a '/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg' -w :2 -c :2
PROCS OK: 2 processes with args '/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg' | procs=2;:2;:2;0;

[tgriep]

On busier systems, it could be that the system didn't wait long enough to restart the nagios process when an Apply Config was ran and that could cause the multiple nagios processes.
To give the system more time edit the following file
/etc/init.d/nagios
Change this line from (around line 212)

Code: Select all

for i in 1 2 3 4 5 6 7 8 9 10 ; do

to

Code: Select all

for i in [1..30] ; do

Save the file and now the wait time will be increased to 30 seconds hopefully resolving the duplicate processes.

[Fred Kroeger]

Thanks for the tip, in fact I remember doing that some time ago. Looks like the init.d nagios file got updated during an upgrade and those mods are no longer there.
BTW - I think those should be curly braces -

Code: Select all

for i in {1..30}

Fred

[tgriep]

Yes, you are correct curly braces are correct.

[Fred Kroeger]

Is it possible to get this longer delay incorporated pemanently into the init.d nagios file ?

[zahik]

thanks for the reply did the change in the file
now will see ,hope this fix the issue
also should i expect this to change in the next update?i mean the file reset to it original config after upgrade to newer version?

#for i in 1 2 3 4 5 6 7 8 9 10 ; do
for i in {1..30} ; do

again thanks for the info

[tgriep]

Your welcome, I put in a feature request to get this added. The TASK ID is 11196 for your reference.
Let us know if the change works for you.