Nagios Support Forum

Hi there,

I am facing one trouble with my nagios.
It is not detecting changes done in configurations.
I tried below methods.
1. recreation of objects.cache and objects.precache file using option -p and -u
2. deleted retention.dat and status.dat
Is there anything I am missing?
Thanks in Advance.

In Addition - I am facing weird behavior of nagios, sometimes it displays the changes and sometimes it does not.
At one point I thought its working fine, and again on next browser refresh I saw that changes are not visible.

Let's get some basic information:

• What Nagios Core version are you using?
• What OS and version is it installed on?
• Did you compile from source or use something like yum or apt-get to install?
• After you are changing the configs, are you making sure to restart the nagios process?

Hi,
Thanks for your reply.

I upgraded the nagios 4.0.8 to 4.1.1 on centos version 6.x
Yes I did compile the nagios from source.
and yes i am restarting the nagios process after every change.
please let me know if u need any more details.

Can you post the full output of ps -aef? It sounds like you may have multiple processes running.

Hi,
Here is the output of ps -aef | grep nagios
NOTE : I deleted plugin processes which were exposing confidential data.

0 S <user> 5195 1 1 80 0 - 5201 ep_pol 06:41 ? 00:04:03 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
1 S <user> 5200 5195 0 80 0 - 3439 poll_s 06:41 ? 00:00:03 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
0 R <user> 8389 1 0 80 0 - 5200 - Jul27 ? 00:08:15 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
0 Z <user> 8391 8389 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
0 Z <user> 8392 8389 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
0 Z <user> 8393 8389 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
1 S <user> 8394 8389 0 80 0 - 3438 poll_s Jul27 ? 00:00:07 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
0 S <user> 8470 1 0 80 0 - 5232 ep_pol Jul27 ? 00:08:16 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
0 Z <user> 8471 8470 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
0 Z <user> 8472 8470 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
0 Z <user> 8473 8470 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
0 Z <user> 8474 8470 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
1 S <user> 8475 8470 0 80 0 - 3438 poll_s Jul27 ? 00:00:08 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
0 R <user> 8727 8729 0 80 0 - 2504 - 11:53 ? 00:00:28 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 8728 8729 0 80 0 - 2504 ep_pol 11:53 ? 00:00:29 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
1 S <user> 8729 1 0 80 0 - 6260 ep_pol 10:24 ? 00:01:06 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
0 R <user> 8731 8729 0 80 0 - 2504 - 11:53 ? 00:00:29 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 8732 8729 0 80 0 - 2504 ep_pol 11:53 ? 00:00:29 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
1 S <user> 8802 8729 0 80 0 - 6112 poll_s 10:24 ? 00:00:01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
0 S <user> 9109 1 0 80 0 - 5199 ep_pol Jul27 ? 00:08:05 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
0 Z <user> 9110 9109 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
0 Z <user> 9111 9109 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
0 Z <user> 9112 9109 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
0 Z <user> 9113 9109 0 80 0 - 0 exit Jul27 ? 00:00:00 [nagios] <defunct>
1 S <user> 9114 9109 0 80 0 - 3438 poll_s Jul27 ? 00:00:09 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
0 R <user> 9460 1 1 80 0 - 5181 - 12:58 ? 00:00:11 ../bin/nagios -u ../etc/nagios.cfg
0 S <user> 9503 9460 0 80 0 - 2504 ep_pol 12:58 ? 00:00:04 /usr/local/nagios-4.1.1/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 9504 9460 0 80 0 - 2504 ep_pol 12:58 ? 00:00:04 /usr/local/nagios-4.1.1/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 9505 9460 0 80 0 - 2504 ep_pol 12:58 ? 00:00:03 /usr/local/nagios-4.1.1/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 9506 9460 0 80 0 - 2504 ep_pol 12:58 ? 00:00:04 /usr/local/nagios-4.1.1/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
1 S <user> 9778 9460 0 80 0 - 5023 poll_s 12:58 ? 00:00:00 ../bin/nagios -u ../etc/nagios.cfg
1 R <user> 11549 1 0 80 0 - 5186 ? 13:12 ? 00:00:02 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
0 R <user> 11564 11549 0 80 0 - 2504 - 13:12 ? 00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 R <user> 11565 11549 0 80 0 - 2504 - 13:12 ? 00:00:01 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 11566 11549 0 80 0 - 2504 ep_pol 13:12 ? 00:00:01 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 R <user> 11567 11549 0 80 0 - 2504 - 13:12 ? 00:00:00 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 11767 6967 0 80 0 - 25828 pipe_w 13:16 pts/15 00:00:00 grep nagios
1 S <user> 12441 11549 0 80 0 - 5057 poll_s 13:12 ? 00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
0 S <user> 12782 8389 0 80 0 - 2504 ep_pol Jul27 ? 00:07:55 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 12783 8389 0 80 0 - 2504 ep_pol Jul27 ? 00:08:02 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 12784 8389 0 80 0 - 2504 ep_pol Jul27 ? 00:07:54 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 12785 8389 0 80 0 - 2504 ep_pol Jul27 ? 00:07:52 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 R <user> 14695 9109 0 80 0 - 2504 - Jul27 ? 00:07:42 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 14696 9109 0 80 0 - 2504 ep_pol Jul27 ? 00:07:35 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 R <user> 14697 9109 0 80 0 - 2504 - Jul27 ? 00:07:44 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 14698 9109 0 80 0 - 2504 ep_pol Jul27 ? 00:07:46 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 15940 8470 0 80 0 - 2504 ep_pol Jul27 ? 00:07:54 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 R <user> 15941 8470 0 80 0 - 2504 - Jul27 ? 00:08:01 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 15942 8470 0 80 0 - 2504 ep_pol Jul27 ? 00:07:58 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 15943 8470 0 80 0 - 2504 ep_pol Jul27 ? 00:07:50 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 19780 5195 0 80 0 - 2504 ep_pol 08:59 ? 00:01:36 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 R <user> 19781 5195 0 80 0 - 2504 - 08:59 ? 00:01:33 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 S <user> 19782 5195 0 80 0 - 2504 ep_pol 08:59 ? 00:01:32 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 R <user> 19783 5195 0 80 0 - 2504 - 08:59 ? 00:01:32 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
0 Z <user> 26277 8729 0 80 0 - 0 exit 11:47 ? 00:00:02 [nagios] <defunct>
0 Z <user> 26278 8729 0 80 0 - 0 exit 11:47 ? 00:00:02 [nagios] <defunct>
0 Z <user> 26280 8729 0 80 0 - 0 exit 11:47 ? 00:00:02 [nagios] <defunct>
<user> 5195 1 0 Jul28 ? 00:34:10 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
<user> 5200 5195 0 Jul28 ? 00:00:51 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
<user> 8389 1 0 Jul27 ? 00:38:45 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
<user> 8391 8389 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 8392 8389 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 8393 8389 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 8394 8389 0 Jul27 ? 00:01:08 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
<user> 8470 1 0 Jul27 ? 00:39:00 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
<user> 8471 8470 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 8472 8470 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 8473 8470 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 8474 8470 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 8475 8470 0 Jul27 ? 00:01:02 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
<user> 8727 8729 0 Jul28 ? 00:31:52 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 8728 8729 0 Jul28 ? 00:32:15 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 8729 1 0 Jul28 ? 00:32:52 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
<user> 8731 8729 0 Jul28 ? 00:32:08 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 8732 8729 0 Jul28 ? 00:32:13 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 8802 8729 0 Jul28 ? 00:00:42 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
<user> 9109 1 0 Jul27 ? 00:38:53 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
<user> 9110 9109 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 9111 9109 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 9112 9109 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 9113 9109 0 Jul27 ? 00:00:00 [nagios] <defunct>
<user> 9114 9109 0 Jul27 ? 00:01:08 /usr/local/nagios/bin/nagios -u /usr/local/nagios/etc/nagios.cfg
<user> 9460 1 0 Jul28 ? 00:47:26 ../bin/nagios -u ../etc/nagios.cfg
<user> 9503 9460 0 Jul28 ? 00:22:23 /usr/local/nagios-4.1.1/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 9504 9460 0 Jul28 ? 00:22:59 /usr/local/nagios-4.1.1/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 9505 9460 0 Jul28 ? 00:22:10 /usr/local/nagios-4.1.1/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 9506 9460 0 Jul28 ? 00:22:09 /usr/local/nagios-4.1.1/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 9778 9460 0 Jul28 ? 00:00:49 ../bin/nagios -u ../etc/nagios.cfg
<user> 11549 1 0 Jul28 ? 00:48:48 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
<user> 11564 11549 0 Jul28 ? 00:22:23 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 11565 11549 0 Jul28 ? 00:21:59 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 11566 11549 0 Jul28 ? 00:22:10 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 11567 11549 0 Jul28 ? 00:22:10 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 12441 11549 0 Jul28 ? 00:00:52 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
<user> 12782 8389 0 Jul27 ? 00:41:37 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 12783 8389 0 Jul27 ? 00:41:59 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 12784 8389 0 Jul27 ? 00:41:32 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 12785 8389 0 Jul27 ? 00:41:56 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 14695 9109 0 Jul27 ? 00:41:16 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 14696 9109 0 Jul27 ? 00:40:46 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 14697 9109 0 Jul27 ? 00:41:29 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 14698 9109 0 Jul27 ? 00:41:34 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 15940 8470 0 Jul27 ? 00:41:35 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 15941 8470 0 Jul27 ? 00:41:41 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 15942 8470 0 Jul27 ? 00:41:28 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 15943 8470 0 Jul27 ? 00:41:20 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 19780 5195 0 Jul28 ? 00:33:17 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 19781 5195 0 Jul28 ? 00:32:10 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 19782 5195 0 Jul28 ? 00:32:23 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 19783 5195 0 Jul28 ? 00:32:45 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 21725 3484 0 05:11 pts/10 00:00:00 grep nagios
<user> 26277 8729 0 Jul28 ? 00:00:02 [nagios] <defunct>
<user> 26278 8729 0 Jul28 ? 00:00:02 [nagios] <defunct>
<user> 26280 8729 0 Jul28 ? 00:00:02 [nagios] <defunct>

Hi,
Seems like problem is resolved.
I stopped nagios and then killed all the extra processes displayed by ps -elf | grep nagios.
Now there are two nagios -d processes running. and 4 worker processes are running.
following is the output.

<user> 12539 1 2 07:28 ? 00:02:29 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
<user> 12550 12539 0 07:28 ? 00:00:02 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
<user> 17887 12539 0 09:13 ? 00:00:01 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 17888 12539 0 09:13 ? 00:00:01 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 17889 12539 0 09:13 ? 00:00:01 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 17890 12539 0 09:13 ? 00:00:01 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
<user> 25115 30451 0 09:18 pts/13 00:00:00 grep nagios
is this okay? or still it is problematic??
But couldn't understand the reason why nagios was creating multiple -d processes.

Yes multiple Nagios processes running will definitely cause a multitude of problems. The Nagios process forks itself to run checks therefore subprocesses are normal, as you now see:
-- this is normal (one parent and one child). But multiple parent processes as you saw earlier is most certainly a problem and a number of things could cause such.

is this okay? or still it is problematic??

This output looks fine - there are no multiple instances of nagios running as there is only one "main" process with PID=12539 and the rest of them are "child" processes that spun from this "parent" (PPID=12539). The bottom line is for the grep command, which is irrelevant.

But couldn't understand the reason why nagios was creating multiple -d processes.

Probably nagios was not stopped "cleanly". Perhaps you have a check that takes a very long time to execute. It's hard to say after the fact.
Let us know if you have any more issues/questions. Is it all right if we lock this topic, and mark it as "resolved"?

It looks like you had a handful of processes running before. What OS are you running? It looks like the process isn't properly restarting, and you're then being left with old processes running.

Hi All,

Thanks for you kind replies.
you can mark it as resolved topic.
Thanks once again.