Nagios Freezes and Host Status not getting Updated

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Teja
Posts: 53
Joined: Tue Jun 13, 2017 8:13 am

Nagios Freezes and Host Status not getting Updated

Post by Teja »

Hi All,
I have "Nagios 4.0.2" running on "Ubuntu 14.04.5 LTS". Now the problem i am facing is that it freezes frequently and host status is not updating .So once i kill the "ndo2db processes" and Restart its working but only for few hours and again the same problem, and FYI earlier i used to have this issue but once i have restarted the process it could have worked properly, Now it very frequent and the only change i made in the database is that Setting the "Expire binlog days = 7" as bin logs are taking lot of volume. Will that effect ? So please could anyone help me on this issue.

-> No Errors when i check with "/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg" Command.

-> Below are the commands that i shoot when it freezes,correct me if i am doing anything wrong.


Thanks in Advance!!

Code: Select all

ps -aux | grep 'ndo2db.cfg'
kill -9 <process id>
rm /usr/local/nagios/var/ndo.sock
/usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nagios Freezes and Host Status not getting Updated

Post by scottwilkerson »

Can you share the following files?
/usr/local/nagios/etc/ndo2db.cfg
/usr/local/nagios/etc/nagios.cfg
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Teja
Posts: 53
Joined: Tue Jun 13, 2017 8:13 am

Re: Nagios Freezes and Host Status not getting Updated

Post by Teja »

Hi scottwilkerson,
Here are the Attachments. Thank You!
Attachments
nagios.cfg
(45.56 KiB) Downloaded 412 times
ndo2db.cfg
(4.43 KiB) Downloaded 443 times
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nagios Freezes and Host Status not getting Updated

Post by scottwilkerson »

in your ndo2db.cfg you have the following

Code: Select all

debug_level=2
This will cause an ton of logging to be done, which on a busy system can really slow down as the disk takes a hit.

Change to

Code: Select all

debug_level=0
and then restart ndo2db

Let us know if you have further problems.
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Teja
Posts: 53
Joined: Tue Jun 13, 2017 8:13 am

Re: Nagios Freezes and Host Status not getting Updated

Post by Teja »

Thanks much scottwilkerson. I have just changed the "debug_level=0" & Restarted the service. After Verifying for couple of hours will confirm the Status and could you please tell me is their any other process for restarting the service rather following the below steps?

Code: Select all

ps -aux | grep 'ndo2db.cfg'
kill -9 <process id>
rm /usr/local/nagios/var/ndo.sock
/usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
As when i do "service ndo2db restart" it says unrecognised service.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nagios Freezes and Host Status not getting Updated

Post by scottwilkerson »

This sounds like you didn't do part of the make process

Code: Select all

make install-init
would add the init script so the service would be recognized.
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Teja
Posts: 53
Joined: Tue Jun 13, 2017 8:13 am

Re: Nagios Freezes and Host Status not getting Updated

Post by Teja »

Hi scottwilkerson,
When i Just Hit tail command on todays Log it shows the below Output,will that effect the system??

Code: Select all

[1504075803] Caught SIGTERM, shutting down...
[1504075803] ndomod: Error writing to data sink!  Some output may get lost...
[1504075803] ndomod: Please check remote ndo2db log, database connection or SSL Parameters
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nagios Freezes and Host Status not getting Updated

Post by scottwilkerson »

This is likely from you killing the ndo2db process and then ndomod cannot write to the data sink (ndo2db) which happened the same second.
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Teja
Posts: 53
Joined: Tue Jun 13, 2017 8:13 am

Re: Nagios Freezes and Host Status not getting Updated

Post by Teja »

So how can i resolve this friend? Do i need to restart the process again or should i wait?

Thank you!
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nagios Freezes and Host Status not getting Updated

Post by scottwilkerson »

Teja wrote:So how can i resolve this friend? Do i need to restart the process again or should i wait?

Thank you!
If you do not continue to get more of these logs there isn't anything to resolve, it would be normal to get this error when you kill the ndo2db process
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Locked