Page 1 of 2
Nagios is Frozen
Posted: Tue May 06, 2014 7:30 am
by furqan
Hello everybody, i am using nagios core 4.0.6 on centos 6.x. last wednesday we had a power shortage and as soon as the power returned nagios server was restarted , but the problem i am facing now is that web interace is accessebile but its frozen. what i mean by frozen is : no email notifications, log file not updating since last wednesday, cannot commit restart from web page, cannot commit scheduling queue. personally i have tried every possibile thing which i could, like checking any unwanted process, killing process. restarting service nagios, restarting apache, restarting syslog.
and i dont have any idea what and how it went wrong. please try to reply me even if you know anything closer to that.
thankyou
Re: Nagios is Frozen
Posted: Tue May 06, 2014 9:30 am
by slansing
What is the current output of:
Code: Select all
service nagios status
service ndo2db status
You said that the web interface is accessible, but what happens when you do try to use it? Do you get strange error messages? Can you not actually click links or buttons? Can you scroll through a page which has a scroll bar? Does the nagios log say anything at all? From around that time?:
Code: Select all
tail -30 /usr/local/nagios/var/nagios.log
tail -30 /var/log/httpd/error_log
Please use CODE wraps, highlight any of the text that you are pasting from your SSH session and click the "Code" button above your reply's text box.
Re: Nagios is Frozen
Posted: Tue May 06, 2014 10:21 am
by furqan
Code: Select all
service nagios status
is running
service ndo2db status
unrecognized service
yes web interface is accessebile with all its functionalities but the last check of the services i see there is of 30-04-2014 12:00 , and same is the case if i go to event log via web interface i see log till 30-04-2014 12:00 . trying to commit to force the new schedulling time it takes a lot of time and in the end tells me gateway timout error. looking inside /usr/local/nagios/var/archives i see files till 30/04/2014.
[img]
[/img]
[img]
[/img]
[img]
[/img]
i am sorry i was unable to copy it so i upload some screen shots.
thanks a million for your support.
Re: Nagios is Frozen
Posted: Tue May 06, 2014 11:21 am
by sreinhardt
In your last image, you are showing a nagios version of 3.5.1, could you show what you did to install 4.0.6 and where it is showing you that it is that version? /var/log/messages, doesn't seem to show anything in particular pertaining to nagios, unless you have vastly changed the directory names. I would agree though, that your nagios log does not show any activity since 4/30 for one reason or another.
Re: Nagios is Frozen
Posted: Tue May 06, 2014 12:40 pm
by furqan
followed this link to upgrade
http://nagios.sourceforge.net/docs/3_0/upgrading.html . followed each step except first two. and yes i didnt change any directory, and i say i have an upgraded version because i see it on my web interface (homepage), anyway i dont get any solution or any hint how to solve this issue. If you can find me any solution i would be very thankful.
Re: Nagios is Frozen
Posted: Tue May 06, 2014 1:02 pm
by lmiltchev
Can you run the following commands and show us the output in code wraps?
Code: Select all
/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Note: Modify the paths to nagios and nagios.cfg if you need to.
Re: Nagios is Frozen
Posted: Wed May 07, 2014 1:53 am
by furqan
Code: Select all
/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Results
[img]
[/img]
Results
Code: Select all
nagios:x:500:apache
nagcmd:x:501:nagios,apache
Re: Nagios is Frozen
Posted: Wed May 07, 2014 2:30 am
by furqan
Well checking the archive log messages in /var/log
that server is only executing checks on the localhost.
btw i am sending you the logs from the moment it stopped working, there are several thousands line which i dont understand. please help me if they are comprehensive for you.
i am attaching the file because it has too many characters.
hint:power went off at this hour Apr 30 12:05:33.
Want to add one more thing, i tried to execute checks on remote hosts via terminal and they work fine.
Re: Nagios is Frozen
Posted: Wed May 07, 2014 4:32 pm
by sreinhardt
When you installed nagios previously, or the past tech did, was this done via source install or rpm? If you aren't sure, the easiest way to tell is:
Re: Nagios is Frozen
Posted: Thu May 08, 2014 7:30 am
by furqan
i tried this command and i the results, i am sorry that everytime i attach a screen shot , just because i am unable to copy and paste from terminal.
[img]
[/img]
i am available for any further information.