Nagios is Frozen

[furqan]

Hello everybody, i am using nagios core 4.0.6 on centos 6.x. last wednesday we had a power shortage and as soon as the power returned nagios server was restarted , but the problem i am facing now is that web interace is accessebile but its frozen. what i mean by frozen is : no email notifications, log file not updating since last wednesday, cannot commit restart from web page, cannot commit scheduling queue. personally i have tried every possibile thing which i could, like checking any unwanted process, killing process. restarting service nagios, restarting apache, restarting syslog.
and i dont have any idea what and how it went wrong. please try to reply me even if you know anything closer to that.
thankyou

[slansing]

What is the current output of:

Code: Select all

service nagios status

service ndo2db status

Code: Select all

tail -50 /var/log/messages

You said that the web interface is accessible, but what happens when you do try to use it? Do you get strange error messages? Can you not actually click links or buttons? Can you scroll through a page which has a scroll bar? Does the nagios log say anything at all? From around that time?:

Code: Select all

tail -30 /usr/local/nagios/var/nagios.log

tail -30 /var/log/httpd/error_log

Please use CODE wraps, highlight any of the text that you are pasting from your SSH session and click the "Code" button above your reply's text box.

[furqan]

Code: Select all

    service nagios status
is running
    service ndo2db status
unrecognized service 

yes web interface is accessebile with all its functionalities but the last check of the services i see there is of 30-04-2014 12:00 , and same is the case if i go to event log via web interface i see log till 30-04-2014 12:00 . trying to commit to force the new schedulling time it takes a lot of time and in the end tells me gateway timout error. looking inside /usr/local/nagios/var/archives i see files till 30/04/2014.


[img]

[/img]
[img]

[/img]
[img]

[/img]

i am sorry i was unable to copy it so i upload some screen shots.
thanks a million for your support.

[sreinhardt]

In your last image, you are showing a nagios version of 3.5.1, could you show what you did to install 4.0.6 and where it is showing you that it is that version? /var/log/messages, doesn't seem to show anything in particular pertaining to nagios, unless you have vastly changed the directory names. I would agree though, that your nagios log does not show any activity since 4/30 for one reason or another.

[furqan]

followed this link to upgrade http://nagios.sourceforge.net/docs/3_0/upgrading.html . followed each step except first two. and yes i didnt change any directory, and i say i have an upgraded version because i see it on my web interface (homepage), anyway i dont get any solution or any hint how to solve this issue. If you can find me any solution i would be very thankful.

[lmiltchev]

Can you run the following commands and show us the output in code wraps?

Code: Select all

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Note: Modify the paths to nagios and nagios.cfg if you need to.

Code: Select all

grep nag /etc/group

[furqan]

Code: Select all

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Results
[img]

[/img]

Code: Select all

grep nag /etc/group

Results

Code: Select all

nagios:x:500:apache
nagcmd:x:501:nagios,apache

[furqan]

Well checking the archive log messages in /var/log
that server is only executing checks on the localhost.
btw i am sending you the logs from the moment it stopped working, there are several thousands line which i dont understand. please help me if they are comprehensive for you.
i am attaching the file because it has too many characters.


hint:power went off at this hour Apr 30 12:05:33.

Want to add one more thing, i tried to execute checks on remote hosts via terminal and they work fine.

[sreinhardt]

When you installed nagios previously, or the past tech did, was this done via source install or rpm? If you aren't sure, the easiest way to tell is:

Code: Select all

rpm -qa | grep -i nagios

[furqan]

i tried this command and i the results, i am sorry that everytime i attach a screen shot , just because i am unable to copy and paste from terminal.
[img]

[/img]
i am available for any further information.