I am setting up nagios on a new server (RHEL 7) and it was working fine for a couple weeks and then all of a sudden it just stalled. Nagios itself still runs, but nothing is happening. status.dat doesn't update, there is nothing in nagios.log or nagios.debug. The data in spool/checkresults just piles up. We saw a few zombie processes, so we restarted nagios, but then it would run some tests for less than a minute and then just stall.
We can't think of anything that changed on the system to have caused the behavior to change.
Today I tried running nagios with strace to see if we could see what was causing it to stall. But when I run it with strace, nagios works flawlessly. Our theory is that it slows nagios down enough to get around whatever the issue is. But we're not sure what to do from here.
Has anyone else seen anything like this before?
nagios stalls unless I use strace
- Box293
- Too Basu
- Posts: 5126
- Joined: Sun Feb 07, 2010 10:55 pm
- Location: Deniliquin, Australia
- Contact:
Re: nagios stalls unless I use strace
Run this command to see if the nagios user is expired:
If it is, run this command to enable the expired nagios user:
If that isn't the problem then try setting the debug level on and then restart Nagios.
Do you see anything logged in the file /usr/local/nagios/var/nagios.debug
When you are finished this turns debugging off:
Code: Select all
chage -l nagios
Code: Select all
chage -I -1 -m 0 -M 99999 -E -1 nagios
Code: Select all
sed -i 's/.*debug_level=.*/debug_level=-1/g' /usr/local/nagios/etc/nagios.cfg
service nagios restart
When you are finished this turns debugging off:
Code: Select all
sed -i 's/.*debug_level=.*/debug_level=0/g' /usr/local/nagios/etc/nagios.cfg
service nagios restart
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Re: nagios stalls unless I use strace
It turns out that it was set up to write the perflogs to a different filesystem and once I changed them to be on the same filesystem, everything started working. So we'll need to look into what's going on with those filesystems, but that's not a Nagios problem. Thanks!
- Box293
- Too Basu
- Posts: 5126
- Joined: Sun Feb 07, 2010 10:55 pm
- Location: Deniliquin, Australia
- Contact:
Re: nagios stalls unless I use strace
Awesome, great to hear.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.