Page 1 of 1

2011R1.8 and Broken Graphs

Posted: Mon Nov 14, 2011 5:59 pm
by niebais
Ok,
We updated to version 2011R1.8 on a Centos 5.5 system just this last week. None of our performance graphs display any more. When we click on the "performance graphs" tab we see the following error:
No performance graphs were found for this service.

Here are the steps I've taken so far.

I went into "Admin and Reset Security Credentials". I reset the admin password.
Next, I went into the DB fixup mode and I ran the /usr/local/nagios/libexec/repairmysql.sh nagios * command (following the DB repair PDF document of course)
and then I ran "reset_config_perms".

That didn't get things working so I went and started deleting rrd and xml files of some of the systems to see if it would repopulate the xml and rrd files. It doesn't and npcd is definitely running on the system. nagios 27102 1 2 15:11 ? 00:00:57 /usr/local/nagios/bin/npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg

Our junior admin tried running a "fix perms" script on the system and that didn't fix it either. At this point, is there some way to completely reset the graphs? If you guys can come up with a cleaner way of fixing this up, that would be great.

Re: 2011R1.8 and Broken Graphs

Posted: Tue Nov 15, 2011 4:48 am
by SDohmen
We have the same thing and also no solution for this yet.

Re: 2011R1.8 and Broken Graphs

Posted: Tue Nov 15, 2011 10:48 am
by mguthrie
Can you access the following URL on your XI servers (make sure you're logged in) and post any error message you get in either the web browser or the apache log from accessing this page.

http://<youraddress>/nagiosxi/includes/components/perfdata/graphApi.php?host=localhost

Re: 2011R1.8 and Broken Graphs

Posted: Tue Nov 15, 2011 11:33 am
by mguthrie
Ok, we found it. We lucked out and had a test box that experienced the same thing. Here's the fix:

Code: Select all

chmod -R +x /usr/local/nagios/share/perfdata/
We'll update our install/upgrade scripts accordingly. It appears that doesn't affect all systems.

Re: 2011R1.8 and Broken Graphs

Posted: Tue Nov 15, 2011 12:07 pm
by niebais
Ok, here's what I see so far.

When I connect to the http site I get this message
mguthrie wrote:Can you access the following URL on your XI servers (make sure you're logged in) and post any error message you get in either the web browser or the apache log from accessing this page.

http://<youraddress>/nagiosxi/includes/components/perfdata/graphApi.php?host=localhost
I get this message
No performance data available
mguthrie wrote:Ok, we found it. We lucked out and had a test box that experienced the same thing. Here's the fix:

Code: Select all

chmod -R +x /usr/local/nagios/share/perfdata/
We'll update our install/upgrade scripts accordingly. It appears that doesn't affect all systems.
The permissions were already 0755 drwxr-xr-x 991 nagios nagios 36864 Nov 15 07:47 perfdata

I also found this in the performance data logs.
2011-11-14 05:09:36 [1588] [0] *** TIMEOUT: Please check your npcd.cfg
2011-11-14 05:09:36 [1588] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//service-perfdata.1321272566-PID-1588 deleted
2011-11-14 05:09:36 [1588] [0] *** Timeout while processing Host: "myhost.nuskin.net" Service: "MEMORY__usage"
2011-11-14 05:09:36 [1588] [0] *** process_perfdata.pl terminated on signal ALRM

It's really odd we're seeing this all over. This is the last entry we see in the performance logs.

Re: 2011R1.8 and Broken Graphs

Posted: Tue Nov 15, 2011 12:23 pm
by mguthrie
Just for the sake of troubleshooting, lets try a few things.

Code: Select all

chmod -R 777 /usr/local/nagios/share/perfdata
killall npcd
service npcd start
service httpd restart
Let me know what you see in the npcd.log file and also if this allows the graphs to show up in the browser.

Re: 2011R1.8 and Broken Graphs

Posted: Tue Nov 15, 2011 1:22 pm
by niebais
There we go! Thanks! The graphs are coming back up.

Re: 2011R1.8 and Broken Graphs

Posted: Tue Nov 15, 2011 5:59 pm
by mguthrie
Good deal, thanks for the update!