2011R1.8 and Broken Graphs

[niebais]

Ok,
We updated to version 2011R1.8 on a Centos 5.5 system just this last week. None of our performance graphs display any more. When we click on the "performance graphs" tab we see the following error:
No performance graphs were found for this service.

Here are the steps I've taken so far.

I went into "Admin and Reset Security Credentials". I reset the admin password.
Next, I went into the DB fixup mode and I ran the /usr/local/nagios/libexec/repairmysql.sh nagios * command (following the DB repair PDF document of course)
and then I ran "reset_config_perms".

That didn't get things working so I went and started deleting rrd and xml files of some of the systems to see if it would repopulate the xml and rrd files. It doesn't and npcd is definitely running on the system. nagios 27102 1 2 15:11 ? 00:00:57 /usr/local/nagios/bin/npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg

Our junior admin tried running a "fix perms" script on the system and that didn't fix it either. At this point, is there some way to completely reset the graphs? If you guys can come up with a cleaner way of fixing this up, that would be great.

[SDohmen]

We have the same thing and also no solution for this yet.

[mguthrie]

Can you access the following URL on your XI servers (make sure you're logged in) and post any error message you get in either the web browser or the apache log from accessing this page.

http://<youraddress>/nagiosxi/includes/components/perfdata/graphApi.php?host=localhost

[mguthrie]

Ok, we found it. We lucked out and had a test box that experienced the same thing. Here's the fix:

Code: Select all

chmod -R +x /usr/local/nagios/share/perfdata/

We'll update our install/upgrade scripts accordingly. It appears that doesn't affect all systems.

[niebais]

Ok, here's what I see so far.

When I connect to the http site I get this message

Can you access the following URL on your XI servers (make sure you're logged in) and post any error message you get in either the web browser or the apache log from accessing this page.

http://<youraddress>/nagiosxi/includes/components/perfdata/graphApi.php?host=localhost

I get this message
No performance data available

Ok, we found it. We lucked out and had a test box that experienced the same thing. Here's the fix:

Code: Select all

chmod -R +x /usr/local/nagios/share/perfdata/

We'll update our install/upgrade scripts accordingly. It appears that doesn't affect all systems.

The permissions were already 0755 drwxr-xr-x 991 nagios nagios 36864 Nov 15 07:47 perfdata

I also found this in the performance data logs.
2011-11-14 05:09:36 [1588] [0] *** TIMEOUT: Please check your npcd.cfg
2011-11-14 05:09:36 [1588] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//service-perfdata.1321272566-PID-1588 deleted
2011-11-14 05:09:36 [1588] [0] *** Timeout while processing Host: "myhost.nuskin.net" Service: "MEMORY__usage"
2011-11-14 05:09:36 [1588] [0] *** process_perfdata.pl terminated on signal ALRM

It's really odd we're seeing this all over. This is the last entry we see in the performance logs.

[mguthrie]

Just for the sake of troubleshooting, lets try a few things.

Code: Select all

chmod -R 777 /usr/local/nagios/share/perfdata
killall npcd
service npcd start
service httpd restart

Let me know what you see in the npcd.log file and also if this allows the graphs to show up in the browser.

[niebais]

There we go! Thanks! The graphs are coming back up.

[mguthrie]

Good deal, thanks for the update!