Page 1 of 4
Performance Graphs tab freezing browser
Posted: Fri Oct 06, 2017 9:51 am
by gwakem
XI 5.4.8, tested in Firefox 55.0.3 64 bit and Chrome 61.0.3163.100 (Official Build) (64-bit).
When I access the performance Graphs tab of a specific service check, the browser freezes and has to be restarted. This does NOT happen with all service performance graphs, and it does not happen when I access the graph symbol to bring up the pop up window with graphs from the host, showing all services.
I do not want to loose all the data in this specific graph, is there a way to "fix" the rrd or diagnose the issue to determine next steps?
Re: Performance Graphs tab freezing browser
Posted: Fri Oct 06, 2017 2:16 pm
by tgriep
It could be some Apache and PHP settings causing the issue, try doing the following on the Nagios server.
Edit the /etc/php.ini file and change the following from
Code: Select all
max_execution_time = 30
max_input_time = 60
memory_limit = 128M
to
Code: Select all
max_execution_time = 60
max_input_time = 120
memory_limit = 512M
add this to the bottom of that file
Save the file
Then edit this file
add this to the bottom of that file
Save the file and restart Apache for the changes to take affect.
Then clear your browsers cache and see if that fixes the issue.
Re: Performance Graphs tab freezing browser
Posted: Tue Oct 10, 2017 12:34 pm
by gwakem
It turns out I already had most of those settings adjusted at or above your recommended settings. I upped the memory_limit from 256M to 512M in the php.ini, and added LimitRequestLine 100000 to the bottom of the httpd.conf, and then cleared cache and restarted.
Unfortunately it still happened.
I used the firefox developer tools to attempt to diagnose where the slowdown was occurring, and it appears it's stalling out on the call to the gauge. Now, this check is dealing with byte size on a file on hpux, so the output is something like 2048141587, and I suspect that the gauge is having a liiiiiiiiittle issue handling that. Here is the call itself that makes the browser start asking if I want to stop the scripts running that are slowing things down:
Code: Select all
https://$SERVER/nagiosxi//includes/dashlets/gauges/getdata.php?host=$HOST&service=NRPE%20-%20DRY%20TRAN.DAT&ds=_triprod_dry_TRAN.DAT
Any idea on how best to handle this?
A interesting side note, I was watching my memory. By the time I closed the page after about 40 seconds or so, the used memory on my system had ballooned from the normal 3.5GB to 10.6GB. Killing the perfdata tab immediately returned my memory usage back to 3.5GB.
Re: Performance Graphs tab freezing browser
Posted: Tue Oct 10, 2017 1:06 pm
by tgriep
It could be that the rrd file is corrupted on the system and the following process is a way to dump the rrd file and restore it back as an rrd file to see if that fixes it.
Run the following commands and replace the text between the < > with valid names.
Code: Select all
cd /usr/local/nagios/share/perfdata/<hostname>
rrdtool dump <original servicename>.rrd > temp.xml
mv <original service>.rrd backup.rrd
rrdtool restore temp.xml <original servicename>.rrd
Then see if you can view the Graph.
If not, can you post the .rrd and .xml file here so we can view them?
Re: Performance Graphs tab freezing browser
Posted: Fri Oct 13, 2017 7:45 am
by gwakem
Unfortunately, that didn't do the trick. I verified the permissions on the file were good, but its displaying the same behavior. One thing I should note: when using the graph link next to the service itself, (when looking at the host with all services below the host, there is the red and blue lines that you can click to open the graph in a pop up window,) it doesn't freeze and you can adjust to view specific lengths of time with no issues. The RRD displays fine there. It's only when using the performance graphs tab does it freeze when attempting to pull the gauge unit.
Re: Performance Graphs tab freezing browser
Posted: Fri Oct 13, 2017 7:48 am
by gwakem
I have attached the RRD and XML.
Re: Performance Graphs tab freezing browser
Posted: Fri Oct 13, 2017 2:19 pm
by dwasswa
Hi
@ gwakem,
Try running this command
while you try viewing the graph and post the output.
Re: Performance Graphs tab freezing browser
Posted: Mon Oct 16, 2017 8:43 am
by gwakem
I have attached two files: one that shows the entirety of the output, which is busy as multiple people are hitting it, and a second one (the http_output_filtered.txt) that greps based on my workstation's IP.
Re: Performance Graphs tab freezing browser
Posted: Mon Oct 16, 2017 11:34 am
by dwasswa
Hi
@gwakem,
Can you please run the following comands and post the output.
Code: Select all
ls /usr/local/nagios/var/spool/perfdata/ | wc -l
ls /usr/local/nagios/var/spool/xidpe/ | wc -l
Re: Performance Graphs tab freezing browser
Posted: Mon Oct 16, 2017 2:41 pm
by gwakem
Here you go:
Code: Select all
# ls /usr/local/nagios/var/spool/perfdata/ | wc -l
3
# ll /usr/local/nagios/var/spool/perfdata/
total 28K
drwxr-xr-x 2 nagios nagios 131 Oct 16 15:42 .
drwxr-xr-x 5 nagios nagios 52 Feb 18 2016 ..
-rw-r--r-- 1 nagios nagios 0 Feb 22 2016 1456142744.perfdata.host-PID-20394
-rw-r--r-- 1 nagios nagios 0 Feb 22 2016 1456142744.perfdata.service-PID-20395
-rw-r--r-- 1 nagios nagios 25K Mar 23 2016 1458754280.perfdata.host-PID-3348
# ls /usr/local/nagios/var/spool/xidpe/ | wc -l
0