Page 1 of 1
Availabibty Report
Posted: Mon Aug 31, 2015 3:11 am
by whitest
When I try to build a report more then for 7 days I see empty result (see attach) after ~1.5 minutes response. Avail.report for 7 days works fine.
Nagios 4.0.8. My installation is Centos 5.11 (kernal 2.6.18-371.1.2.el5), 4 Gb RAM (3 visible by OS), 4CPU:
Code: Select all
[root@gml-nms-01 ~]# [b]grep -c ^processor /proc/cpuinfo[/b]
4
Code: Select all
[root@gml-nms-01 ~]# [b]free -m[/b]
total used free shared buffers cached
Mem: 3034 600 2433 0 43 403
-/+ buffers/cache: 153 2881
Swap: 4031 0 4031
I have a lot of .log files in archive folder. Each of them is about 10Mb:
Code: Select all
[root@gml-nms-01 ~]# [b]du -sh /mnt/sdb1/nagioslogs/* | more[/b]
9.3M /mnt/sdb1/nagioslogs/nagios-01-01-2015-00.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-01-2015-01.log
9.6M /mnt/sdb1/nagioslogs/nagios-01-01-2015-02.log
9.4M /mnt/sdb1/nagioslogs/nagios-01-01-2015-03.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-01-2015-04.log
9.5M /mnt/sdb1/nagioslogs/nagios-01-01-2015-05.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-01-2015-06.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-01-2015-07.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-01-2015-08.log
11M /mnt/sdb1/nagioslogs/nagios-01-01-2015-09.log
10M /mnt/sdb1/nagioslogs/nagios-01-01-2015-10.log
10M /mnt/sdb1/nagioslogs/nagios-01-01-2015-11.log
10M /mnt/sdb1/nagioslogs/nagios-01-01-2015-12.log
9.9M /mnt/sdb1/nagioslogs/nagios-01-01-2015-13.log
9.9M /mnt/sdb1/nagioslogs/nagios-01-01-2015-14.log
10M /mnt/sdb1/nagioslogs/nagios-01-01-2015-15.log
9.9M /mnt/sdb1/nagioslogs/nagios-01-01-2015-16.log
9.9M /mnt/sdb1/nagioslogs/nagios-01-01-2015-17.log
9.2M /mnt/sdb1/nagioslogs/nagios-01-01-2015-18.log
9.2M /mnt/sdb1/nagioslogs/nagios-01-01-2015-19.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-01-2015-20.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-01-2015-21.log
9.4M /mnt/sdb1/nagioslogs/nagios-01-01-2015-22.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-01-2015-23.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-02-2015-00.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-02-2015-01.log
9.6M /mnt/sdb1/nagioslogs/nagios-01-02-2015-02.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-02-2015-03.log
9.2M /mnt/sdb1/nagioslogs/nagios-01-02-2015-04.log
9.5M /mnt/sdb1/nagioslogs/nagios-01-02-2015-05.log
9.2M /mnt/sdb1/nagioslogs/nagios-01-02-2015-06.log
9.3M /mnt/sdb1/nagioslogs/nagios-01-02-2015-07.log
9.2M /mnt/sdb1/nagioslogs/nagios-01-02-2015-08.log
10M /mnt/sdb1/nagioslogs/nagios-01-02-2015-09.log
9.9M /mnt/sdb1/nagioslogs/nagios-01-02-2015-10.log
9.8M /mnt/sdb1/nagioslogs/nagios-01-02-2015-11.log
9.9M /mnt/sdb1/nagioslogs/nagios-01-02-2015-12.log
9.8M /mnt/sdb1/nagioslogs/nagios-01-02-2015-13.log
9.9M /mnt/sdb1/nagioslogs/nagios-01-02-2015-14.log
9.8M /mnt/sdb1/nagioslogs/nagios-01-02-2015-15.log
9.8M /mnt/sdb1/nagioslogs/nagios-01-02-2015-16.log
9.9M /mnt/sdb1/nagioslogs/nagios-01-02-2015-17.log
9.2M /mnt/sdb1/nagioslogs/nagios-01-02-2015-18.log
... and so on...
In php.ini I tried to confure, but without success:
Code: Select all
max_input_time = 180
memory_limit = 1024M
In nagios.conf I use the following settings:
Code: Select all
status_update_interval=10
log_rotation_method=h
use_syslog=0
log_notifications=1
log_service_retries=1
log_host_retries=1
log_event_handlers=1
log_initial_states=0
log_external_commands=1
log_passive_checks=1
Can somebody give me advice how to get reports to work? What is the best free solution?
Re: Availabibty Report
Posted: Mon Aug 31, 2015 4:19 pm
by tgriep
Can you run the following tail command, recreate the report and post the output of the tail command here?
Re: Availabibty Report
Posted: Mon Aug 31, 2015 11:20 pm
by whitest
Here is log When I user report for 31 days:
Code: Select all
[Tue Sep 01 07:11:11 2015] [warn] [client 10.3.100.242] Timeout waiting for output from CGI script /usr/local/nagios/sbin/avail.cgi, referer: http://1.1.1.1/nagios/cgi-bin/avail.cgi?get_date_parts=&hostgroup=gml-azs370006
[Tue Sep 01 07:11:11 2015] [error] [client 10.3.100.242] (70007)The timeout specified has expired: ap_content_length_filter: apr_bucket_read() failed, referer: http://1.1.1.1/nagios/cgi-bin/avail.cgi?get_date_parts=&hostgroup=gml-azs370006
I upgraded kernel for this box, so this virtual machine now can see all 4 GB RAM:
Code: Select all
[root@gml-nms-01 ~]# free -m
total used free shared buffers cached
Mem: 4050 2935 1114 0 188 2597
-/+ buffers/cache: 149 3901
Swap: 4031 0 4031
Report more than 7 days not working on all my large nagios-servers. Servers with small load works fine.
Re: Availabibty Report
Posted: Tue Sep 01, 2015 11:12 am
by jdalrymple
When you run this report, Nagios chews through all of your archive files to generate it. If you have a lot of files and/or slow disks the report obviously can
timeout. That's why we made NDO, but unfortunately there aren't many projects taking advantage of NDO to replace the availability functionality using NDO.
http://httpd.apache.org/docs/current/mo ... ml#timeout
Adjust this to a suitable number and your problems will likely go away - but be aware you could be waiting for a long time if you don't have fast enough disks to handle the amount of data that needs to be looked at.
Re: Availabibty Report
Posted: Tue Sep 01, 2015 2:36 pm
by whitest
Thank you for your advice,
jdalrymple.
I have increased
Apache TimeOut directive from 120 to 360 seconds and it solved my problem. Of cause it's not perfect, but something is better then nothing
I tried to generate availability report for hostgroup with custom Report Period (from 1st May to 1st Sept) and it's worked. Avail.cgi spent ~180 seconds for it.
The same report with 7 days Report Period spent 15 seconds.
Backtracked Archives (To Scan For Initial States) = 0.
I have 7 841 files in archive folder. Most of them have size about 10 Mb, rest are ~1 Mb. The oldest file is from 31.12.2014.
Few questions:
1. I agree Files is not good for large installations. May be it's newbie question, but why Nagios don't use RRD (as pnp4nagios) as built-in storage of archive data? I guess RRD is much more rapid and stable than MySQL. I afraid high CPU usage from MySQL =|
2. Is NDOUtils it broker? I use mod_gearman to recieve check results from remote nagioses. Will I have able to use mod_gearman?
3. Could you please explain how to rename
host_name or
service_description and retain data for reports?
Re: Availabibty Report
Posted: Tue Sep 01, 2015 4:51 pm
by jdalrymple
1) The reporting component of NagiosXI is much more advanced. It offers instant results as well as many other features. I don't mean to sound like a salesman, but that's the quick way to get advanced reporting.
2) NDOutils and ModGearman can coexist, and either one can exist without the other. They are both useful tools that do very different things.
3)I'm not sure I understand the question. Are you simply wanting some of the labels or verbiage on the report page to be changed? If so that will require a recompile of the CGIs.
Re: Availabibty Report
Posted: Wed Sep 02, 2015 1:54 am
by whitest
jdalrymple wrote:3)I'm not sure I understand the question. Are you simply wanting some of the labels or verbiage on the report page to be changed? If so that will require a recompile of the CGIs.
Example: I have the host "ASA5510". Now I want to rename the host to "ASA5520" and this host must inhirit all history from ASA5510.
Re: Availabibty Report
Posted: Wed Sep 02, 2015 9:54 am
by jdalrymple
I see. Unfortunately that would require manual search/replace in all of your logs. The good news is that's fairly trivial, the bad news is though that there is no way for Nagios to handle this. Once host_name changes it's effectively a new host to Nagios with a totally new history.
Re: Availabibty Report
Posted: Wed Sep 02, 2015 10:40 am
by whitest
jdalrymple wrote:I see. Unfortunately that would require manual search/replace in all of your logs. The good news is that's fairly trivial, the bad news is though that there is no way for Nagios to handle this. Once host_name changes it's effectively a new host to Nagios with a totally new history.
Thanks!