Nagios XI reorts are not coming or very very slow

[tgriep]

The top output does show that the system has a bit of I/O wait. At the screen capture, 1.6% I/O Wait.

Is the XI server hosted on AWS?
If so, what it is provisioned at on AWS?

Run the following commands and post the output so I can verify the patches were done.

Code: Select all

ls -l /usr/local/nagiosxi/html/reports/sla.php
ls -l /usr/local/nagiosxi/html/reports/availability.php
ls -l /usr/local/nagiosxi/html/includes/utils-reports.inc.php
ls -l /usr/local/nagios/sbin

[biswajit.banerjee]

Yes XI is provisioned on AWS
Second Question is not clear

# ls -l /usr/local/nagiosxi/html/reports/sla.php
-rw-r--r-- 1 nagios nagios 685509 Feb 24 07:48 /usr/local/nagiosxi/html/reports/sla.php

# ls -l /usr/local/nagiosxi/html/reports/availability.php
-rwxr-xr-- 1 nagios nagios 136505 Jan 2 11:33 /usr/local/nagiosxi/html/reports/availability.php

# ls -l /usr/local/nagiosxi/html/includes/utils-reports.inc.php
-rwxr-xr-- 1 nagios nagios 62142 Jan 2 11:33 /usr/local/nagiosxi/html/includes/utils-reports.inc.php

# ls -l /usr/local/nagios/sbin
total 5900
-rwxrwxr-x 1 nagios nagios 324760 Feb 24 07:41 archivejson.cgi
-rwxrwxr-x 1 nagios nagios 310160 Feb 24 07:41 avail.cgi
-rwxrwxr-x 1 nagios nagios 301664 Feb 24 07:41 cmd.cgi
-rwxrwxr-x 1 nagios nagios 273208 Feb 24 07:41 config.cgi
-rwxrwxr-x 1 nagios nagios 314224 Feb 24 07:41 extinfo.cgi
-rwxrwxr-x 1 nagios nagios 269304 Feb 24 07:41 histogram.cgi
-rwxrwxr-x 1 nagios nagios 248688 Feb 24 07:41 history.cgi
-rwxrwxr-x 1 nagios nagios 244576 Feb 24 07:41 notifications.cgi
-rwxrwxr-x 1 nagios nagios 322456 Feb 24 07:41 objectjson.cgi
-rwxrwxr-x 1 nagios nagios 240440 Feb 24 07:41 outages.cgi
-rwxrwxr-x 1 nagios nagios 236304 Oct 2 08:54 outages-xml.cgi
-rwxrwxr-x 1 nagios nagios 240456 Feb 24 07:41 showlog.cgi
-rwxrwxr-x 1 nagios nagios 318336 Feb 24 07:41 status.cgi
-rwxrwxr-x 1 nagios nagios 240392 Oct 2 08:54 status-json.cgi
-rwxrwxr-x 1 nagios nagios 320600 Feb 24 07:41 statusjson.cgi
-rwxrwxr-x 1 nagios nagios 265272 Feb 24 07:41 statusmap.cgi
-rwxrwxr-x 1 nagios nagios 261000 Feb 24 07:41 statuswml.cgi
-rwxrwxr-x 1 nagios nagios 244584 Feb 24 07:41 statuswrl.cgi
-rwxrwxr-x 1 nagios nagios 265112 Feb 24 07:41 summary.cgi
-rwxrwxr-x 1 nagios nagios 256920 Feb 24 07:41 tac.cgi
-rwxrwxr-x 1 nagios nagios 240488 Oct 2 08:54 tac-xml.cgi
-rwxrwxr-x 1 nagios nagios 273400 Feb 24 07:41 trends.cgi

[tgriep]

Sorry about the confusion, I was asking what instance type the server is hosted on at AWS. See this link.
https://aws.amazon.com/ec2/instance-types/


Run this as root on the nagios server and capture the console log to a file if possible as we need this to capture a lot of data for a long period of time.

Code: Select all

tail -f /var/log/httpd/*error_log

Run the Availability report and let it run for an hour.
After that, get the output of the tail command and post it to the forum.

[biswajit.banerjee]

Our Nagios XI is Instance Type c5.4xlarge

As per your instructions we initiated the Report generation Process and captured the error Logs . I have attached for your review

[tgriep]

The nagios processes do access the disk drives a lot and will benefit from a faster disk subsystem to run and if you can move the server is an I, D or H instance, that will help the server to run efficiently and generate reports quicker.
How long does it take to create this 100 Meg file on your server?

Code: Select all

time dd if=/dev/zero of=100m.img bs=1 count=10000000 seek=100M

[biswajit.banerjee]

The Output is

Code: Select all

# time dd if=/dev/zero of=100m.img bs=1 count=10000000 seek=100M
10000000+0 records in
10000000+0 records out
10000000 bytes (10 MB) copied, 12.1726 s, 822 kB/s

real    0m12.175s
user    0m2.017s
sys     0m10.154s

[tgriep]

Run the following as root on the Nagios server and post the output.

Code: Select all

hostname
hostname -i
ip addr
cat /etc/hosts

Go to the Admin > System Settings in the XI interface and show the settings that are in the Program and External URL.
Are those settings resolvable in DNS on your Nagios server as well as the desktop you are using to access the Nagios server?
Do they resolve to the same IP address?

[biswajit.banerjee]

Please find output as below

# hostname
usvanagiospxi3

# hostname -i
10.144.1.211

# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
link/ether 12:d2:df:47:ba:24 brd ff:ff:ff:ff:ff:ff
inet 10.144.1.211/23 brd 10.144.1.255 scope global dynamic eth0
valid_lft 3286sec preferred_lft 3286sec
inet 192.168.200.121/24 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80:dfff:fe47:ba24/64 scope link
valid_lft forever preferred_lft forever

# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
#10.140.140.10 dca-dc1.bose.com dca-dc1
#10.140.44.10 dca-dc2.bose.com dca-dc2

Among Program and External URL , External URL is not defined .
Program URL is resolvable in DNS on Server and desktop
They Resolve on same IP , But IP is of AWS Load Balancer .

Hope it helps

[tgriep]

Edit the External URL to be like the Program URL but use the IP address of the load balance and not the host name.
Run the report and if it is still slow, use the actual IP address in the External URL and try again.

If you are running Firefox, press Ctrl-Shift-E to open the Network Debugger.
Run the report and look at the debugger and see what is taking the most time and if you can use that to determine the issue.