Nagiosxi Performance graphing not working

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
dinush
Posts: 19
Joined: Thu Jun 13, 2013 4:42 am

Nagiosxi Performance graphing not working

Post by dinush »

Nagiosxi performance graphs not available after disk usage reached 100%. Though we have cleaned the logs and rebooted the server, still the graphing not working. We did as below and still the same.

https://support.nagios.com/kb/article/n ... blems.html
User avatar
lmiltchev
Bugs find me
Posts: 13589
Joined: Mon May 23, 2011 12:15 pm

Re: Nagiosxi Performance graphing not working

Post by lmiltchev »

How much disk space do you have now? Please run the following command, and show the output:

Code: Select all

df -h
If you still need to clear some disk space, you could delete old logs in the "/var/log" directory and sub-directories. To find the 10 largest files in /var/log/ directory, run:

Code: Select all

du -a /var/log | sort -n -r | head -n 10
You can also delete whatever you don't need in the /tmp/" directory. Maybe you have some old backups that you can delete in "/store/backups/nagiosxi".

If you are running Nagios XI on a VM, you could add more disk space by following the document below:
http://assets.nagios.com/downloads/nagi ... M_Disk.pdf

Once you have the disk space under control, you could start troubleshooting the performance graphs by going through each step, outlined in the KB article below:
https://support.nagios.com/kb/article/n ... ems-9.html

If the KB article doesn't help you solve the issue, run the following commands and show the output in code wraps:

Code: Select all

uptime
service npcd restart
service npcd status
tail -100 /usr/local/nagios/var/perfdata.log
tail -100 /usr/local/nagios/var/npcd.log
ls /usr/local/nagios/var/spool/xidpe | wc -l
ls /usr/local/nagios/var/spool/perfdata | wc -l
ls /usr/local/nagios/var/spool/checkresults | wc -l
or if you are using ramdisk, run the command below (instead of the last three commands above):

Code: Select all

ls /var/nagiosramdisk/spool/xidpe | wc -l
ls /var/nagiosramdisk/spool/perfdata/ | wc -l
ls /var/nagiosramdisk/spool/checkresults/ | wc -l
The "uptime" command will show us what's the load on the server. We will see if ncpd runs, and look for clues in the npcd.log and perfdata.log. We will also make sure that you don't have bunch of perfdata files piled in in any of the three directories - xidpe, perfdata, and checkresults.
Be sure to check out our Knowledgebase for helpful articles and solutions!
dinush
Posts: 19
Joined: Thu Jun 13, 2013 4:42 am

Re: Nagiosxi Performance graphing not working

Post by dinush »

df -h output as below.

[root@nagiosxi ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
32G 25G 5.5G 83% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/sda1 477M 66M 386M 15% /boot
/dev/mapper/vol_store-lv_store
40G 31G 6.3G 84% /backup

and as per the article we issued below command and still no graphs.

find /usr/local/nagios/var/spool/perfdata/ -type f -delete


further we found below on logs.

# tail -100 /usr/local/nagios/var/perfdata.log
2017-11-21 04:27:21 [10781] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-11-21 04:27:21 [10781] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-11-21 04:27:21 [10781] [0] *** TIMEOUT: Please check your npcd.cfg
2017-11-21 04:27:21 [10781] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1511150671.perfdata.host-PID-10781 deleted
2017-11-21 04:27:21 [10781] [0] *** Timeout while processing Host: "stash-JIRA" Service: "_HOST_"
2017-11-21 04:27:21 [10781] [0] *** process_perfdata.pl terminated on signal ALRM
2017-11-21 04:27:21 [10778] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-11-21 04:27:21 [10778] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-11-21 04:27:21 [10778] [0] *** TIMEOUT: Please check your npcd.cfg
2017-11-21 04:27:21 [10778] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1511150641.perfdata.service-PID-10778 deleted
2017-11-21 04:27:21 [10778] [0] *** Timeout while processing Host: "172.25.70.184-SURV" Service: "Ping"
2017-11-21 04:27:21 [10778] [0] *** process_perfdata.pl terminated on signal ALRM


[root@nagiosxi ~]# tail -100 /usr/local/nagios/var/npcd.log
[11-21-2017 10:20:22] NPCD: WARN: MAX load reached: load 31.970000/10.000000 at i=117
[11-21-2017 10:20:37] NPCD: WARN: MAX load reached: load 25.650000/10.000000 at i=117
[11-21-2017 10:20:52] NPCD: WARN: MAX load reached: load 20.400000/10.000000 at i=117
[11-21-2017 10:21:07] NPCD: WARN: MAX load reached: load 17.130000/10.000000 at i=117
[11-21-2017 10:21:22] NPCD: WARN: MAX load reached: load 53.940000/10.000000 at i=117
dinush
Posts: 19
Joined: Thu Jun 13, 2013 4:42 am

Re: Nagiosxi Performance graphing not working

Post by dinush »

i have changed npcd.cfg file and set the
load_threshold = 70

but still gives the load error. Why is the load is high and how to reduce it.

[root@nagiosxi ~]# tailf /usr/local/nagios/var/npcd.log
[11-21-2017 11:18:35] NPCD: npcd Daemon (0.4.14) started with PID=1811
[11-21-2017 11:18:35] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 11:18:35] NPCD: HINT: load_threshold is enabled - ('50.000000')
[11-21-2017 11:19:53] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 11:19:54] NPCD: npcd Daemon (0.4.14) started with PID=6398
[11-21-2017 11:19:54] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 11:19:54] NPCD: HINT: load_threshold is enabled - ('70.000000')
[11-21-2017 11:19:54] NPCD: WARN: MAX load reached: load 85.500000/70.000000 at i=0
[11-21-2017 11:20:09] NPCD: WARN: MAX load reached: load 75.060000/70.000000 at i=1
[11-21-2017 11:20:24] NPCD: Error while getting file status
npolovenko
Support Tech
Posts: 3457
Joined: Mon May 15, 2017 5:00 pm

Re: Nagiosxi Performance graphing not working

Post by npolovenko »

@dinush, Based on the log file readings, it seems like the NPCD load just keeps on growing. We need to find out what's causing it to grow before moving on to the graphing issue. Can we see what processes are running on your system? Please run ps -ef. How many hosts and services are you monitoring at the moment? There might be a possibility that one of the tables in Nagios DB has crashed.
If you could upload the whole system profile that would be very helpful. To send us your system profile:
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and attach it to your next post/or upload it to a cloud service of your choice and share a link with me in pm.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
dinush
Posts: 19
Joined: Thu Jun 13, 2013 4:42 am

Re: Nagiosxi Performance graphing not working

Post by dinush »

attached profile.zip file.


we have about 240 hosts and 1862 services . As you said may be a table crash. How to find it out?

below is the ps -ef output

Code: Select all

[root@nagiosxi ~]# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 Nov21 ?        00:00:03 /sbin/init
root         2     0  0 Nov21 ?        00:00:00 [kthreadd]
root         3     2  0 Nov21 ?        00:00:03 [migration/0]
root         4     2  0 Nov21 ?        00:00:00 [ksoftirqd/0]
root         5     2  0 Nov21 ?        00:00:00 [stopper/0]
root         6     2  0 Nov21 ?        00:00:00 [watchdog/0]
root         7     2  0 Nov21 ?        00:00:02 [migration/1]
root         8     2  0 Nov21 ?        00:00:00 [stopper/1]
root         9     2  0 Nov21 ?        00:00:00 [ksoftirqd/1]
root        10     2  0 Nov21 ?        00:00:00 [watchdog/1]
root        11     2  0 Nov21 ?        00:00:28 [events/0]
root        12     2  0 Nov21 ?        00:00:09 [events/1]
root        13     2  0 Nov21 ?        00:00:00 [cgroup]
root        14     2  0 Nov21 ?        00:00:00 [khelper]
root        15     2  0 Nov21 ?        00:00:00 [netns]
root        16     2  0 Nov21 ?        00:00:00 [async/mgr]
root        17     2  0 Nov21 ?        00:00:00 [pm]
root        18     2  0 Nov21 ?        00:00:00 [sync_supers]
root        19     2  0 Nov21 ?        00:00:00 [bdi-default]
root        20     2  0 Nov21 ?        00:00:00 [kintegrityd/0]
root        21     2  0 Nov21 ?        00:00:00 [kintegrityd/1]
root        22     2  0 Nov21 ?        00:00:27 [kblockd/0]
root        23     2  0 Nov21 ?        00:00:31 [kblockd/1]
root        24     2  0 Nov21 ?        00:00:00 [kacpid]
root        25     2  0 Nov21 ?        00:00:00 [kacpi_notify]
root        26     2  0 Nov21 ?        00:00:00 [kacpi_hotplug]
root        27     2  0 Nov21 ?        00:00:00 [ata_aux]
root        28     2  0 Nov21 ?        00:00:00 [ata_sff/0]
root        29     2  0 Nov21 ?        00:00:00 [ata_sff/1]
root        30     2  0 Nov21 ?        00:00:00 [ksuspend_usbd]
root        31     2  0 Nov21 ?        00:00:00 [khubd]
root        32     2  0 Nov21 ?        00:00:00 [kseriod]
root        33     2  0 Nov21 ?        00:00:00 [md/0]
root        34     2  0 Nov21 ?        00:00:00 [md/1]
root        35     2  0 Nov21 ?        00:00:00 [md_misc/0]
root        36     2  0 Nov21 ?        00:00:00 [md_misc/1]
root        37     2  0 Nov21 ?        00:00:00 [linkwatch]
root        39     2  0 Nov21 ?        00:00:00 [khungtaskd]
root        40     2  0 Nov21 ?        00:00:14 [kswapd0]
root        41     2  0 Nov21 ?        00:00:00 [ksmd]
root        42     2  0 Nov21 ?        00:00:25 [khugepaged]
root        43     2  0 Nov21 ?        00:00:00 [aio/0]
root        44     2  0 Nov21 ?        00:00:00 [aio/1]
root        45     2  0 Nov21 ?        00:00:00 [crypto/0]
root        46     2  0 Nov21 ?        00:00:00 [crypto/1]
root        54     2  0 Nov21 ?        00:00:00 [kthrotld/0]
root        55     2  0 Nov21 ?        00:00:00 [kthrotld/1]
root        56     2  0 Nov21 ?        00:00:00 [pciehpd]
root        58     2  0 Nov21 ?        00:00:00 [kpsmoused]
root        59     2  0 Nov21 ?        00:00:00 [usbhid_resumer]
root        60     2  0 Nov21 ?        00:00:00 [deferwq]
root        91     2  0 Nov21 ?        00:00:00 [kdmremove]
root        92     2  0 Nov21 ?        00:00:00 [kstriped]
root       182     2  0 Nov21 ?        00:00:00 [scsi_eh_0]
root       187     2  0 Nov21 ?        00:00:00 [scsi_eh_1]
root       198     2  0 Nov21 ?        00:00:01 [mpt_poll_0]
root       199     2  0 Nov21 ?        00:00:00 [mpt/0]
root       200     2  0 Nov21 ?        00:00:00 [scsi_eh_2]
root       429     2  0 Nov21 ?        00:00:00 [kdmflush]
root       431     2  0 Nov21 ?        00:00:00 [kdmflush]
root       448     2  0 Nov21 ?        00:00:44 [jbd2/dm-0-8]
root       449     2  0 Nov21 ?        00:00:00 [ext4-dio-unwrit]
root       522     1  0 Nov21 ?        00:00:00 /sbin/udevd -d
apache     727  1790  0 Nov21 ?        00:08:53 /usr/sbin/httpd
root       785     2  0 Nov21 ?        00:00:00 [vmmemctl]
root       906   522  0 Nov21 ?        00:00:00 /sbin/udevd -d
root       909     2  0 Nov21 ?        00:00:00 [kdmflush]
root       910   522  0 Nov21 ?        00:00:00 /sbin/udevd -d
root       944     2  0 Nov21 ?        00:00:00 [jbd2/sda1-8]
root       945     2  0 Nov21 ?        00:00:00 [ext4-dio-unwrit]
root       946     2  0 Nov21 ?        00:00:00 [jbd2/dm-2-8]
root       947     2  0 Nov21 ?        00:00:00 [ext4-dio-unwrit]
root       967     2  0 Nov21 ?        00:02:28 [flush-253:0]
root       984     2  0 Nov21 ?        00:00:00 [kauditd]
apache    1144  1790  1 01:07 ?        00:04:48 /usr/sbin/httpd
postgres  1230  1687  0 Nov21 ?        00:00:48 postgres: nagiosxi nagiosxi 127.0.0.1(58400) idle
postgres  1265  1687  0 07:02 ?        00:00:05 postgres: nagiosxi nagiosxi 127.0.0.1(56619) idle
root      1339     1  0 Nov21 ?        00:00:47 /usr/sbin/vmtoolsd
root      1417     1  0 Nov21 ?        00:00:02 auditd
root      1437     1  0 Nov21 ?        00:00:43 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5
dbus      1450     1  0 Nov21 ?        00:00:00 dbus-daemon --system
root      1492     1  0 Nov21 ?        00:00:00 /usr/sbin/sshd
root      1501     1  0 Nov21 ?        00:00:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
ntp       1510     1  0 Nov21 ?        00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
root      1545     1  0 Nov21 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld.pid --basedir=/usr --use
postgres  1574  1687  0 01:07 ?        00:00:25 postgres: nagiosxi nagiosxi 127.0.0.1(43788) idle
mysql     1647  1545 10 Nov21 ?        02:18:10 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --so
postgres  1687     1  0 Nov21 ?        00:00:16 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data
postgres  1734  1687  0 Nov21 ?        00:00:02 postgres: logger process
postgres  1736  1687  0 Nov21 ?        00:00:15 postgres: writer process
postgres  1737  1687  0 Nov21 ?        00:00:08 postgres: wal writer process
postgres  1738  1687  0 Nov21 ?        00:00:05 postgres: autovacuum launcher process
postgres  1739  1687  0 Nov21 ?        00:00:13 postgres: stats collector process
root      1775     1  0 Nov21 ?        00:00:06 /usr/libexec/postfix/master
postfix   1785  1775  0 Nov21 ?        00:00:11 qmgr -l -t fifo -u
root      1790     1  0 Nov21 ?        00:00:03 /usr/sbin/httpd
root      1800     1  0 Nov21 ?        00:00:02 crond
nagios    1823     1  0 Nov21 ?        00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
ajaxterm  1843     1  0 Nov21 ?        00:00:20 python /usr/share/ajaxterm/ajaxterm.py --daemon --port=8022 --uid=ajaxterm
root      1949     1  0 Nov21 tty1     00:00:00 /sbin/mingetty /dev/tty1
root      1951     1  0 Nov21 tty2     00:00:00 /sbin/mingetty /dev/tty2
root      1953     1  0 Nov21 tty3     00:00:00 /sbin/mingetty /dev/tty3
root      1955     1  0 Nov21 tty4     00:00:00 /sbin/mingetty /dev/tty4
root      1957     1  0 Nov21 tty5     00:00:00 /sbin/mingetty /dev/tty5
root      1959     1  0 Nov21 tty6     00:00:00 /sbin/mingetty /dev/tty6
postfix   2140  1775  0 08:49 ?        00:00:00 pickup -l -t fifo -u
apache    3280  1790  0 03:38 ?        00:03:07 /usr/sbin/httpd
apache    3281  1790  0 03:38 ?        00:03:02 /usr/sbin/httpd
apache    3282  1790  0 03:38 ?        00:02:56 /usr/sbin/httpd
postgres  4033  1687  0 03:41 ?        00:00:16 postgres: nagiosxi nagiosxi 127.0.0.1(56851) idle
postgres  4034  1687  0 03:41 ?        00:00:18 postgres: nagiosxi nagiosxi 127.0.0.1(56871) idle
postgres  4039  1687  0 03:41 ?        00:00:16 postgres: nagiosxi nagiosxi 127.0.0.1(56875) idle
apache    4378  1790  1 03:41 ?        00:03:14 /usr/sbin/httpd
apache    4379  1790  0 03:41 ?        00:03:02 /usr/sbin/httpd
postgres  5033  1687  0 03:41 ?        00:00:17 postgres: nagiosxi nagiosxi 127.0.0.1(57221) idle
postgres  5034  1687  0 03:41 ?        00:00:19 postgres: nagiosxi nagiosxi 127.0.0.1(57222) idle
apache    5909  1790  1 Nov21 ?        00:09:13 /usr/sbin/httpd
postgres  6083  1687  0 Nov21 ?        00:00:50 postgres: nagiosxi nagiosxi 127.0.0.1(37451) idle
nagios    6398     1  0 Nov21 ?        00:00:10 /usr/local/nagios/bin/npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg
apache    7956  1790  0 Nov21 ?        00:06:38 /usr/sbin/httpd
postgres  8847  1687  0 Nov21 ?        00:00:34 postgres: nagiosxi nagiosxi 127.0.0.1(50344) idle
postfix  11166  1775  0 08:53 ?        00:00:00 cleanup -z -t unix -u
root     12174  1800  0 08:54 ?        00:00:00 CROND
root     12175  1800  0 08:54 ?        00:00:00 CROND
root     12176  1800  0 08:54 ?        00:00:00 CROND
root     12177  1800  0 08:54 ?        00:00:00 CROND
root     12178  1800  0 08:54 ?        00:00:00 CROND
root     12179  1800  0 08:54 ?        00:00:00 CROND
nagios   12180 12175  0 08:54 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
nagios   12183 12177  0 08:54 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
nagios   12185 12178  0 08:54 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
nagios   12186 12179  0 08:54 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
nagios   12187 12176  0 08:54 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php > /usr/local/nagiosxi/var/event_handler.log 2>&1
nagios   12188 12174  0 08:54 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
nagios   12190 12180  0 08:54 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php
nagios   12193 12183  0 08:54 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
nagios   12194 12187  0 08:54 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php
nagios   12195 12185  0 08:54 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php
nagios   12196 12186  0 08:54 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php
nagios   12197 12188  0 08:54 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
postgres 12231  1687  0 08:54 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(38167) idle
postgres 12232  1687  0 08:54 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(38168) idle
postgres 12235  1687  0 08:54 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(38171) idle
postgres 12244  1687  0 08:54 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(38173) idle
postgres 12249  1687  0 08:54 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(38174) idle
postgres 12276  1687  0 08:54 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(38189) idle
nagios   12326     1  1 03:00 ?        00:04:36 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   12328 12326  0 03:00 ?        00:00:53 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   12329 12326  0 03:00 ?        00:00:54 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   12330 12326  0 03:00 ?        00:00:55 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   12331 12326  0 03:00 ?        00:00:54 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   12335  1823  0 03:00 ?        00:00:34 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   12338 12335  8 03:00 ?        00:30:35 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   12344 12326  0 03:00 ?        00:00:01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   13002 12330  0 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_netsnmp_memory.pl -L Memory Utilization -A * -H 172.25.80.168 -C public -a -C public -a %user_real
nagios   13071 12330  0 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_netsnmp_memory.pl -L Memory Utilization -A * -H 172.25.76.108 -C public -a -C public -a %user_real
nagios   13117 12330  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_icmp -H 172.25.83.212 -w 3000.0 80  -c 5000.0 100  -p 5
nagios   13190 12331  0 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.92.22 -C public --v2c -m ^/$ -w 80 -c 95 -f$
nagios   13213 12328  1 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.38.21 -C public --v2c -m ^/x06$ -w 80 -c 90 -f$
nagios   13232 12329  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_icmp -H 172.25.76.108 -w 3000.0 80  -c 5000.0 100  -p 5
nagios   13277 12331  1 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_netsnmp_memory.pl -L Memory Utilization -A * -H 172.25.70.26 -C public -a -C public -a %user_real
nagios   13283 12330  1 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.38.21 -C public --v2c -m ^/x03$ -w 80 -c 90 -f$
nagios   13312 12329  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.70.102 -t 30 -c check_diskio -a -I -w 400,300 -c 600,500 -p sda1,sda2,sda3,sda4,sda5,sda6,sdb1,sdc1,
nagios   13313 12330  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.70.102 -t 30 -c check_swap -a -w 50 -c 20
nagios   13327 12331  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.70.209 -t 30 -c check_cpu_stats -a -w 85 -c 95
nagios   13332 12329  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.129.7 -t 30 -c check_cpu_stats -a -w 85 -c 95
nagios   13336 12329  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.70.123 -t 30 -c check_cpu_stats -a -w 85 -c 95
nagios   13347 12331  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.129.5 -t 30 -c check_diskio -a -I -w 100,70 -c 150,100 -p sda1,sda2,sda3,sda4,sda5,sda6,sdb1,sdc1,sd
nagios   13353 12330  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.88.5 -t 30 -c check_nwio -a -N -w 30000000 -c 100000000 -p eth0,eth1,eth2,eth3
nagios   13358  6398  8 08:54 ?        00:00:00 /usr/bin/perl /usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1511321052.perfdata.service
nagios   13367 12328  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.70.209 -t 30 -c check_diskio -a -I -w 400,300 -c 600,500 -p sda1,sda2,sda3,sda4,sda5,sda6,sdb1,sdc1,
nagios   13369 12329  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.88.5 -t 30 -c check_cpu_stats -a -w 85 -c 95
nagios   13370 12331  3 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.76.218 -C public --v2c -m Swap -w 80 -c 90 -f
nagios   13371 12329  3 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.38.12 -C public --v2c -m Swap -w 80 -c 90 -f
nagios   13376 12330  2 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.33.29 -C public --v2c -m ^/x02$ -w 80 -c 90 -f$
nagios   13377 12330  2 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_netsnmp_memory.pl -L Memory Utilization -A * -H 172.25.71.49 -C public -a -C public -a %user_real
nagios   13378 12329  2 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.31.10 -C public --v2c -m ^/x02$ -w 80 -c 90 -f$
nagios   13380 12328  3 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.31.19 -C public --v2c -m Physical -w 80 -c 90 -f
nagios   13387 12330  2 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_load_wizard.pl -H 172.25.31.11 -C public --v2c -w 80 -c 90 -f
nagios   13389 12331  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.70.108 -t 30 -c check_tcp_stat -a -S ESTABLISHED -w 1500 -c 2000
nagios   13394 12329  2 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.40.68 -C public --v2c -m Physical -w 80 -c 90 -f
nagios   13395 12330  2 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H 172.25.83.191 -C public --v2c -m ^/$ -w 80 -c 95 -f$
nagios   13396 12328  2 08:54 ?        00:00:00 /usr/bin/perl -w /usr/local/nagios/libexec/check_snmp_load_wizard.pl -H 172.25.83.151 -C public --v2c -w 80 -c 90 -f
nagios   13398 12329  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.38.80 -t 30 -c check_tcp_stat -a -S ESTABLISHED -w 1500 -c 2000
nagios   13399 12330  0 08:54 ?        00:00:00 /usr/local/nagios/libexec/check_nrpe -H 172.25.80.239 -t 30 -c check_nwio -a -N -w 30000000 -c 100000000 -p eth0,eth1,eth2,eth3
root     13400 31335  0 08:54 pts/0    00:00:00 ps -ef
apache   16366  1790  0 03:28 ?        00:03:14 /usr/sbin/httpd
apache   16368  1790  0 03:28 ?        00:03:07 /usr/sbin/httpd
postgres 16882  1687  0 03:31 ?        00:00:18 postgres: nagiosxi nagiosxi 127.0.0.1(50026) idle
postgres 16918  1687  0 03:31 ?        00:00:17 postgres: nagiosxi nagiosxi 127.0.0.1(50055) idle
apache   17054  1790  0 03:31 ?        00:02:58 /usr/sbin/httpd
apache   17154  1790  0 03:31 ?        00:03:00 /usr/sbin/httpd
postgres 18109  1687  0 03:31 ?        00:00:16 postgres: nagiosxi nagiosxi 127.0.0.1(50577) idle
postgres 18174  1687  0 03:31 ?        00:00:16 postgres: nagiosxi nagiosxi 127.0.0.1(50594) idle
apache   20445  1790  0 Nov21 ?        00:08:42 /usr/sbin/httpd
postgres 20591  1687  0 Nov21 ?        00:00:46 postgres: nagiosxi nagiosxi 127.0.0.1(36979) idle
apache   21902  1790  0 Nov21 ?        00:08:42 /usr/sbin/httpd
postfix  21917  1775  0 08:30 ?        00:00:00 smtp -t unix -u
postgres 22154  1687  0 Nov21 ?        00:00:46 postgres: nagiosxi nagiosxi 127.0.0.1(54553) idle
apache   25297  1790  0 03:50 ?        00:02:59 /usr/sbin/httpd
postfix  25336  1775  0 08:45 ?        00:00:00 smtp -t unix -u
postfix  25338  1775  0 08:45 ?        00:00:00 scache -l -t unix -u
postfix  25341  1775  0 08:45 ?        00:00:00 smtp -t unix -u
postgres 25891  1687  0 03:50 ?        00:00:16 postgres: nagiosxi nagiosxi 127.0.0.1(36140) idle
postfix  29149  1775  0 08:33 ?        00:00:00 smtp -t unix -u
postfix  29844  1775  0 08:47 ?        00:00:00 smtp -t unix -u
postfix  29851  1775  0 08:47 ?        00:00:00 smtp -t unix -u
postfix  29891  1775  0 08:47 ?        00:00:00 smtp -t unix -u
postfix  29913  1775  0 08:47 ?        00:00:00 smtp -t unix -u
apache   30173  1790  0 07:00 ?        00:00:58 /usr/sbin/httpd
apache   30432  1790  0 07:00 ?        00:01:04 /usr/sbin/httpd
apache   30433  1790  1 Nov21 ?        00:07:09 /usr/sbin/httpd
postgres 30766  1687  0 Nov21 ?        00:00:39 postgres: nagiosxi nagiosxi 127.0.0.1(46669) idle
root     31199  1492  0 08:48 ?        00:00:00 sshd: root@pts/0
postgres 31211  1687  0 07:00 ?        00:00:04 postgres: nagiosxi nagiosxi 127.0.0.1(55735) idle
apache   31231  1790  0 07:01 ?        00:01:05 /usr/sbin/httpd
root     31335 31199  0 08:48 pts/0    00:00:00 -bash
postgres 31517  1687  0 07:01 ?        00:00:05 postgres: nagiosxi nagiosxi 127.0.0.1(55834) idle
[root@nagiosxi ~]#
You do not have the required permissions to view the files attached to this post.
User avatar
lmiltchev
Bugs find me
Posts: 13589
Joined: Mon May 23, 2011 12:15 pm

Re: Nagiosxi Performance graphing not working

Post by lmiltchev »

The performance grapher won't be running if the load threshold is exceeded. Your load has exceeded the specified value of 70 numerous times, i.e.

Code: Select all

[11-21-2017 21:16:40] NPCD: WARN: MAX load reached: load 220.390000/70.000000 at i=0
[11-21-2017 21:16:55] NPCD: WARN: MAX load reached: load 206.920000/70.000000 at i=1
[11-21-2017 21:17:10] NPCD: WARN: MAX load reached: load 173.850000/70.000000 at i=1
[11-21-2017 21:17:25] NPCD: WARN: MAX load reached: load 137.040000/70.000000 at i=1
Are you still having a space issue? I can see the following entry in the mysql log:
171122 1:13:46 [Warning] Disk is full writing '/var/lib/mysql/nagios/nagios_logentries.MYI' (Errcode: 28). Waiting for someone to free space... (Expect up to 60 secs delay for server to continue after freeing disk space)
Make sure the space issue is resolved first, then restart all services:

Code: Select all

service mysqld restart
service nagios stop
service ndo2db restart
service nagios start
service postgresql restart
service npcd restart
Double check the mysqld.log for errors/crashed tables:

Code: Select all

tail /var/log/mysqld.log
If you see any crashed tables, run our DB repair script:

Code: Select all

/usr/local/nagiosxi/scripts/repair_databases.sh
Next, run:

Code: Select all

top
and watch to load for a while to see if it is going to exceed 70. If the load is kept below 70, the npcd would be running, processing the perf data, and graphs should start reappearing.
Be sure to check out our Knowledgebase for helpful articles and solutions!
Locked