Page 1 of 3
Service Attributes - Performance Data - RED
Posted: Tue Jan 19, 2016 12:28 pm
by brandon.pal
Hi,
A lot of my checks are no longer showing Perf Data.
Blank_diskSpace.png
If I look at Advanced --> Service Attributes --> Performance Data it's red.
ServiceAttributes.png
In Admin --> System Status all is green.
Code: Select all
tail -f /usr/local/nagiosxi/var/perfdataproc.log
tail: /usr/local/nagiosxi/var/perfdataproc.log: file truncated
Outbound data DISABLED Tue, 19 Jan 2016 12:18:01 -0500
mv: cannot stat `/usr/local/nagios/var/spool/xidpe/*': No such file or directory
mv: cannot stat `/usr/local/nagios/var/spool/xidpe/*': No such file or directory
tail: /usr/local/nagiosxi/var/perfdataproc.log: file truncated
Outbound data DISABLED Tue, 19 Jan 2016 12:19:01 -0500
DONE. Processed 0 files.
tail: /usr/local/nagiosxi/var/perfdataproc.log: file truncated
DONE. Processed 0 files.
tail: /usr/local/nagiosxi/var/perfdataproc.log: file truncated
Outbound data DISABLED Tue, 19 Jan 2016 12:21:01 -0500
DONE. Processed 0 files.
tail: /usr/local/nagiosxi/var/perfdataproc.log: file truncated
I have restarted cron as well as npcd.
Please help.
Re: Service Attributes - Performance Data - RED
Posted: Tue Jan 19, 2016 2:43 pm
by rkennedy
I wonder if the user expired, or if it's permissions related.
What is the result of these following commands? -
Code: Select all
chage -l nagios
ls -l /usr/local/nagiosxi/var/
ls -l /usr/local/nagios/var/spool/
Re: Service Attributes - Performance Data - RED
Posted: Tue Jan 19, 2016 3:47 pm
by brandon.pal
chage -l nagios
Code: Select all
Last password change : Feb 12, 2014
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7
Code: Select all
total 344
-rw-r--r-- 1 nagios nagios 303 Jan 19 15:45 cleaner.log
-rw-r--r-- 1 nagios nagios 303 Dec 27 03:36 cleaner.log-20151227
-rw-r--r-- 1 nagios nagios 303 Jan 3 03:36 cleaner.log-20160103
-rw-r--r-- 1 nagios nagios 303 Jan 10 03:48 cleaner.log-20160110
-rw-r--r-- 1 nagios nagios 303 Jan 17 03:10 cleaner.log-20160117
-rw-r--r-- 1 nagios nagios 82 Jan 19 15:45 cmdsubsys.log
-rw-r--r-- 1 nagios nagios 82 Dec 27 03:37 cmdsubsys.log-20151227
-rw-r--r-- 1 nagios nagios 82 Jan 3 03:37 cmdsubsys.log-20160103
-rw-r--r-- 1 nagios nagios 82 Jan 10 03:49 cmdsubsys.log-20160110
-rw-r--r-- 1 nagios nagios 82 Jan 17 03:11 cmdsubsys.log-20160117
drwsrwsr-x 4 apache nagios 4096 May 27 2015 components
-rw-r--r-- 1 nagios nagios 7 Jul 15 2014 corelog.data
-rw-r--r-- 1 nagios nagios 495 Jul 15 2014 corelog.diff
-rwxrwxr-x 1 nagios nagios 2037 Jun 20 2014 corelog.newobjects
-rw-r--r-- 1 nagios nagios 2681 Jan 19 15:45 dbmaint.log
-rw-r--r-- 1 nagios nagios 2681 Dec 27 03:35 dbmaint.log-20151227
-rw-r--r-- 1 nagios nagios 2681 Jan 3 03:35 dbmaint.log-20160103
-rw-r--r-- 1 nagios nagios 2681 Jan 10 03:45 dbmaint.log-20160110
-rw-r--r-- 1 nagios nagios 2681 Jan 17 03:10 dbmaint.log-20160117
-rw-r--r-- 1 nagios nagios 75 Jan 19 15:45 deadpool.log
-rw-r--r-- 1 nagios nagios 75 Dec 27 03:35 deadpool.log-20151227
-rw-r--r-- 1 nagios nagios 75 Jan 3 03:35 deadpool.log-20160103
-rw-r--r-- 1 nagios nagios 75 Jan 10 03:45 deadpool.log-20160110
-rw-r--r-- 1 nagios nagios 75 Jan 18 03:30 deadpool.log-20160118
-rw-r--r-- 1 nagios nagios 3348 Jan 19 15:45 eventman.log
-rw-r--r-- 1 nagios nagios 40 Dec 27 03:37 eventman.log-20151227
-rw-r--r-- 1 nagios nagios 40 Jan 3 03:37 eventman.log-20160103
-rw-r--r-- 1 nagios nagios 40 Jan 10 03:49 eventman.log-20160110
-rw-r--r-- 1 nagios nagios 40 Jan 17 03:11 eventman.log-20160117
-rw-r--r-- 1 nagios nagios 18 Jan 19 15:45 feedproc.log
-rw-r--r-- 1 nagios nagios 18 Dec 27 03:36 feedproc.log-20151227
-rw-r--r-- 1 nagios nagios 18 Jan 3 03:36 feedproc.log-20160103
-rw-r--r-- 1 nagios nagios 18 Jan 10 03:48 feedproc.log-20160110
-rw-r--r-- 1 nagios nagios 18 Jan 17 03:10 feedproc.log-20160117
-rw-r--r-- 1 nagios nagios 904 Jan 19 03:43 load_url.log
-rw-r--r-- 1 nagios nagios 905 Jan 2 03:49 load_url.log-20160102
-rw-r--r-- 1 nagios nagios 905 Jan 4 03:23 load_url.log-20160104
-rw-r--r-- 1 nagios nagios 905 Jan 10 03:48 load_url.log-20160110
-rw-r--r-- 1 nagios nagios 904 Jan 11 03:40 load_url.log-20160117
-rw-r--r-- 1 nagios nagios 0 Jan 19 15:45 nom.log
-rw-r--r-- 1 nagios nagios 745 Jun 15 2015 nom.log-20150615
-rw-r--r-- 1 nagios nagios 243 Jan 19 15:45 perfdataproc.log
-rw-r--r-- 1 nagios nagios 243 Dec 27 03:37 perfdataproc.log-20151227
-rw-r--r-- 1 nagios nagios 243 Jan 3 03:37 perfdataproc.log-20160103
-rw-r--r-- 1 nagios nagios 243 Jan 10 03:49 perfdataproc.log-20160110
-rw-r--r-- 1 nagios nagios 243 Jan 17 03:11 perfdataproc.log-20160117
-rw-r--r-- 1 nagios nagios 16894 Jan 19 15:01 recurringdowntime.log
-rw-r--r-- 1 nagios nagios 16894 Dec 27 03:01 recurringdowntime.log-20151227
-rw-r--r-- 1 nagios nagios 16892 Jan 3 03:01 recurringdowntime.log-20160103
-rw-r--r-- 1 nagios nagios 16893 Jan 10 03:01 recurringdowntime.log-20160110
-rw-r--r-- 1 nagios nagios 16893 Jan 17 03:01 recurringdowntime.log-20160117
-rw-r--r-- 1 nagios nagios 0 Jan 19 15:45 reportengine.log
-rw-r--r-- 1 nagios nagios 745 Jun 15 2015 reportengine.log-20150615
drwxr-xr-x 2 nagios nagios 4096 Jan 19 12:22 subsys
-rw-r--r-- 1 nagios nagios 8169 Jan 19 15:45 sysstat.log
-rw-r--r-- 1 nagios nagios 8180 Dec 27 03:37 sysstat.log-20151227
-rw-r--r-- 1 nagios nagios 8165 Jan 3 03:37 sysstat.log-20160103
-rw-r--r-- 1 nagios nagios 8167 Jan 10 03:49 sysstat.log-20160110
-rw-r--r-- 1 nagios nagios 8149 Jan 17 03:11 sysstat.log-20160117
drwxr-xr-x 2 apache nagios 4096 May 19 2015 upgrades
-rw-r--r-- 1 nagios nagios 6326 May 19 2015 xi-sys.cfg
-rw-r--r-- 1 nagios nagios 202 May 19 2015 xiversion
Code: Select all
total 28
drwxrwsr-x 2 nagios nagcmd 4096 Jan 19 13:22 checkresults
drwxr-xr-x 2 nagios nagios 12288 Jan 19 15:45 perfdata
drwxr-xr-x 2 nagios nagios 12288 Jan 19 15:45 xidpe
Re: Service Attributes - Performance Data - RED
Posted: Tue Jan 19, 2016 3:52 pm
by hsmith
How are you doing on disk space?
Is there anything in the cron logs?
Re: Service Attributes - Performance Data - RED
Posted: Tue Jan 19, 2016 3:57 pm
by brandon.pal
Code: Select all
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
7.5G 5.7G 1.5G 81% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/sda1 485M 50M 410M 11% /boot
/dev/sdb1 60G 9.8G 47G 18% /data
***-***-05.***-**.com:/vfmlfs/md03-vd05/nagiosBackup
1006G 154G 801G 17% /backup
Code: Select all
df -ih
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/VolGroup-lv_root
484K 97K 387K 20% /
tmpfs 984K 1 984K 1% /dev/shm
/dev/sda1 126K 44 125K 1% /boot
/dev/sdb1 3.8M 1.3M 2.5M 35% /data
***-***-05.***-***.com:/vfmlfs/md03-vd05/nagiosBackup
64M 1.5M 63M 3% /backup
Code: Select all
tail -n75 /var/log/cron
Jan 19 15:50:01 NAGIOS CROND[5824]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php > /usr/local/nagiosxi/var/dbmaint.log 2>&1)
Jan 19 15:50:01 NAGIOS CROND[5831]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jan 19 15:50:01 NAGIOS CROND[5827]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1)
Jan 19 15:50:01 NAGIOS CROND[5830]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jan 19 15:50:01 NAGIOS CROND[5829]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jan 19 15:50:01 NAGIOS CROND[5823]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php > /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jan 19 15:50:01 NAGIOS CROND[5833]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jan 19 15:50:01 NAGIOS CROND[5832]: (splunker) CMD (/home/splunker/powerData/bin/Splunk-Servertech-PDU-Load.sh 10.91.12.245 PDU-B-Rack108)
Jan 19 15:50:01 NAGIOS CROND[5828]: (nagios) CMD (/usr/local/bin/pagerduty_nagios.pl flush)
Jan 19 15:50:01 NAGIOS CROND[5841]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1)
Jan 19 15:50:01 NAGIOS CROND[5835]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jan 19 15:50:01 NAGIOS CROND[5852]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jan 19 15:51:01 NAGIOS CROND[7803]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1)
Jan 19 15:51:01 NAGIOS CROND[7804]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jan 19 15:51:01 NAGIOS CROND[7806]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jan 19 15:51:01 NAGIOS CROND[7807]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jan 19 15:51:01 NAGIOS CROND[7811]: (splunker) CMD (/home/splunker/powerData/bin/ps.sh 3)
Jan 19 15:51:01 NAGIOS CROND[7805]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jan 19 15:51:01 NAGIOS CROND[7812]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jan 19 15:51:01 NAGIOS CROND[7825]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1)
Jan 19 15:51:01 NAGIOS CROND[7815]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jan 19 15:51:01 NAGIOS CROND[7828]: (nagios) CMD (/usr/local/bin/pagerduty_nagios.pl flush)
Jan 19 15:52:01 NAGIOS CROND[11572]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jan 19 15:52:01 NAGIOS CROND[11574]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1)
Jan 19 15:52:01 NAGIOS CROND[11573]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jan 19 15:52:01 NAGIOS CROND[11575]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jan 19 15:52:01 NAGIOS CROND[11576]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1)
Jan 19 15:52:01 NAGIOS CROND[11581]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jan 19 15:52:01 NAGIOS CROND[11583]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jan 19 15:52:01 NAGIOS CROND[11585]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jan 19 15:52:01 NAGIOS CROND[11587]: (nagios) CMD (/usr/local/bin/pagerduty_nagios.pl flush)
Jan 19 15:52:01 NAGIOS CROND[11590]: (splunker) CMD (/home/splunker/powerData/bin/ps.sh 1)
Jan 19 15:53:01 NAGIOS CROND[13251]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1)
Jan 19 15:53:01 NAGIOS CROND[13253]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jan 19 15:53:01 NAGIOS CROND[13252]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jan 19 15:53:01 NAGIOS CROND[13256]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jan 19 15:53:01 NAGIOS CROND[13258]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1)
Jan 19 15:53:01 NAGIOS CROND[13263]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jan 19 15:53:01 NAGIOS CROND[13250]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jan 19 15:53:01 NAGIOS CROND[13261]: (nagios) CMD (/usr/local/bin/pagerduty_nagios.pl flush)
Jan 19 15:53:01 NAGIOS CROND[13266]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jan 19 15:54:01 NAGIOS CROND[15677]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jan 19 15:54:01 NAGIOS CROND[15678]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jan 19 15:54:01 NAGIOS CROND[15676]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jan 19 15:54:01 NAGIOS CROND[15679]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jan 19 15:54:01 NAGIOS CROND[15683]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jan 19 15:54:01 NAGIOS CROND[15685]: (splunker) CMD (/home/splunker/bandwidthData/bin/bandwidthRRDConvert.sh 1)
Jan 19 15:54:01 NAGIOS CROND[15686]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1)
Jan 19 15:54:01 NAGIOS CROND[15680]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jan 19 15:54:01 NAGIOS CROND[15688]: (nagios) CMD (/usr/local/bin/pagerduty_nagios.pl flush)
Jan 19 15:54:01 NAGIOS CROND[15690]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18026]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php > /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18025]: (root) CMD (LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lock/mrtg/mrtg_l --confcache-file /var/lib/mrtg/mrtg.ok)
Jan 19 15:55:01 NAGIOS CROND[18028]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18032]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18030]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18029]: (splunker) CMD (/home/splunker/powerData/bin/Splunk-Servertech-PDU-Load.sh 10.91.12.245 PDU-B-Rack108)
Jan 19 15:55:01 NAGIOS CROND[18024]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18035]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18027]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php > /usr/local/nagiosxi/var/dbmaint.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18041]: (nagios) CMD (/usr/local/bin/pagerduty_nagios.pl flush)
Jan 19 15:55:01 NAGIOS CROND[18031]: (splunker) CMD (/home/splunker/powerData/bin/Splunk-Servertech-PDU-Load.sh 10.91.12.244 PDU-A-Rack108)
Jan 19 15:55:01 NAGIOS CROND[18045]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18044]: (splunker) CMD (/home/splunker/bandwidthData/bin/bandwidthRRDConvert.sh 2)
Jan 19 15:55:01 NAGIOS CROND[18042]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jan 19 15:55:01 NAGIOS CROND[18048]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1)
Jan 19 15:56:01 NAGIOS CROND[20196]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jan 19 15:56:01 NAGIOS CROND[20198]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jan 19 15:56:01 NAGIOS CROND[20195]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jan 19 15:56:01 NAGIOS CROND[20199]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jan 19 15:56:01 NAGIOS CROND[20200]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jan 19 15:56:01 NAGIOS CROND[20201]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1)
Jan 19 15:56:01 NAGIOS CROND[20197]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1)
Jan 19 15:56:01 NAGIOS CROND[20203]: (nagios) CMD (/usr/local/bin/pagerduty_nagios.pl flush)
Jan 19 15:56:01 NAGIOS CROND[20206]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Re: Service Attributes - Performance Data - RED
Posted: Wed Jan 20, 2016 10:24 am
by lmiltchev
Can you post the "/usr/local/nagios/etc/nagios.cfg" and "/usr/local/nagios/etc/commands.cfg" file?
Also, run the following commands and show the output in code wraps:
Code: Select all
ls /usr/local/nagios/var/spool/xidpe | wc -l
ls /usr/local/nagios/var/spool/perfdata | wc -l
ls /usr/local/nagios/var/spool/checkresults | wc -l
tail -50 /usr/local/nagios/var/npcd.log
Re: Service Attributes - Performance Data - RED
Posted: Wed Jan 20, 2016 11:05 am
by brandon.pal
Code: Select all
ls /usr/local/nagios/var/spool/xidpe | wc -l
0
ls /usr/local/nagios/var/spool/perfdata | wc -l
0
ls /usr/local/nagios/var/spool/checkresults | wc -l
0
Code: Select all
tail -50 /usr/local/nagios/var/npcd.log
[root@**-***-05:~]$ tail -50 /usr/local/nagios/var/npcd.log
[09-13-2015 06:01:35] NPCD: ERROR: Executed command exits with return code '1'
[09-13-2015 06:01:35] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1442138468.perfdata.host'
[09-13-2015 06:01:35] NPCD: ERROR: Executed command exits with return code '1'
[09-13-2015 06:01:35] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1442138468.perfdata.service'
[10-21-2015 16:16:54] NPCD: WARN: MAX load reached: load 21.710000/10.000000 at i=0
[10-21-2015 16:17:09] NPCD: WARN: MAX load reached: load 16.970000/10.000000 at i=1
[10-21-2015 16:17:24] NPCD: WARN: MAX load reached: load 13.210000/10.000000 at i=1
[10-21-2015 16:17:39] NPCD: WARN: MAX load reached: load 10.350000/10.000000 at i=1
[10-26-2015 10:22:01] NPCD: WARN: MAX load reached: load 22.930000/10.000000 at i=0
[10-26-2015 10:22:16] NPCD: WARN: MAX load reached: load 19.870000/10.000000 at i=1
[10-26-2015 10:22:31] NPCD: WARN: MAX load reached: load 18.460000/10.000000 at i=1
[10-26-2015 10:22:46] NPCD: WARN: MAX load reached: load 16.190000/10.000000 at i=1
[10-26-2015 10:23:01] NPCD: WARN: MAX load reached: load 23.810000/10.000000 at i=1
[10-26-2015 10:23:16] NPCD: WARN: MAX load reached: load 20.200000/10.000000 at i=1
[10-26-2015 10:23:31] NPCD: WARN: MAX load reached: load 19.720000/10.000000 at i=1
[10-26-2015 10:23:46] NPCD: WARN: MAX load reached: load 15.950000/10.000000 at i=1
[10-26-2015 10:24:01] NPCD: WARN: MAX load reached: load 14.790000/10.000000 at i=1
[10-26-2015 10:24:16] NPCD: WARN: MAX load reached: load 13.700000/10.000000 at i=1
[10-26-2015 10:24:31] NPCD: WARN: MAX load reached: load 12.150000/10.000000 at i=1
[10-26-2015 10:24:46] NPCD: WARN: MAX load reached: load 11.130000/10.000000 at i=1
[10-26-2015 10:25:01] NPCD: WARN: MAX load reached: load 10.880000/10.000000 at i=1
[10-26-2015 10:25:16] NPCD: WARN: MAX load reached: load 12.200000/10.000000 at i=1
[10-26-2015 10:25:31] NPCD: WARN: MAX load reached: load 19.760000/10.000000 at i=1
[10-26-2015 10:25:46] NPCD: WARN: MAX load reached: load 17.650000/10.000000 at i=1
[10-26-2015 10:26:01] NPCD: WARN: MAX load reached: load 16.770000/10.000000 at i=1
[10-26-2015 10:26:16] NPCD: WARN: MAX load reached: load 24.590000/10.000000 at i=1
[10-26-2015 10:26:31] NPCD: WARN: MAX load reached: load 20.610000/10.000000 at i=1
[10-26-2015 10:26:46] NPCD: WARN: MAX load reached: load 31.050000/10.000000 at i=1
[10-26-2015 10:27:01] NPCD: WARN: MAX load reached: load 39.440000/10.000000 at i=1
[10-26-2015 10:27:16] NPCD: WARN: MAX load reached: load 36.930000/10.000000 at i=1
[10-26-2015 10:27:31] NPCD: WARN: MAX load reached: load 35.140000/10.000000 at i=1
[10-26-2015 10:27:46] NPCD: WARN: MAX load reached: load 34.560000/10.000000 at i=1
[10-26-2015 10:28:01] NPCD: WARN: MAX load reached: load 28.380000/10.000000 at i=1
[10-26-2015 10:28:16] NPCD: WARN: MAX load reached: load 24.570000/10.000000 at i=1
[10-26-2015 10:28:31] NPCD: WARN: MAX load reached: load 19.410000/10.000000 at i=1
[10-26-2015 10:28:46] NPCD: WARN: MAX load reached: load 15.270000/10.000000 at i=1
[10-26-2015 10:29:01] NPCD: WARN: MAX load reached: load 12.110000/10.000000 at i=1
[11-28-2015 00:00:06] NPCD: WARN: MAX load reached: load 10.590000/10.000000 at i=0
[12-30-2015 17:19:18] NPCD: Caught Termination Signal - Hasta la vista... baby
[12-30-2015 20:43:16] NPCD: npcd Daemon (0.4.14) started with PID=9424
[12-30-2015 20:43:16] NPCD: Please have a look at 'npcd -V' to get license information
[12-30-2015 20:43:16] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-19-2016 11:47:01] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-19-2016 11:47:01] NPCD: npcd Daemon (0.4.14) started with PID=416
[01-19-2016 11:47:01] NPCD: Please have a look at 'npcd -V' to get license information
[01-19-2016 11:47:01] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-19-2016 12:22:43] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-19-2016 12:22:43] NPCD: npcd Daemon (0.4.14) started with PID=21988
[01-19-2016 12:22:43] NPCD: Please have a look at 'npcd -V' to get license information
[01-19-2016 12:22:43] NPCD: HINT: load_threshold is enabled - ('10.000000')
Re: Service Attributes - Performance Data - RED
Posted: Wed Jan 20, 2016 11:11 am
by WillemDH
HINT: load_threshold is enabled - ('10.000000')
WARN: MAX load reached: load 36.930000/10.000000 at i=1
How many CPU's does your server has? How many hosts / services? Can you show us a screenshot of CPU Load over time, let's say one month and 7 days?
Re: Service Attributes - Performance Data - RED
Posted: Wed Jan 20, 2016 11:26 am
by brandon.pal
WillemDH wrote:HINT: load_threshold is enabled - ('10.000000')
WARN: MAX load reached: load 36.930000/10.000000 at i=1
How many CPU's does your server has? How many hosts / services? Can you show us a screenshot of CPU Load over time, let's say one month and 7 days?
CPU's: 2 sockets, 4 cores per socket.
Hosts: 128
Services: 1778
Unfortunately the only CPU stats I have are:
Nagios_Prod_ServerStats.png
Re: Service Attributes - Performance Data - RED
Posted: Wed Jan 20, 2016 11:47 am
by WillemDH
Please put it on your to do to monitor your Nagios XI server more. My personal advice would be to start monitoring asap CPU Load, CPU Usage, Memory Usage, Open Files, disk usage, disk load, swap usage, Also follow the Nagiostats Wizard to monitor internal performance.
Priority should be to find out why NPCD is exceeding max load threshold. Can you please post the content of /usr/local/nagios/etc/pnp/npcd.cfg? a CPU load of 36 with 8 cores and only 128 hosts and 1778 services does not seem normal to me.