CPU Usage / Load is high?!

[dkowalski]

My nagiosxi instalation is vmware 2011r1.9, upgraded to 2.0. At present, Im only monitoring 1 service on a sql server machine. my usage should not be this high. well... at least I dont think it should.

Nagios XI
Server Stats
Server Statistics Metric

Value Load
1-min 6.85
5-min 6.52
15-min 4.65

CPU Stats
User 15.40%
Nice 0.00%
System 84.60%
I/O Wait 0.00%
Steal 0.00%
Idle 0.00%

Memory
Total 2022 MB
Used 522 MB
Free 1499 MB
Shared 0 MB
Buffers 25 MB
Cached 119 MB

Swap
Total 255 MB
Used 0 MB
Free 255 MB
Last Updated: 09/02/2012 07:21:12

NPCD LOG

Code: Select all

[02-09-2012 07:42:25] NPCD: Processing file 'service-perfdata.1328737084'
[02-09-2012 07:42:25] NPCD: ERROR: Executed command exits with return code '126'
[02-09-2012 07:42:25] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/p                                 rocess_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//service-perfdata.                                 1328737039'
[02-09-2012 07:42:25] NPCD: ERROR: Executed command exits with return code '126'
[02-09-2012 07:42:25] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/p                                 rocess_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//service-perfdata.                                 1328737054'
[02-09-2012 07:42:25] NPCD: ERROR: Executed command exits with return code '126'
[02-09-2012 07:42:25] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/p                                 rocess_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//service-perfdata.                                 1328737069'
[02-09-2012 07:42:25] NPCD: ERROR: Executed command exits with return code '126'
[02-09-2012 07:42:25] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/p                                 rocess_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//service-perfdata.                                 1328737084'
[02-09-2012 07:42:25] NPCD: ERROR: Executed command exits with return code '126'
[02-09-2012 07:42:25] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/p                                 rocess_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//service-perfdata.                                 1328737099'

nagios log

Code: Select all

[1328796680] SERVICE ALERT: localhost;Current Load;WARNING;HARD;4;WARNING - load average: 7.21, 5.31, 2.79
[1328796680] SERVICE NOTIFICATION: russell;localhost;Current Load;WARNING;xi_service_notification_handler;WARNING - load average: 7.21, 5.31, 2.79
[1328796980] SERVICE ALERT: localhost;Current Load;CRITICAL;HARD;4;CRITICAL - load average: 8.64, 6.59, 3.99
[1328796980] SERVICE NOTIFICATION: russell;localhost;Current Load;CRITICAL;xi_service_notification_handler;CRITICAL - load average: 8.64, 6.59, 3.99

Perfdatalog does not exist

[scottwilkerson]

error code 126 is usually “command not executable”

Can we check the permissions on on the following

Code: Select all

ls -l /usr/local/nagios/libexec/process_perfdata.pl

Additionally you can turn logging on for perfdata in /usr/local/nagios/etc/pnp/process_perfdata.cfg

By setting LOG_LEVEL = 1 or 2

[dkowalski]

Code: Select all

[dkowalski@nagiosxi var]# ls -l /usr/local/nagios/libexec/process_perfdata.pl
-rw-r--r-- 1 nagios nagios 42724 Feb  7 08:47 /usr/local/nagios/libexec/process_perfdata.pl

Ive enabled the perfdata log but dont have anything as yet.

[dkowalski]

When I stop the performance graph service my load and cpu usage return to sane levels

[mguthrie]

Try:

Code: Select all

chmod +x /usr/local/nagios/libexec/process_perfdata.pl

And then restart the performance graphing service.

Code: Select all

service npcd restart

If the performance grapher has fallen behind somehow in processing results, there will be in increase in CPU load while it catches up.

The logging options in

/usr/local/nagios/etc/pnp/npcd.cfg and process_perfdata.cfg

will also increase CPU load quite a bit, so once you're done debugging things it's a good idea to set logging back to 0 in those files.

[dkowalski]

didnt think to make the .pl file executable...

Ive done that and restart the service and im getting a log file now.

Not getting any hard errors per say but getting "/usr/local/nagios/share/perfdata/.pnp-internal/runtime_errors.rrd updated" at the bottom of this log snippit

Code: Select all

2012-02-09 13:29:35 [27994] [1] PNP exiting (runtime 0.000988s) ...
2012-02-09 13:29:35 [27998] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:35 [27995] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:35 [27995] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656631
2012-02-09 13:29:35 [27995] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656631 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656631-PID-27995 for bulk update
2012-02-09 13:29:35 [27995] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656631-PID-27995 for bulk update
2012-02-09 13:29:35 [27995] [1] 0 lines processed
2012-02-09 13:29:35 [27995] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656631-PID-27995 deleted
2012-02-09 13:29:35 [27995] [1] PNP exiting (runtime 0.001117s) ...
2012-02-09 13:29:35 [27996] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656601
2012-02-09 13:29:35 [27996] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656601 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656601-PID-27996 for bulk update
2012-02-09 13:29:35 [27996] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656601-PID-27996 for bulk update
2012-02-09 13:29:35 [27996] [1] 0 lines processed
2012-02-09 13:29:35 [27996] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656601-PID-27996 deleted
2012-02-09 13:29:35 [27996] [1] PNP exiting (runtime 0.009386s) ...
2012-02-09 13:29:35 [27997] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:35 [27997] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656586
2012-02-09 13:29:35 [27997] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656586 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656586-PID-27997 for bulk update
2012-02-09 13:29:35 [27997] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656586-PID-27997 for bulk update
2012-02-09 13:29:35 [27997] [1] 0 lines processed
2012-02-09 13:29:35 [27997] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656586-PID-27997 deleted
2012-02-09 13:29:35 [27997] [1] PNP exiting (runtime 0.000715s) ...
2012-02-09 13:29:35 [27998] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656571
2012-02-09 13:29:35 [27998] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656571 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656571-PID-27998 for bulk update
2012-02-09 13:29:35 [27998] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656571-PID-27998 for bulk update
2012-02-09 13:29:35 [27998] [1] 0 lines processed
2012-02-09 13:29:35 [27998] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656571-PID-27998 deleted
2012-02-09 13:29:35 [27998] [1] PNP exiting (runtime 0.017842s) ...
2012-02-09 13:29:36 [28004] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28005] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28006] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28008] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28007] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28007] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28005] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28005] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656706
2012-02-09 13:29:36 [28005] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656706 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656706-PID-28005 for bulk update
2012-02-09 13:29:36 [28005] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656706-PID-28005 for bulk update
2012-02-09 13:29:36 [28005] [1] 0 lines processed
2012-02-09 13:29:36 [28005] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656706-PID-28005 deleted
2012-02-09 13:29:36 [28005] [1] PNP exiting (runtime 0.000987s) ...
2012-02-09 13:29:36 [28004] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28004] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656691
2012-02-09 13:29:36 [28004] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656691 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656691-PID-28004 for bulk update
2012-02-09 13:29:36 [28004] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656691-PID-28004 for bulk update
2012-02-09 13:29:36 [28006] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28006] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656676
2012-02-09 13:29:36 [28006] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656676 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656676-PID-28006 for bulk update
2012-02-09 13:29:36 [28006] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656676-PID-28006 for bulk update
2012-02-09 13:29:36 [28006] [1] 0 lines processed
2012-02-09 13:29:36 [28006] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656676-PID-28006 deleted
2012-02-09 13:29:36 [28006] [1] PNP exiting (runtime 0.000744s) ...
2012-02-09 13:29:36 [28004] [2] Processing Line 1
2012-02-09 13:29:36 [28004] [2] Datatype set to 'HOSTPERFDATA'
2012-02-09 13:29:36 [28004] [1] Found Performance Data for sql64.lvm1.local / _HOST_ (rta=0.239ms;3000.000;5000.000;0; pl=0%;80;100                           ;;)
2012-02-09 13:29:36 [28004] [2] No Custom Template found for check_xi_host_ping (/usr/local/nagios/etc/pnp/check_commands/check_xi_                           host_ping.cfg)
2012-02-09 13:29:36 [28004] [2] Template is check_xi_host_ping.php
2012-02-09 13:29:36 [28004] [2] No Custom Template found for check_xi_host_ping (/usr/local/nagios/etc/pnp/check_commands/check_xi_                           host_ping.cfg)
2012-02-09 13:29:36 [28007] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656661
2012-02-09 13:29:36 [28007] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656661 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656661-PID-28007 for bulk update
2012-02-09 13:29:36 [28007] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656661-PID-28007 for bulk update
2012-02-09 13:29:36 [28007] [1] 0 lines processed
2012-02-09 13:29:36 [28007] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656661-PID-28007 deleted
2012-02-09 13:29:36 [28008] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28008] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656646
2012-02-09 13:29:36 [28008] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656646 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656646-PID-28008 for bulk update
2012-02-09 13:29:36 [28008] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656646-PID-28008 for bulk update
2012-02-09 13:29:36 [28004] [2] Template is check_xi_host_ping.php
2012-02-09 13:29:36 [28004] [2] data2rrd called
2012-02-09 13:29:36 [28004] [2] RRDs::update /usr/local/nagios/share/perfdata/sql64.lvm1.local/_HOST_.rrd 1328656681:0.239:0
2012-02-09 13:29:36 [28004] [2] /usr/local/nagios/share/perfdata/sql64.lvm1.local/_HOST_.rrd updated
2012-02-09 13:29:36 [28007] [1] PNP exiting (runtime 0.009267s) ...
2012-02-09 13:29:36 [28008] [2] Processing Line 1
2012-02-09 13:29:36 [28008] [2] Datatype set to 'HOSTPERFDATA'
2012-02-09 13:29:36 [28008] [1] Found Performance Data for localhost / _HOST_ (rta=0.027ms;3000.000;5000.000;0; pl=0%;80;100;;)
2012-02-09 13:29:36 [28008] [2] No Custom Template found for check-host-alive (/usr/local/nagios/etc/pnp/check_commands/check-host-                           alive.cfg)
2012-02-09 13:29:36 [28008] [2] Template is check-host-alive.php
2012-02-09 13:29:36 [28008] [2] No Custom Template found for check-host-alive (/usr/local/nagios/etc/pnp/check_commands/check-host-                           alive.cfg)
2012-02-09 13:29:36 [28008] [2] Template is check-host-alive.php
2012-02-09 13:29:36 [28008] [2] data2rrd called
2012-02-09 13:29:36 [28008] [2] RRDs::update /usr/local/nagios/share/perfdata/localhost/_HOST_.rrd 1328656641:0.027:0
2012-02-09 13:29:36 [28008] [2] /usr/local/nagios/share/perfdata/localhost/_HOST_.rrd updated
2012-02-09 13:29:36 [28004] [1] 1 lines processed
2012-02-09 13:29:36 [28004] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656691-PID-28004 deleted
2012-02-09 13:29:36 [28004] [1] PNP exiting (runtime 0.018359s) ...
2012-02-09 13:29:36 [28008] [1] 1 lines processed
2012-02-09 13:29:36 [28008] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656646-PID-28008 deleted
2012-02-09 13:29:36 [28008] [1] PNP exiting (runtime 0.029581s) ...
2012-02-09 13:29:36 [28016] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28015] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28017] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28014] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28018] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:36 [28014] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28017] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28015] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28018] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28014] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656766
2012-02-09 13:29:36 [28014] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656766 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656766-PID-28014 for bulk update
2012-02-09 13:29:36 [28014] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656766-PID-28014 for bulk update
2012-02-09 13:29:36 [28014] [1] 0 lines processed
2012-02-09 13:29:36 [28014] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656766-PID-28014 deleted
2012-02-09 13:29:36 [28014] [1] PNP exiting (runtime 0.008208s) ...
2012-02-09 13:29:36 [28015] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656781
2012-02-09 13:29:36 [28015] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656781 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656781-PID-28015 for bulk update
2012-02-09 13:29:36 [28015] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656781-PID-28015 for bulk update
2012-02-09 13:29:36 [28015] [1] 0 lines processed
2012-02-09 13:29:36 [28015] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656781-PID-28015 deleted
2012-02-09 13:29:36 [28015] [1] PNP exiting (runtime 0.007772s) ...
2012-02-09 13:29:36 [28017] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656736
2012-02-09 13:29:36 [28017] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656736 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656736-PID-28017 for bulk update
2012-02-09 13:29:36 [28017] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656736-PID-28017 for bulk update
2012-02-09 13:29:36 [28017] [1] 0 lines processed
2012-02-09 13:29:36 [28017] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656736-PID-28017 deleted
2012-02-09 13:29:36 [28017] [1] PNP exiting (runtime 0.013343s) ...
2012-02-09 13:29:36 [28016] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:36 [28016] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656751
2012-02-09 13:29:36 [28016] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656751 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656751-PID-28016 for bulk update
2012-02-09 13:29:36 [28016] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656751-PID-28016 for bulk update
2012-02-09 13:29:36 [28016] [1] 0 lines processed
2012-02-09 13:29:36 [28016] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656751-PID-28016 deleted
2012-02-09 13:29:36 [28016] [1] PNP exiting (runtime 0.00068s) ...
2012-02-09 13:29:36 [28018] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656721
2012-02-09 13:29:36 [28018] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656721 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656721-PID-28018 for bulk update
2012-02-09 13:29:36 [28018] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656721-PID-28018 for bulk update
2012-02-09 13:29:36 [28018] [1] 0 lines processed
2012-02-09 13:29:36 [28018] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656721-PID-28018 deleted
2012-02-09 13:29:36 [28018] [1] PNP exiting (runtime 0.019263s) ...
2012-02-09 13:29:37 [28026] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28024] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28027] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28028] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28025] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28025] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28024] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28024] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656841
2012-02-09 13:29:37 [28024] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656841 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656841-PID-28024 for bulk update
2012-02-09 13:29:37 [28024] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656841-PID-28024 for bulk update
2012-02-09 13:29:37 [28024] [1] 0 lines processed
2012-02-09 13:29:37 [28024] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656841-PID-28024 deleted
2012-02-09 13:29:37 [28024] [1] PNP exiting (runtime 0.000892s) ...
2012-02-09 13:29:37 [28026] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28026] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656826
2012-02-09 13:29:37 [28026] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656826 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656826-PID-28026 for bulk update
2012-02-09 13:29:37 [28026] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656826-PID-28026 for bulk update
2012-02-09 13:29:37 [28026] [1] 0 lines processed
2012-02-09 13:29:37 [28026] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656826-PID-28026 deleted
2012-02-09 13:29:37 [28025] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656856
2012-02-09 13:29:37 [28025] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656856 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656856-PID-28025 for bulk update
2012-02-09 13:29:37 [28025] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656856-PID-28025 for bulk update
2012-02-09 13:29:37 [28025] [1] 0 lines processed
2012-02-09 13:29:37 [28025] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656856-PID-28025 deleted
2012-02-09 13:29:37 [28027] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28027] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656811
2012-02-09 13:29:37 [28027] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656811 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656811-PID-28027 for bulk update
2012-02-09 13:29:37 [28027] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656811-PID-28027 for bulk update
2012-02-09 13:29:37 [28027] [1] 0 lines processed
2012-02-09 13:29:37 [28027] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656811-PID-28027 deleted
2012-02-09 13:29:37 [28028] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28028] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656796
2012-02-09 13:29:37 [28028] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656796 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656796-PID-28028 for bulk update
2012-02-09 13:29:37 [28026] [1] PNP exiting (runtime 0.000655s) ...
2012-02-09 13:29:37 [28025] [1] PNP exiting (runtime 0.008263s) ...
2012-02-09 13:29:37 [28027] [1] PNP exiting (runtime 0.010828s) ...
2012-02-09 13:29:37 [28028] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656796-PID-28028 for bulk update
2012-02-09 13:29:37 [28028] [1] 0 lines processed
2012-02-09 13:29:37 [28028] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656796-PID-28028 deleted
2012-02-09 13:29:37 [28028] [1] PNP exiting (runtime 0.012185s) ...
2012-02-09 13:29:37 [28035] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28034] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28036] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28037] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28038] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28035] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28035] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656931
2012-02-09 13:29:37 [28035] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656931 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656931-PID-28035 for bulk update
2012-02-09 13:29:37 [28035] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656931-PID-28035 for bulk update
2012-02-09 13:29:37 [28035] [1] 0 lines processed
2012-02-09 13:29:37 [28035] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656931-PID-28035 deleted
2012-02-09 13:29:37 [28034] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28034] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656916
2012-02-09 13:29:37 [28034] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656916 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656916-PID-28034 for bulk update
2012-02-09 13:29:37 [28034] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656916-PID-28034 for bulk update
2012-02-09 13:29:37 [28034] [1] 0 lines processed
2012-02-09 13:29:37 [28034] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656916-PID-28034 deleted
2012-02-09 13:29:37 [28034] [1] PNP exiting (runtime 0.000953s) ...
2012-02-09 13:29:37 [28038] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28037] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28036] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:37 [28036] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656901
2012-02-09 13:29:37 [28036] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656901 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656901-PID-28036 for bulk update
2012-02-09 13:29:37 [28036] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656901-PID-28036 for bulk update
2012-02-09 13:29:37 [28036] [1] 0 lines processed
2012-02-09 13:29:37 [28036] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656901-PID-28036 deleted
2012-02-09 13:29:37 [28036] [1] PNP exiting (runtime 0.000823s) ...
2012-02-09 13:29:37 [28035] [1] PNP exiting (runtime 0.013413s) ...
2012-02-09 13:29:37 [28037] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656886
2012-02-09 13:29:37 [28037] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656886 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656886-PID-28037 for bulk update
2012-02-09 13:29:37 [28037] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656886-PID-28037 for bulk update
2012-02-09 13:29:37 [28037] [1] 0 lines processed
2012-02-09 13:29:37 [28037] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656886-PID-28037 deleted
2012-02-09 13:29:37 [28038] [2] searching for /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656871
2012-02-09 13:29:37 [28038] [2] renaming /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656871 to /usr/local/nagios/var/sp                           ool/perfdata//host-perfdata.1328656871-PID-28038 for bulk update
2012-02-09 13:29:37 [28038] [2] reading /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656871-PID-28038 for bulk update
2012-02-09 13:29:37 [28038] [1] 0 lines processed
2012-02-09 13:29:37 [28038] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328656871-PID-28038 deleted
2012-02-09 13:29:37 [28037] [1] PNP exiting (runtime 0.012536s) ...
2012-02-09 13:29:37 [28038] [1] PNP exiting (runtime 0.020905s) ...
2012-02-09 13:29:37 [28050] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28048] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28049] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28051] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:37 [28052] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2012-02-09 13:29:38 [28050] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2012-02-09 13:29:38 [28051] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
^C
[root@nagiosxi var]# less perfdata.log
2012-02-09 13:26:41 [23704] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328629677-PID-23704 deleted
2012-02-09 13:26:41 [23703] [1] PNP exiting (runtime 0.000463s) ...
2012-02-09 13:26:41 [23701] [1] PNP exiting (runtime 0.007469s) ...
2012-02-09 13:26:41 [23704] [1] PNP exiting (runtime 0.000464s) ...
2012-02-09 13:26:41 [23705] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg)
2012-02-09 13:26:41 [23705] [2] Template is pnp-runtime.php
2012-02-09 13:26:41 [23705] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg)
2012-02-09 13:26:41 [23705] [2] Template is pnp-runtime.php
2012-02-09 13:26:41 [23705] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg)
2012-02-09 13:26:41 [23705] [2] Template is pnp-runtime.php
2012-02-09 13:26:41 [23705] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg)
2012-02-09 13:26:41 [23705] [2] Template is pnp-runtime.php
2012-02-09 13:26:41 [23705] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg)
2012-02-09 13:26:41 [23705] [2] Template is pnp-runtime.php
2012-02-09 13:26:41 [23705] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg)
2012-02-09 13:26:41 [23705] [2] Template is pnp-runtime.php
2012-02-09 13:26:41 [23705] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg)
2012-02-09 13:26:41 [23705] [2] Template is pnp-runtime.php
2012-02-09 13:26:41 [23702] [2] Processing Line 1
2012-02-09 13:26:41 [23702] [2] Datatype set to 'HOSTPERFDATA'
2012-02-09 13:26:41 [23702] [1] Found Performance Data for localhost / _HOST_ (rta=0.025ms;3000.000;5000.000;0; pl=0%;80;100;;)
2012-02-09 13:26:41 [23702] [2] No Custom Template found for check-host-alive (/usr/local/nagios/etc/pnp/check_commands/check-host-alive.cfg)
2012-02-09 13:26:41 [23702] [2] Template is check-host-alive.php
2012-02-09 13:26:41 [23702] [2] No Custom Template found for check-host-alive (/usr/local/nagios/etc/pnp/check_commands/check-host-alive.cfg)
2012-02-09 13:26:41 [23702] [2] Template is check-host-alive.php
2012-02-09 13:26:41 [23702] [2] data2rrd called
2012-02-09 13:26:41 [23705] [2] data2rrd called
2012-02-09 13:26:41 [23705] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_runtime.rrd 1328629648:0.000248
2012-02-09 13:26:41 [23702] [2] RRDs::update /usr/local/nagios/share/perfdata/localhost/_HOST_.rrd 1328629687:0.025:0
2012-02-09 13:26:41 [23705] [2] /usr/local/nagios/share/perfdata/.pnp-internal/runtime_runtime.rrd updated
2012-02-09 13:26:41 [23705] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_rows.rrd 1328629648:0
2012-02-09 13:26:41 [23705] [2] /usr/local/nagios/share/perfdata/.pnp-internal/runtime_rows.rrd updated
2012-02-09 13:26:41 [23705] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_errors.rrd 1328629648:0
2012-02-09 13:26:41 [23702] [2] /usr/local/nagios/share/perfdata/localhost/_HOST_.rrd updated
2012-02-09 13:26:41 [23702] [1] 1 lines processed
2012-02-09 13:26:41 [23702] [1] /usr/local/nagios/var/spool/perfdata//host-perfdata.1328629692-PID-23702 deleted
2012-02-09 13:26:41 [23702] [1] PNP exiting (runtime 0.058673s) ...
2012-02-09 13:26:41 [23705] [2] /usr/local/nagios/share/perfdata/.pnp-internal/runtime_errors.rrd updated
2012-02-09 13:26:41 [23705] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_invalid.rrd 1328629648:0

CPU and load are still very high especially for not having a working load on the server.

[scottwilkerson]

CPU and load are still very high especially for not having a working load on the server.

As mguthrie mentioned above:

If the performance grapher has fallen behind somehow in processing results, there will be in increase in CPU load while it catches up.

The logging options in

/usr/local/nagios/etc/pnp/npcd.cfg and process_perfdata.cfg

will also increase CPU load quite a bit, so once you're done debugging things it's a good idea to set logging back to 0 in those files.

[dkowalski]

This has worked as you said. my load and cpu usage is now normal. Ive reset my logging back to minimal

Thanks for your help. I have one more issues of being logged out after 30 seconds but Ill start a new thread.

[scottwilkerson]

Great to hear the CPU has went down.