Performance graphs stopped updating

tonyyarusso · Post by **tonyyarusso** » Tue Mar 01, 2011 11:24 am

Okay, I'll have to admit that I can't come up with a good explanation for what you're seeing at this point, so I'm going to start taking shots in the dark here. First let's try making sure the database and filesystem can pass an integrity check. To do this, run the following:

Stop various daemons:

Code: Select all

service nagiosxi stop
service npcd stop
service ndo2db stop
service nagios stop
service postgresql stop
service httpd stop

Check a repair on some tables:

Code: Select all

/usr/local/nagiosxi/scripts/repairmysql.sh nagios nagios_logentries
/usr/local/nagiosxi/scripts/repairmysql.sh nagios nagios_statehistory
/usr/local/nagiosxi/scripts/repairmysql.sh nagios nagios_notifications
/usr/local/nagiosxi/scripts/repairmysql.sh nagios nagios_contactnotificationmethods
/usr/local/nagiosxi/scripts/repairmysql.sh nagios nagios_contactnotifications
/usr/local/nagiosxi/scripts/repairmysql.sh nagios nagios_externalcommands
/usr/local/nagiosxi/scripts/repairmysql.sh nagios nagios_eventhandlers

Restart MySQL:

Code: Select all

service mysqld restart

Force a filesystem check on next boot:

Code: Select all

touch /forcefsck

Reboot the server:

Code: Select all

shutdown -r now

Once the system is up and stabilized, run the following and give me the full output (there will be a lot, and it will take a few minutes to run - you can put '> somefile' to redirect it to a file if your terminal doesn't support scrolling):

Code: Select all

c=0; while [ $c -lt 6 ]; do top -b -c | head -27; echo ""; sleep 72; c=$(($c+1)); done

FLCUISIT · Post by **FLCUISIT** » Tue Mar 01, 2011 12:55 pm

Here is the output of the script:

Code: Select all

top - 12:20:32 up 9 min,  1 user,  load average: 18.54, 9.98, 4.33
Tasks: 257 total,  36 running, 221 sleeping,   0 stopped,   0 zombie
Cpu(s):  9.2%us, 46.5%sy,  0.0%ni, 36.5%id,  7.2%wa,  0.1%hi,  0.5%si,  0.0%st
Mem:   3107560k total,  1201348k used,  1906212k free,   660416k buffers
Swap:  1048568k total,        0k used,  1048568k free,   234272k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
24943 nagios    25   0 10144 6084 1720 R 19.2  0.2   0:00.31 /usr/bin/perl -w /u
 3126 mysql     15   0  144m  22m 4116 S 15.3  0.8   0:24.47 /usr/libexec/mysqld
 3579 nagios    25   0 15840 3908  876 R 14.4  0.1   0:23.90 /usr/local/nagios/b
25187 nagios    25   0  6608 2404 1564 R  9.6  0.1   0:00.10 /usr/bin/perl -w /u
25040 nagios    25   0  4620 1100  972 S  4.8  0.0   0:00.05 /bin/sh /usr/local/
 3517 nagios    15   0  7308 1568  912 S  2.9  0.1   0:02.29 /usr/local/nagios/b
25078 root      15   0  2420 1012  696 R  1.9  0.0   0:00.08 top -b -c
25179 nagios    25   0  4544  700  600 R  1.9  0.0   0:00.02 sh -c /usr/local/na
25184 nagios    22   0  4320  828  732 R  1.9  0.0   0:00.02 awk {x=$2} END { pr
25196 nagios    22   0  2244  424  356 R  1.9  0.0   0:00.02 awk {x=$3} END { pr
25201 nagios    23   0  2112  392  324 R  1.9  0.0   0:00.02 awk {x=$3} END { pr
25205 nagios    23   0     0    0    0 R  1.9  0.0   0:00.02 [bc]
25210 nagios    18   0  4620  448  320 S  1.9  0.0   0:00.02 /bin/sh /usr/local/
25222 nagios    25   0  4620 1076  956 R  1.9  0.0   0:00.02 sh -c /usr/local/na
25230 nagios    22   0  4292  520  444 R  1.9  0.0   0:00.02 awk {x=$3} END { pr
24669 nagios    18   0  4620 1208 1044 S  1.0  0.0   0:00.09 /bin/sh /usr/local/
24677 nagios    18   0  4624 1216 1044 S  1.0  0.0   0:00.08 /bin/sh /usr/local/
24715 nagios    18   0  4620 1212 1044 R  1.0  0.0   0:00.08 /bin/sh /usr/local/
24755 nagios    18   0  4620 1200 1036 S  1.0  0.0   0:00.09 /bin/sh /usr/local/
24832 nagios    18   0  4620 1204 1036 S  1.0  0.0   0:00.07 /bin/sh /usr/local/

top - 12:21:49 up 10 min,  1 user,  load average: 28.50, 14.75, 6.39
Tasks: 239 total,  22 running, 216 sleeping,   0 stopped,   1 zombie
Cpu(s):  9.9%us, 51.3%sy,  0.0%ni, 32.0%id,  6.3%wa,  0.1%hi,  0.5%si,  0.0%st
Mem:   3107560k total,  1216548k used,  1891012k free,   660656k buffers
Swap:  1048568k total,        0k used,  1048568k free,   234972k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
31813 nagios    25   0  9688 5576 1680 R 30.9  0.2   0:00.29 /usr/bin/perl -w /u
31853 nagios    25   0  9820 5680 1680 R 30.9  0.2   0:00.30 /usr/bin/perl -w /u
31859 nagios    25   0  6476 2304 1564 R 13.9  0.1   0:00.09 /usr/bin/perl -w /u
 3579 nagios    25   0 15836 3904  876 R 10.8  0.1   0:31.32 /usr/local/nagios/b
31922 nagios    25   0  5832 1356 1104 R  9.3  0.0   0:00.06 /usr/bin/perl -w /u
31953 nagios    20   0  4620 1100  972 R  9.3  0.0   0:00.06 /bin/sh /usr/local/
28274 nagios    15   0 29388  13m 5532 S  3.1  0.5   0:00.92 /usr/bin/php -q /us
31899 root      15   0  2420 1000  696 R  3.1  0.0   0:00.07 top -b -c
31940 nagios    22   0  1800  340  292 R  3.1  0.0   0:00.02 sed -e s,[\/][^\/][
31943 nagios    23   0  4292  520  444 R  3.1  0.0   0:00.02 awk {x=$3} END { pr
31944 nagios    22   0  2020  436  332 R  3.1  0.0   0:00.02 bc
31951 nagios    23   0  4292  500  424 R  3.1  0.0   0:00.02 awk {x=$3} END { pr
31967 nagios    23   0  2020  380  304 R  3.1  0.0   0:00.02 bc
 3126 mysql     15   0  148m  23m 4116 S  1.5  0.8   0:27.55 /usr/libexec/mysqld
31639 nagios    18   0  4620 1204 1036 S  1.5  0.0   0:00.08 /bin/sh /usr/local/
31784 nagios    18   0  4620 1116  984 S  1.5  0.0   0:00.07 /bin/sh /usr/local/
31785 nagios    18   0     0    0    0 Z  1.5  0.0   0:00.02 [nagios] <defunct>
31786 nagios    18   0  4620 1204 1036 S  1.5  0.0   0:00.07 /bin/sh /usr/local/
31860 nagios    25   0 15840 3264  232 S  1.5  0.1   0:00.01 /usr/local/nagios/b
31924 nagios    21   0  4620  484  320 S  1.5  0.0   0:00.01 /bin/sh /usr/local/

top - 12:23:05 up 12 min,  1 user,  load average: 27.05, 17.07, 7.76
Tasks: 226 total,  11 running, 215 sleeping,   0 stopped,   0 zombie
Cpu(s): 10.6%us, 55.0%sy,  0.0%ni, 28.3%id,  5.6%wa,  0.0%hi,  0.5%si,  0.0%st
Mem:   3107560k total,  1269184k used,  1838376k free,   660948k buffers
Swap:  1048568k total,        0k used,  1048568k free,   235472k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 5842 nagios    25   0 29264  14m 5840 R 56.2  0.5   0:00.76 /usr/bin/php -q /us
 5810 nagios    25   0 28996  13m 5560 S 29.0  0.4   0:00.76 /usr/bin/php -q /us
 6017 nagios    18   0  2500  952  816 S 10.9  0.0   0:00.07 /bin/sh /usr/local/
 6019 postgres  15   0 21904 4056 3124 S 10.9  0.1   0:00.08 postgres: nagiosxi
 6031 nagios    25   0  4012  628  548 S 10.9  0.0   0:00.06 /usr/local/nagios/l
 3579 nagios    25   0 15836 3904  876 R  9.1  0.1   0:39.26 /usr/local/nagios/b
 6072 postgres  16   0 21904 2768 1984 S  9.1  0.1   0:00.05 postgres: nagiosxi
 6020 nagios    24   0  4012  728  640 S  7.3  0.0   0:00.05 /usr/local/nagios/l
 6033 nagios    24   0  4012  632  552 S  7.3  0.0   0:00.04 /usr/local/nagios/l
 6045 nagios    25   0  4012  632  548 S  7.3  0.0   0:00.04 /usr/local/nagios/l
 6064 nagios    25   0  1832  332  256 R  7.3  0.0   0:00.04 /usr/local/nagios/l
 5806 nagios    17   0 31956  16m 6024 S  5.4  0.6   0:00.88 /usr/bin/php -q /us
 6069 nagios    25   0  4544  760  656 R  3.6  0.0   0:00.02 sh -c /usr/local/na
 6077 nagios    23   0     0    0    0 R  3.6  0.0   0:00.02 [bc]
 6078 nagios    23   0  2020  496  388 R  3.6  0.0   0:00.02 bc
 3208 postgres  15   0 21232 2896 2492 S  1.8  0.1   0:00.77 /usr/bin/postmaster
 5861 nagios    18   0  4624 1220 1044 S  1.8  0.0   0:00.09 /bin/sh /usr/local/
 5864 nagios    18   0  4620 1212 1044 S  1.8  0.0   0:00.10 /bin/sh /usr/local/
 5917 nagios    18   0  4620 1204 1036 S  1.8  0.0   0:00.08 /bin/sh /usr/local/
 5980 postgres  15   0 21904 3696 2808 S  1.8  0.1   0:00.08 postgres: nagiosxi

top - 12:24:21 up 13 min,  1 user,  load average: 12.64, 14.71, 7.61
Tasks: 169 total,   1 running, 168 sleeping,   0 stopped,   0 zombie
Cpu(s): 10.5%us, 54.8%sy,  0.0%ni, 29.2%id,  5.0%wa,  0.0%hi,  0.4%si,  0.0%st
Mem:   3107560k total,  1195724k used,  1911836k free,   661152k buffers
Swap:  1048568k total,        0k used,  1048568k free,   235896k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 3579 nagios    15   0 15836 3904  876 S  3.8  0.1   0:44.12 /usr/local/nagios/b
 3126 mysql     15   0  148m  23m 4116 S  1.9  0.8   0:33.22 /usr/libexec/mysqld
10813 root      15   0  2420  944  696 R  1.9  0.0   0:00.02 top -b -c
    1 root      15   0  2156  640  552 S  0.0  0.0   0:02.29 init [3]
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.13 [migration/0]
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/0]
    4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/0]
    5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.11 [migration/1]
    6 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/1]
    7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/1]
    8 root      RT  -5     0    0    0 S  0.0  0.0   0:00.11 [migration/2]
    9 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/2]
   10 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/2]
   11 root      RT  -5     0    0    0 S  0.0  0.0   0:00.12 [migration/3]
   12 root      39  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/3]
   13 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/3]
   14 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [events/0]
   15 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [events/1]
   16 root      10  -5     0    0    0 S  0.0  0.0   0:00.02 [events/2]
   17 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [events/3]

top - 12:25:33 up 14 min,  1 user,  load average: 15.97, 14.80, 8.06
Tasks: 261 total,  31 running, 228 sleeping,   0 stopped,   2 zombie
Cpu(s): 10.8%us, 55.5%sy,  0.0%ni, 28.5%id,  4.7%wa,  0.0%hi,  0.5%si,  0.0%st
Mem:   3107560k total,  1237568k used,  1869992k free,   661376k buffers
Swap:  1048568k total,        0k used,  1048568k free,   243844k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
15382 nagios    25   0  9004 4900 1644 R 25.3  0.2   0:00.24 /usr/bin/perl -w /u
15386 nagios    25   0  8588 4492 1624 R 22.3  0.1   0:00.20 /usr/bin/perl -w /u
15522 nagios    24   0  4620 1108  972 S  8.9  0.0   0:00.06 /bin/sh /usr/local/
15430 nagios    25   0  4620 1100  972 S  6.0  0.0   0:00.05 /bin/sh /usr/local/
15569 nagios    23   0  1832  316  244 R  6.0  0.0   0:00.04 /usr/local/nagios/l
 3579 nagios    19   0 15836 3900  876 S  4.5  0.1   0:48.59 /usr/local/nagios/b
13475 nagios    16   0 29388  14m 5992 S  3.0  0.5   0:00.96 /usr/bin/php -q /us
15470 nagios    24   0  4620 1164 1028 S  3.0  0.0   0:00.06 /bin/sh /usr/local/
15494 root      15   0  2420 1012  696 R  3.0  0.0   0:00.07 top -b -c
15545 nagios    23   0  2024  548  436 R  3.0  0.0   0:00.02 bc
15556 nagios    22   0  2020  484  376 R  3.0  0.0   0:00.02 bc
15558 nagios    23   0     0    0    0 R  3.0  0.0   0:00.02 [bc]
15567 nagios    23   0  3876  548  480 R  3.0  0.0   0:00.02 sed -e s,[\/][^\/][
15568 nagios    21   0  4000  692  612 S  3.0  0.0   0:00.02 grep -vi nan
15576 nagios    23   0  4320  668  572 R  3.0  0.0   0:00.02 awk {x=$3} END { pr
15578 nagios    22   0  2020  528  420 R  3.0  0.0   0:00.02 bc
 3126 mysql     15   0  148m  23m 4148 S  1.5  0.8   0:36.19 /usr/libexec/mysqld
13644 postgres  15   0 21904 3720 2832 S  1.5  0.1   0:00.12 postgres: nagiosxi
14898 nagios    18   0     0    0    0 Z  1.5  0.0   0:00.02 [nagios] <defunct>
14973 nagios    18   0  4620 1212 1044 S  1.5  0.0   0:00.09 /bin/sh /usr/local/

top - 12:26:55 up 16 min,  1 user,  load average: 25.40, 17.54, 9.37
Tasks: 246 total,  33 running, 210 sleeping,   0 stopped,   3 zombie
Cpu(s): 11.1%us, 57.6%sy,  0.0%ni, 26.5%id,  4.3%wa,  0.0%hi,  0.5%si,  0.0%st
Mem:   3107560k total,  1245100k used,  1862460k free,   661584k buffers
Swap:  1048568k total,        0k used,  1048568k free,   244268k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
20914 nagios    25   0 10804 6656 1720 R 17.8  0.2   0:00.35 /usr/bin/perl -w /u
20912 nagios    25   0 10408 6316 1720 R 10.7  0.2   0:00.33 /usr/bin/perl -w /u
20812 nagios    25   0 11380 7400 1856 R  9.5  0.2   0:00.38 /usr/bin/perl -w /u
  632 root      11  -5     0    0    0 S  8.3  0.0   0:04.14 [kjournald]
21011 nagios    25   0  4620  964  844 R  7.1  0.0   0:00.06 /bin/sh /usr/local/
 3579 nagios    25   0 15832 3900  876 R  5.9  0.1   0:55.14 /usr/local/nagios/b
20946 nagios    25   0  4620 1104  972 S  4.7  0.0   0:00.06 /bin/sh /usr/local/
20973 nagios    25   0  1896  224  168 R  4.7  0.0   0:00.04 /usr/bin/perl -w /u
21023 nagios    25   0  3552  312  240 R  4.7  0.0   0:00.04 /usr/bin/perl -w /u
20965 root      15   0  2420 1008  696 R  3.6  0.0   0:00.08 top -b -c
21018 nagios    23   0  2112  360  300 R  2.4  0.0   0:00.02 awk {x=$3} END { pr
21026 nagios    23   0  2020  412  316 R  2.4  0.0   0:00.02 bc
 3126 mysql     15   0  148m  23m 4148 S  1.2  0.8   0:39.04 /usr/libexec/mysqld
20763 nagios    18   0  4620 1212 1044 S  1.2  0.0   0:00.08 /bin/sh /usr/local/
20771 nagios    18   0  4620 1208 1036 S  1.2  0.0   0:00.07 /bin/sh /usr/local/
20856 nagios    18   0  4620 1120  984 S  1.2  0.0   0:00.06 /bin/sh /usr/local/
20880 nagios    18   0  4620 1208 1036 S  1.2  0.0   0:00.07 /bin/sh /usr/local/
20970 nagios    18   0     0    0    0 Z  1.2  0.0   0:00.01 [nagios] <defunct>
20989 nagios    19   0  4620  488  320 S  1.2  0.0   0:00.01 /bin/sh /usr/local/
20991 nagios    18   0  4620  488  320 S  1.2  0.0   0:00.01 /bin/sh /usr/local/

tonyyarusso · Post by **tonyyarusso** » Tue Mar 01, 2011 1:05 pm

Err, shoot. You're cutting off at 80 characters. Can you use a terminal emulator that allows you to see the complete output?

FLCUISIT · Post by **FLCUISIT** » Tue Mar 01, 2011 1:09 pm

When I piped it to a text file, it appears to have cut it off. I am going to rerun the script and post the file.

FLCUISIT · Post by **FLCUISIT** » Tue Mar 01, 2011 1:22 pm

top - 13:09:40 up 58 min, 1 user, load average: 14.10, 19.90, 18.65
Tasks: 223 total, 13 running, 190 sleeping, 0 stopped, 20 zombie
Cpu(s): 14.0%us, 65.8%sy, 0.0%ni, 17.4%id, 2.1%wa, 0.1%hi, 0.7%si, 0.0%st
Mem: 3107560k total, 1726736k used, 1380824k free, 673692k buffers
Swap: 1048568k total, 0k used, 1048568k free, 699824k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31638 nagios 20 0 27864 9.9m 5356 R 55.1 0.3 0:00.50 /usr/bin/php /usr/local/nagiosxi/scripts/handle_nagioscore_event.php --handler-type=service --host=
29248 nagios 17 0 29536 14m 5888 S 9.2 0.5 0:01.17 /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php
3126 mysql 15 0 150m 30m 4220 S 5.5 1.0 5:46.61 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld
31735 root 15 0 2420 1000 696 R 3.7 0.0 0:00.07 top -b -c
31793 nagios 21 0 0 0 0 Z 3.7 0.0 0:00.02 [check_rrdtraf] <defunct>
29257 postgres 15 0 21904 4192 3228 S 1.8 0.1 0:00.16 postgres: nagiosxi nagiosxi 127.0.0.1(33341) idle
29259 postgres 15 0 21904 4028 3120 S 1.8 0.1 0:00.11 postgres: nagiosxi nagiosxi 127.0.0.1(33343) idle
29279 postgres 15 0 21904 4160 3216 S 1.8 0.1 0:00.12 postgres: nagiosxi nagiosxi 127.0.0.1(33346) idle
31060 nagios 18 0 0 0 0 Z 1.8 0.0 0:00.02 [nagios] <defunct>
31130 nagios 18 0 0 0 0 Z 1.8 0.0 0:00.02 [nagios] <defunct>
31136 nagios 18 0 0 0 0 Z 1.8 0.0 0:00.02 [nagios] <defunct>
31166 nagios 18 0 0 0 0 Z 1.8 0.0 0:00.02 [nagios] <defunct>
31184 nagios 18 0 4624 1216 1044 S 1.8 0.0 0:00.09 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.4.7_3.rrd -w 50,50 -c 80,8
31186 nagios 18 0 0 0 0 Z 1.8 0.0 0:00.02 [nagios] <defunct>
31264 nagios 19 0 4624 1212 1044 S 1.8 0.0 0:00.09 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.4.6_10.rrd -w 50,50 -c 80,
31446 nagios 18 0 4624 1216 1044 S 1.8 0.0 0:00.08 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.3.6_15.rrd -w 50,50 -c 80,
31508 nagios 18 0 4620 1208 1044 S 1.8 0.0 0:00.09 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.3.6_8.rrd -w 50,50 -c 80,8
31529 nagios 18 0 4620 1208 1036 S 1.8 0.0 0:00.08 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.20.4_25.rrd -w 50,50 -c 80
31532 nagios 18 0 0 0 0 Z 1.8 0.0 0:00.02 [nagios] <defunct>
31572 nagios 21 0 4620 1120 984 S 1.8 0.0 0:00.07 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.18.6_1.rrd -w 50,50 -c 80,

top - 13:10:59 up 1:00, 1 user, load average: 11.66, 18.00, 18.08
Tasks: 186 total, 7 running, 178 sleeping, 0 stopped, 1 zombie
Cpu(s): 13.9%us, 65.5%sy, 0.0%ni, 17.8%id, 2.1%wa, 0.1%hi, 0.6%si, 0.0%st
Mem: 3107560k total, 1710200k used, 1397360k free, 674072k buffers
Swap: 1048568k total, 0k used, 1048568k free, 700240k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3579 nagios 25 0 15836 3900 876 R 50.5 0.1 3:37.00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
3126 mysql 16 0 150m 30m 4220 S 18.7 1.0 5:49.61 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld
3959 nagios 25 0 4620 1200 1036 R 11.2 0.0 0:00.06 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.7.4_8.rrd -w 50,50 -c 80,8
3961 nagios 25 0 4620 1112 984 R 11.2 0.0 0:00.06 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.11.6_10.rrd -w 50,50 -c 80
3963 nagios 25 0 4620 1116 984 S 11.2 0.0 0:00.06 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.11.4_12.rrd -w 50,50 -c 80
3965 nagios 25 0 4620 1112 984 S 11.2 0.0 0:00.06 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.7.6_27.rrd -w 50,50 -c 80,
3970 nagios 25 0 4012 732 640 R 9.4 0.0 0:00.05 /usr/local/nagios/libexec/check_nt -H 192.168.3.5 -s @2(n3ptun3|m4rs)^ -p 12489 -v CPULOAD -l 5 85
3994 nagios 25 0 4012 628 548 S 7.5 0.0 0:00.04 /usr/local/nagios/libexec/check_nt -H 192.168.21.182 -s @2(n3ptun3|m4rs)^ -p 12489 -v COUNTER -l \P
3998 nagios 25 0 4012 628 548 S 7.5 0.0 0:00.04 /usr/local/nagios/libexec/check_nt -H 192.168.21.14 -s Castor_Pollux -p 12489 -v COUNTER -l \Paging
4001 nagios 25 0 4544 516 436 R 7.5 0.0 0:00.04 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.7.4_14.rrd -w 50,50 -c 80,
4002 nagios 25 0 0 0 0 Z 3.7 0.0 0:00.02 [basename] <defunct>
1160 postgres 20 0 21904 3684 2812 S 1.9 0.1 0:00.10 postgres: nagiosxi nagiosxi 127.0.0.1(33432) idle
1173 postgres 15 0 21904 3744 2856 S 1.9 0.1 0:00.10 postgres: nagiosxi nagiosxi 127.0.0.1(33435) idle
3517 nagios 15 0 7308 1568 912 S 1.9 0.1 0:10.82 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
3954 root 16 0 2420 948 696 R 1.9 0.0 0:00.05 top -b -c
3960 nagios 25 0 15840 3260 232 S 1.9 0.1 0:00.01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
3962 nagios 25 0 15840 3260 232 S 1.9 0.1 0:00.01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
3964 nagios 25 0 15840 3260 232 S 1.9 0.1 0:00.01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
3967 nagios 25 0 15840 3260 232 S 1.9 0.1 0:00.01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
3990 nagios 25 0 15840 3260 232 S 1.9 0.1 0:00.01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg

top - 13:12:11 up 1:01, 1 user, load average: 15.82, 17.97, 18.06
Tasks: 186 total, 3 running, 183 sleeping, 0 stopped, 0 zombie
Cpu(s): 14.0%us, 65.6%sy, 0.0%ni, 17.7%id, 2.0%wa, 0.1%hi, 0.6%si, 0.0%st
Mem: 3107560k total, 1718884k used, 1388676k free, 674312k buffers
Swap: 1048568k total, 0k used, 1048568k free, 700460k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
9634 nagios 20 0 4620 1208 1044 S 11.1 0.0 0:00.08 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.42.2_10130.rrd -w 50,50 -c
9045 nagios 16 0 29388 14m 5992 S 5.6 0.5 0:00.78 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
9632 nagios 19 0 4620 1272 1100 S 5.6 0.0 0:00.08 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.3.4_13.rrd -w 50,50 -c 80,
3344 apache 17 0 41260 18m 4212 S 3.7 0.6 0:38.30 /usr/sbin/httpd
9631 nagios 18 0 4620 1212 1044 S 3.7 0.0 0:00.08 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.20.4_16.rrd -w 50,50 -c 80
9698 nagios 18 0 0 0 0 R 3.7 0.0 0:00.02 [bc]
3336 apache 16 0 41308 18m 4248 S 1.9 0.6 0:36.97 /usr/sbin/httpd
9635 root 15 0 2420 968 708 R 1.9 0.0 0:00.06 top -b -c
9696 nagios 21 0 4620 488 320 S 1.9 0.0 0:00.01 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.20.4_16.rrd -w 50,50 -c 80
9699 nagios 22 0 4620 488 316 S 1.9 0.0 0:00.01 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.3.4_13.rrd -w 50,50 -c 80,
9701 nagios 23 0 1924 212 160 R 1.9 0.0 0:00.01 bc
1 root 15 0 2156 640 552 S 0.0 0.0 0:02.30 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:00.53 [migration/0]
3 root 34 19 0 0 0 S 0.0 0.0 0:00.02 [ksoftirqd/0]
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
5 root RT -5 0 0 0 S 0.0 0.0 0:00.58 [migration/1]
6 root 39 19 0 0 0 S 0.0 0.0 0:00.00 [ksoftirqd/1]
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/1]
8 root RT -5 0 0 0 S 0.0 0.0 0:00.57 [migration/2]
9 root 34 19 0 0 0 S 0.0 0.0 0:00.98 [ksoftirqd/2]

top - 13:13:27 up 1:02, 1 user, load average: 12.46, 16.66, 17.61
Tasks: 178 total, 1 running, 177 sleeping, 0 stopped, 0 zombie
Cpu(s): 13.9%us, 65.6%sy, 0.0%ni, 17.8%id, 2.0%wa, 0.1%hi, 0.6%si, 0.0%st
Mem: 3107560k total, 1718644k used, 1388916k free, 674532k buffers
Swap: 1048568k total, 0k used, 1048568k free, 700856k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
632 root 10 -5 0 0 0 S 1.9 0.0 0:20.95 [kjournald]
3579 nagios 16 0 15836 3904 876 S 1.9 0.1 3:49.85 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
1 root 15 0 2156 640 552 S 0.0 0.0 0:02.30 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:00.55 [migration/0]
3 root 35 19 0 0 0 S 0.0 0.0 0:00.02 [ksoftirqd/0]
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
5 root RT -5 0 0 0 S 0.0 0.0 0:00.60 [migration/1]
6 root 34 19 0 0 0 S 0.0 0.0 0:00.00 [ksoftirqd/1]
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/1]
8 root RT -5 0 0 0 S 0.0 0.0 0:00.59 [migration/2]
9 root 34 19 0 0 0 S 0.0 0.0 0:00.98 [ksoftirqd/2]
10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/2]
11 root RT -5 0 0 0 S 0.0 0.0 0:00.53 [migration/3]
12 root 34 19 0 0 0 S 0.0 0.0 0:00.00 [ksoftirqd/3]
13 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/3]
14 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 [events/0]
15 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 [events/1]
16 root 10 -5 0 0 0 S 0.0 0.0 0:00.03 [events/2]
17 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 [events/3]
18 root 17 -5 0 0 0 S 0.0 0.0 0:00.00 [khelper]

top - 13:14:40 up 1:03, 1 user, load average: 15.46, 16.36, 17.41
Tasks: 180 total, 5 running, 173 sleeping, 0 stopped, 2 zombie
Cpu(s): 13.9%us, 65.7%sy, 0.0%ni, 17.8%id, 1.9%wa, 0.1%hi, 0.6%si, 0.0%st
Mem: 3107560k total, 1721992k used, 1385568k free, 674740k buffers
Swap: 1048568k total, 0k used, 1048568k free, 703168k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3579 nagios 15 0 15840 3908 880 R 13.9 0.1 3:56.15 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
21573 nagios 18 0 4620 1216 1044 S 3.5 0.0 0:00.08 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.18.6_1.rrd -w 50,50 -c 80,
21811 root 15 0 2420 968 696 R 3.5 0.0 0:00.06 top -b -c
19291 nagios 15 0 29320 14m 5976 S 1.7 0.5 0:00.75 /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php
19293 nagios 16 0 29388 14m 5996 S 1.7 0.5 0:00.89 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
21392 nagios 18 0 0 0 0 Z 1.7 0.0 0:00.02 [nagios] <defunct>
21568 nagios 19 0 4624 1216 1044 S 1.7 0.0 0:00.08 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.20.4_25.rrd -w 50,50 -c 80
21899 nagios 22 0 4620 488 316 R 1.7 0.0 0:00.01 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.18.6_1.rrd -w 50,50 -c 80,
21901 nagios 20 0 4624 464 292 R 1.7 0.0 0:00.01 /bin/sh /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/192.168.20.4_25.rrd -w 50,50 -c 80
1 root 15 0 2156 640 552 S 0.0 0.0 0:02.30 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:00.57 [migration/0]
3 root 34 19 0 0 0 S 0.0 0.0 0:00.02 [ksoftirqd/0]
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
5 root RT -5 0 0 0 S 0.0 0.0 0:00.61 [migration/1]
6 root 34 19 0 0 0 S 0.0 0.0 0:00.00 [ksoftirqd/1]
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/1]
8 root RT -5 0 0 0 S 0.0 0.0 0:00.60 [migration/2]
9 root 34 19 0 0 0 S 0.0 0.0 0:00.98 [ksoftirqd/2]
10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/2]
11 root RT -5 0 0 0 S 0.0 0.0 0:00.54 [migration/3]

top - 13:15:56 up 1:05, 1 user, load average: 10.20, 14.75, 16.78
Tasks: 171 total, 1 running, 170 sleeping, 0 stopped, 0 zombie
Cpu(s): 13.9%us, 65.5%sy, 0.0%ni, 18.1%id, 1.9%wa, 0.1%hi, 0.6%si, 0.0%st
Mem: 3107560k total, 1718736k used, 1388824k free, 674976k buffers
Swap: 1048568k total, 0k used, 1048568k free, 701540k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 15 0 2156 640 552 S 0.0 0.0 0:02.30 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:00.58 [migration/0]
3 root 34 19 0 0 0 S 0.0 0.0 0:00.02 [ksoftirqd/0]
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
5 root RT -5 0 0 0 S 0.0 0.0 0:00.62 [migration/1]
6 root 34 19 0 0 0 S 0.0 0.0 0:00.00 [ksoftirqd/1]
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/1]
8 root RT -5 0 0 0 S 0.0 0.0 0:00.61 [migration/2]
9 root 34 19 0 0 0 S 0.0 0.0 0:00.98 [ksoftirqd/2]
10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/2]
11 root RT -5 0 0 0 S 0.0 0.0 0:00.56 [migration/3]
12 root 39 19 0 0 0 S 0.0 0.0 0:00.00 [ksoftirqd/3]
13 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/3]
14 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 [events/0]
15 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 [events/1]
16 root 10 -5 0 0 0 S 0.0 0.0 0:00.03 [events/2]
17 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 [events/3]
18 root 17 -5 0 0 0 S 0.0 0.0 0:00.00 [khelper]
19 root 16 -5 0 0 0 S 0.0 0.0 0:00.00 [kthread]
25 root 10 -5 0 0 0 S 0.0 0.0 0:00.52 [kblockd/0]

Post by **admin** » Fri Mar 04, 2011 7:12 am

The performance graphing engine (PNP) has a "features" in which it stops updated RRD files (e.g. performance graphs) if the system load exceeds a certain level. The default is a load of 10.0

You can change this by editing the following file:

Code: Select all

/usr/local/nagios/etc/pnp/npcd.cfg

And changing the following value to a higher number:

Code: Select all

load_threshold = 10.0

Once you do that, restart PNP with this command:

Code: Select all

/etc/init.d/npcd restart

Nagios Support Forum

Performance graphs stopped updating

Re: Performance graphs stopped updating

Re: Performance graphs stopped updating

Re: Performance graphs stopped updating

Re: Performance graphs stopped updating

Re: Performance graphs stopped updating

Re: Performance graphs stopped updating