Nagios server slowness

[gm_rajkumar]

Hi,
i noticed that the cpu load and io is pretty high in the nagios xi, i have recently updated in the system Nagios XI 2012R2.1. I have enclosed the screenshot for your reference. We are unable to access the system.

I have followed http://assets.nagios.com/downloads/nagi ... rmance.pdf, however it didnt worked..

Also, im getting this below error:
Error: While connecting to Database
Local host Connection to the database server failed by reason

top result----
Tasks: 678 total, 5 running, 672 sleeping, 0 stopped, 1 zombie
Cpu(s): 42.8%us, 8.0%sy, 0.0%ni, 1.6%id, 43.5%wa, 0.7%hi, 3.5%si, 0.0%st
Mem: 3107040k total, 2963456k used, 143584k free, 4792k buffers
Swap: 2031608k total, 351952k used, 1679656k free, 783680k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22812 apache 18 0 58596 36m 5344 R 92.5 1.2 7:43.60 httpd
26686 root 16 0 1472m 919m 25m R 43.6 30.3 1809:49 firefox
6161 postgres 16 0 23168 11m 9m R 9.6 0.4 0:19.22 postmaster
30738 apache 17 0 58048 36m 5328 R 6.2 1.2 5:19.39 httpd
31466 postgres 16 0 23180 12m 10m D 5.8 0.4 0:19.81 postmaster
26137 postgres 16 0 23224 11m 9.9m S 4.3 0.4 0:08.65 postmaster
32480 postgres 16 0 23072 11m 9.8m D 3.8 0.4 0:02.44 postmaster
4824 nagios 25 0 6540 1988 1288 S 3.4 0.1 0:00.07 snmpwalk
1517 postgres 16 0 23088 11m 9.8m D 2.4 0.4 0:04.32 postmaster
23 root 10 -5 0 0 0 S 1.9 0.0 989:56.93 kblockd/2
1500 postgres 17 0 22980 12m 10m D 1.9 0.4 0:02.66 postmaster
14743 postgres 17 0 23012 11m 10m D 1.9 0.4 0:17.84 postmaster
3498 root 15 0 49796 7720 5904 S 1.4 0.2 45:20.17 vino-server
4511 root 16 0 63324 14m 9560 S 1.4 0.5 0:02.52 gnome-terminal
28623 postgres 16 0 23012 11m 9m D 1.4 0.4 0:08.05 postmaster
1526 postgres 16 0 22880 11m 9768 S 1.0 0.4 0:01.26 postmaster
24 root 10 -5 0 0 0 S 0.5 0.0 20:57.32 kblockd/3
[root@monitoring libexec]#



Kindly let me know if you've any option to improve speed/performance of the system.

Regards,
Raj.

[slansing]

Your I/O is quite high, lets start by running the following:

Code: Select all

service ndo2db stop

killall -9 ndo2db

service ndo2db start

service crond restart

Let us know the output of each command.

[gm_rajkumar]

Please find the screenshot attached, let me know if you need further..
thanks!!

[sreinhardt]

Are you still having the services failing and slowness from your first post? I could see this happening for a bit, as ndo2db did not seem to be started and surely has to process some performance data now.

[gm_rajkumar]

Still having the same problem. I hereby attaching the output of top for your reference.

[root@monitoring libexec]# top

top - 09:36:03 up 36 days, 22:06, 2 users, load average: 31.19, 25.52, 22.12
Tasks: 631 total, 1 running, 629 sleeping, 0 stopped, 1 zombie
Cpu(s): 21.0%us, 6.6%sy, 0.0%ni, 0.4%id, 68.9%wa, 0.3%hi, 2.7%si, 0.0%st
Mem: 3107040k total, 2865084k used, 241956k free, 5988k buffers
Swap: 2031608k total, 992352k used, 1039256k free, 956728k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
26686 root 16 0 1455m 825m 16m S 43.4 27.2 2895:35 firefox
30130 postgres 17 0 23172 12m 10m D 7.9 0.4 0:31.62 postmaster
24986 postgres 18 0 23012 11m 10m D 7.6 0.4 0:20.85 postmaster
3564 postgres 16 0 23168 12m 10m S 7.0 0.4 0:46.23 postmaster
2116 postgres 17 0 22880 10m 8584 D 6.0 0.3 0:02.34 postmaster
15627 postgres 15 0 23168 12m 10m D 6.0 0.4 0:44.23 postmaster
7747 postgres 18 0 23012 11m 9m D 5.3 0.4 0:12.66 postmaster
1094 postgres 17 0 23012 11m 10m D 4.6 0.4 0:40.79 postmaster
23 root 10 -5 0 0 0 S 3.0 0.0 1035:32 kblockd/2
2523 postgres 15 0 23168 12m 10m D 2.3 0.4 0:51.08 postmaster
5157 postgres 15 0 23168 12m 10m D 1.7 0.4 1:00.97 postmaster
7139 postgres 16 0 23168 12m 10m S 1.7 0.4 1:00.30 postmaster
22537 postgres 16 0 23168 12m 10m S 1.7 0.4 1:01.67 postmaster
2161 nagios 15 0 7364 3440 1704 D 1.0 0.1 0:00.19 process_perfdat
3498 root 15 0 49796 6260 5180 S 1.0 0.2 61:44.92 vino-server
21571 postgres 16 0 23048 11m 10m S 1.0 0.4 0:10.73 postmaster
1741 root 16 0 24132 17m 2776 D 0.7 0.6 0:01.94 mrtg
You have new mail in /var/spool/mail/root

[abrist]

You still have massive i/o wait. Do you still have red marks on the "system status" page? If so, which ones?

[gm_rajkumar]

Hi,

I have enclosed the screenshot for your reference, still this issue not resolved yet. I would like to know the progress, the system is quite slow and we would require to have someone intervene to provide us the solution at the earlier.

Thanks & Regards,
Rah.

[abrist]

Looks like crond may not be runnning:

Code: Select all

service crond status
service crond stop
killall crond
service crond start

[gm_rajkumar]

Hi,

Please find the results below for your reference.

You have new mail in /var/spool/mail/root
[root@monitoring ~]# service crond status
crond (pid 21615) is running...
[root@monitoring ~]# service crond stop
Stopping crond: [ OK ]
[root@monitoring ~]# killall crond
[root@monitoring ~]# ps -ef | grep crond
root 22951 4551 0 06:51 pts/1 00:00:00 grep crond
[root@monitoring ~]# service crond start
Starting crond: [ OK ]

[abrist]

Do you still have a number of red dots on the process info page? What is the output of:

Code: Select all

df -i
df -h
service postgresql restart
service mysqld restart