Page 1 of 1

High load average on XI

Posted: Wed Sep 19, 2012 1:59 am
by cikoski
Hi,

i have a big problem with my nagios xi server, the load average is too much high: CRITICAL - load average: 40.75, 39.76, 39.11

how can i try to decrease it?

Thanks !

Re: High load average on XI

Posted: Wed Sep 19, 2012 9:12 am
by mguthrie
Usually when it spikes that high there is some sort of database corruption. Can you post the output from running "top"?

http://assets.nagios.com/downloads/nagi ... tabase.pdf

Re: High load average on XI

Posted: Wed Sep 19, 2012 9:14 am
by cikoski
mguthrie wrote:Usually when it spikes that high there is some sort of database corruption. Can you post the output from running "top"?

http://assets.nagios.com/downloads/nagi ... tabase.pdf
I already o the repairing database action, it's work fine and it repair all the tables, it take me 1h for do that.

I have 3000 services and 65 hosts..

Re: High load average on XI

Posted: Wed Sep 19, 2012 9:19 am
by scottwilkerson
How many CPU's does this server have? Did the database repair drop the load?

Re: High load average on XI

Posted: Wed Sep 19, 2012 9:26 am
by cikoski
scottwilkerson wrote:How many CPU's does this server have? Did the database repair drop the load?
I have a VM CPU 2 cores and 4GB of RAM, the repair don^t drop the load :D

Re: High load average on XI

Posted: Wed Sep 19, 2012 9:50 am
by scottwilkerson
I think you are going to need to do a bit of investigating as to which processes are using the CPU on your system.

You are running with about 1/2 the resources recommended for that many hosts/services if you are checking them on a 5 minute interval
http://assets.nagios.com/downloads/nagi ... ements.pdf

If you would like you can PM me and I would be willing to do a short remote session to see if we can help uncover what is causing the load.

Re: High load average on XI

Posted: Thu Sep 20, 2012 2:14 am
by cikoski
scottwilkerson wrote:I think you are going to need to do a bit of investigating as to which processes are using the CPU on your system.

You are running with about 1/2 the resources recommended for that many hosts/services if you are checking them on a 5 minute interval
http://assets.nagios.com/downloads/nagi ... ements.pdf

If you would like you can PM me and I would be willing to do a short remote session to see if we can help uncover what is causing the load.
Ok, thanks so much for the support, one ask.. How i can watch which processes are using the CPU in my system?

Thanks!

Re: High load average on XI

Posted: Thu Sep 20, 2012 9:21 am
by mguthrie
Run the following command:

Code: Select all

top
Then press 'shift+f', and then 'l' to sort by CPU time. Go ahead and post a screenshot of what you're seeing.

I'm wondering if there's still corruption in the mysql DB even after the repair run. Try the following:

Code: Select all

service mysqld stop
myisamchk -r -f /var/lib/mysql/nagios/*.MYI
service mysqld start



And then lets also make sure postgres is in good shape:

Code: Select all

psql nagiosxi nagiosxi
vacuum;
vacuum analyze;
vaccum full;
\q

psql postgres postgres
vacuum;
vacuum analyze;
vaccum full;
\q

Re: High load average on XI

Posted: Fri Sep 21, 2012 2:09 am
by cikoski
mguthrie wrote:Run the following command:

Code: Select all

top
Then press 'shift+f', and then 'l' to sort by CPU time. Go ahead and post a screenshot of what you're seeing.

I'm wondering if there's still corruption in the mysql DB even after the repair run. Try the following:

Code: Select all

service mysqld stop
myisamchk -r -f /var/lib/mysql/nagios/*.MYI
service mysqld start



And then lets also make sure postgres is in good shape:

Code: Select all

psql nagiosxi nagiosxi
vacuum;
vacuum analyze;
vaccum full;
\q

psql postgres postgres
vacuum;
vacuum analyze;
vaccum full;
\q
Thanks so much for the answers, now i take the screenshot..

Re: High load average on XI

Posted: Fri Sep 21, 2012 8:55 am
by scottwilkerson
Actually this looks like it is sorted by TIME
Run again and use SHIFT < or > to make the CPU the sorted field