Page 4 of 5
Re: CPU Load spike every 7 hours
Posted: Tue Jul 01, 2014 12:34 pm
by BanditBBS
I don't want to jinx myself, but I seem stable.
Re: CPU Load spike every 7 hours
Posted: Tue Jul 01, 2014 12:42 pm
by lmiltchev
BanditBBS, what is the number of hosts/services on this server, and what is the number of workers that you set up in the nagios.cfg?
Re: CPU Load spike every 7 hours
Posted: Tue Jul 01, 2014 12:48 pm
by BanditBBS
lmiltchev wrote:BanditBBS, what is the number of hosts/services on this server, and what is the number of workers that you set up in the nagios.cfg?
I jinxed myself....bah. Current load is 11. Same time every day, so I need to investigate on my infrastructure side as well. 18 hosts and 237 services and I did 6 workers. As I said though, I am still having a load issue now
EDIT: Deep diving on our end and not finding any reason for the spikes. Mine are weird on this server, the spikes happen once daily on it starting between 1:30 and 2pm EST and lasting 2-3 hours. Backups run at night and no other server admin is complaining about slowness.
Re: CPU Load spike every 7 hours
Posted: Tue Jul 01, 2014 1:54 pm
by belvdr
I posted in another thread but will repeat here too:
FWIW, I'm running:
Code: Select all
Guest: CentOS 6.5
Host: Hyper-V 2012
Spec: 4 vCPU 12GB RAM
Nagios: 2014 R1.2
I'm not having any issues. If you need me to look at something on my box, let me know.
Re: CPU Load spike every 7 hours
Posted: Tue Jul 01, 2014 2:58 pm
by snapon_admin
Saw a couple threads about this and figured I'd post as well since I'm pretty sure this issue isn't affecting us either:
Code: Select all
VMWare virtual server
CentOS release 6.4 (Final)
8 core CPU
8 GB RAM
2014R1.2
No gearman (haven't messed with nagios.cfg to add the check_workers line)
1015 hosts
17535 services
Our environment:
environment.png
Last months load:
load.png
Last months CPU % usage:
cpu.png
Note: Our load and cpu usage checks run every minute.
Let me know if you'd like to see anything else. We haven't seen this issue in our environment, and have actually seen a significant improvement in performance from 2012 to 2014.
Re: CPU Load spike every 7 hours
Posted: Tue Jul 01, 2014 3:38 pm
by WillemDH
Well I never said general performance was bad since 2014, it is definitely better then 2012. I would just have liked a description for the sudden load spikes, which in my case are not disrupting the system for now.
Grtz
Willem
Re: CPU Load spike every 7 hours
Posted: Tue Jul 01, 2014 3:55 pm
by BanditBBS
It has to be something in the VM environment. Running some tests here at home as well and the same extremely high load happens on one of my VM hosts but not the other. They are identical hosts too, both HP DL360G5. I'm going to have to ask for it to get moved at work to a different host and see if the issue goes away.
chart (1).jpeg
Re: CPU Load spike every 7 hours
Posted: Tue Jul 01, 2014 3:58 pm
by WillemDH
Well we have Fujitsu blades, other machines on same esx server would have same symptoms. And why isn't this behaviour seen before the upgrade....?
Re: CPU Load spike every 7 hours
Posted: Wed Jul 02, 2014 3:45 pm
by abrist
WillemDH wrote:Well we have Fujitsu blades, other machines on same esx server would have same symptoms. And why isn't this behaviour seen before the upgrade....?
Odd. It could be changes in the Core 4 scheduler functions. Our core devs are working on it, but as the gap of time between issues is so large, it will take some time to hunt down the issue.
Re: CPU Load spike every 7 hours
Posted: Thu Jul 03, 2014 3:55 am
by SDK
abrist wrote:WillemDH wrote:Well we have Fujitsu blades, other machines on same esx server would have same symptoms. And why isn't this behaviour seen before the upgrade....?
Odd. It could be changes in the Core 4 scheduler functions. Our core devs are working on it, but as the gap of time between issues is so large, it will take some time to hunt down the issue.
I also think it's somewhere in the scheduling functions, at least in the core, because when i restart the core daemon the load drops back to normal