CPU Load spike every 7 hours

[BanditBBS]

I don't want to jinx myself, but I seem stable.

[lmiltchev]

BanditBBS, what is the number of hosts/services on this server, and what is the number of workers that you set up in the nagios.cfg?

[BanditBBS]

BanditBBS, what is the number of hosts/services on this server, and what is the number of workers that you set up in the nagios.cfg?

I jinxed myself....bah. Current load is 11. Same time every day, so I need to investigate on my infrastructure side as well. 18 hosts and 237 services and I did 6 workers. As I said though, I am still having a load issue now

EDIT: Deep diving on our end and not finding any reason for the spikes. Mine are weird on this server, the spikes happen once daily on it starting between 1:30 and 2pm EST and lasting 2-3 hours. Backups run at night and no other server admin is complaining about slowness.

[belvdr]

I posted in another thread but will repeat here too:

FWIW, I'm running:

Code: Select all

Guest:     CentOS 6.5
Host:      Hyper-V 2012
Spec:      4 vCPU 12GB RAM
Nagios:    2014 R1.2

I'm not having any issues. If you need me to look at something on my box, let me know.

[snapon_admin]

Saw a couple threads about this and figured I'd post as well since I'm pretty sure this issue isn't affecting us either:

Code: Select all

VMWare virtual server
CentOS release 6.4 (Final)
8 core CPU
8 GB RAM
2014R1.2
No gearman (haven't messed with nagios.cfg to add the check_workers line)
1015 hosts
17535 services

Our environment:

environment.png

Last months load:

load.png

Last months CPU % usage:

cpu.png

Note: Our load and cpu usage checks run every minute.

Let me know if you'd like to see anything else. We haven't seen this issue in our environment, and have actually seen a significant improvement in performance from 2012 to 2014.

[WillemDH]

Well I never said general performance was bad since 2014, it is definitely better then 2012. I would just have liked a description for the sudden load spikes, which in my case are not disrupting the system for now.

Grtz

Willem

[BanditBBS]

It has to be something in the VM environment. Running some tests here at home as well and the same extremely high load happens on one of my VM hosts but not the other. They are identical hosts too, both HP DL360G5. I'm going to have to ask for it to get moved at work to a different host and see if the issue goes away.

chart (1).jpeg

[WillemDH]

Well we have Fujitsu blades, other machines on same esx server would have same symptoms. And why isn't this behaviour seen before the upgrade....?

[abrist]

Well we have Fujitsu blades, other machines on same esx server would have same symptoms. And why isn't this behaviour seen before the upgrade....?

Odd. It could be changes in the Core 4 scheduler functions. Our core devs are working on it, but as the gap of time between issues is so large, it will take some time to hunt down the issue.

[SDK]

Well we have Fujitsu blades, other machines on same esx server would have same symptoms. And why isn't this behaviour seen before the upgrade....?

Odd. It could be changes in the Core 4 scheduler functions. Our core devs are working on it, but as the gap of time between issues is so large, it will take some time to hunt down the issue.

I also think it's somewhere in the scheduling functions, at least in the core, because when i restart the core daemon the load drops back to normal