Monitoring Engine Process 15 minute delayed start?

[ockmeyer]

I haven't waited the full 15 minutes, but after a few minutes it still shows the Monitoring Engine Process "not running".

[mguthrie]

Would you be interested in a remote session to take a look at this, maybe tomorrow? If so let me know and we'll discuss the details in a PM.

[ockmeyer]

sure

[mguthrie]

Details sent over PM.

[ockmeyer]

Mike worked with me on this and we concluded that the retention.dat file was too large. Deleting it and letting it regenerate seems to have fixed the problem.

I verified it with a second server having an identical problem.

Thanks for all the help, Mike!

[mguthrie]

Thanks for updating on this with the solution as well. If you ever see this issue come up again, can we have you send us the oversized retention.dat file so we can try and better trace why that file got so large?

[ockmeyer]

Will do.

[mguthrie]

After looking through the file, I'm noticing that there's almost 90000 host comments all related to:
"This host has been scheduled for fixed downtime..."

Anything goofy showing up in your downtime or recurring downtime pages?

[ockmeyer]

I created three recurring downtime schedules the Friday before all of this started. Each one corresponds to the three maintenance windows we have, and they are based on hostgroups. People were complaining about getting alerts when they were doing maintenance and I thought this would eliminate that, but it appears to have created a bigger problem.

With hundreds of hosts in each hostgroup, is there a better way of configuring this without the side effect of such a large file?

[mguthrie]

Unless you've got 30k hosts+services, then there's something goofy going on with that scheduler. I'm wondering if LOTS of duplicate downtime schedules are being created somehow. Even if you've got 10k checks total, there shouldn't be that many comments in the file.

You did set this up correctly, we'll do some digging on the possibility of duplicate schedules getting created and see what we can find out...