Nagios server memory use

[rkennedy]

Got it.

Are you able to push for a mod_gearman update then (with time)?

Let us know the result, and if it helps with the changes you made.

[gormank]

I can do that, but the thing that makes me nervous is that I see gearman running, but I don't see it installed.

# rpm -qa | grep -i gear
# yum list | grep -i gearman
# yum list installed | grep -i gearman
# yum list | wc -l
12036

I created a test VM w/ the same RHEL version as my live Nagios and and installed Nagios. It also has no output when running rpm -qa and yum list so I guess its normal and I'm worrying about nothing...

I'll do the german update on the test system to verify.

The servers were restarted about 36 hours ago. Right now memory use reported by htop is 9G on one live system and 7G on the other. 1G on the failover servers that aren't running any checks.
Nagios and top report 11G and 10G used on the primary servers.

[rkennedy]

It seems to be leaking, but at this rate you should be good for a month with the leak!

I'll ask the developers about the package issue right now. Let us know how it works on your test system.

[rkennedy]

I spoke with the developers, and here's the response.

Since it's not listed in yum or rpm then make sure to turn the old gearman explicitly off, and then install the new version.

[gormank]

So turn off means stop them as shown?

service gearmand stop
service mod_gearman_worker stop

These are the only gearman services listed by chkconfig...

[rkennedy]

That is part of it, yes, turn them off, and then the second part is to disable them from startup. Once that is done you should be fine to proceed with @bheden's instructions.

[gormank]

Just FYI...
Memory use is at 35G in one server and 29G in the other. Adding RAM isn't helping.

[bheden]

I rewrote the ModGearman install script to install/upgrade automatically and copy the necessary configuration files over.

This is still in testing, but has passed all of our internal tests so far.

If you'd like to give it a go (with the caveat of supplying feedback, of course), the URL is:

http://assets.nagios.com/downloads/nagi ... Install.sh

On your server:

Code: Select all

wget http://assets.nagios.com/downloads/nagiosxi/scripts/ModGearmanInstall.sh
chmod +x ModGearmanInstall.sh
./ModGearmanInstall.sh --server --upgrade

And then follow the prompts. This *should* remove (or ignore) all gearman installs and install the new ones and set them up to be used by Nagios.

It will prompt you for configuration file locations - make sure they are correct before continuing.

Having a good known backup before hand is a good idea, too.

[gormank]

Can I disable Nagios use of gearman by commenting the broker_module line in nagios.cfg and stopping gearmand, and mod_gearman_worker?
My systems aren't that big with ~160 hosts and ~1000 services.

[bheden]

Disabling the broker_module line in nagios.cfg and restarting the nagios should do it.

Stopping gearmand or mod_gearman_worker isn't necessary, but wouldn't hurt either.