Page 1 of 1

Mod_gearman with check_snmp --rate

Posted: Mon Nov 18, 2013 6:08 pm
by stideswell
Hello

I'd like to scale my Nagios better and use Nagios + gearman + PNP4Nagios (pnp_gearman_worker). This all works really well except for some service checks I run using check_snmp with the --rate option. The --rate option involves the plugin writing state information to /var/check_snmp on the local gearman_worker server and since the worker will potentially be different each time the service check runs this will cause at the very least inaccurate rate calculations. I wonder if anyone else is using mod_gearman with check_snmp --rate and has worked out a slick way to handle this issue?

Simon

Re: Mod_gearman with check_snmp --rate

Posted: Tue Nov 19, 2013 1:37 pm
by tmcdonald
Kinda hacky, but you could *possibly* symlink all your /var/check_snmp rate files to a central network location.

Re: Mod_gearman with check_snmp --rate

Posted: Tue Nov 19, 2013 6:10 pm
by stideswell
Yes, I thought of an NFS share or something but feel that, most probably, each worker server will still end up writing separately named files anyway or something and I wouldn't be any further forward. I suppose I need to look at the plugin source code to work out how it names the files and whether that is likely to be consistent across workers.

I managed to get this working by making a new service group for all such checks and a separate Gearman queue to process them ...
servicegroups=SNMP_Rate
... then I only have one worker processing this queue. It's not ideal because I cannot remove the worker server without messing up all of those checks. It would be really cool if the author of check_snmp would give the option of storing the state information in e.g. memcached instead, and ensured that the unique ID of the state file/key was consistent across worker servers then the solution would be totally flexible.

Re: Mod_gearman with check_snmp --rate

Posted: Wed Nov 20, 2013 3:04 pm
by sreinhardt
That would be a pretty good idea! Since check_snmp is done by the nagios plugin guys, you could submit a feature request to them! It might also be a good idea for them to consider it with core 4 remote workers.