Mod_gearman with check_snmp --rate

[stideswell]

Hello

I'd like to scale my Nagios better and use Nagios + gearman + PNP4Nagios (pnp_gearman_worker). This all works really well except for some service checks I run using check_snmp with the --rate option. The --rate option involves the plugin writing state information to /var/check_snmp on the local gearman_worker server and since the worker will potentially be different each time the service check runs this will cause at the very least inaccurate rate calculations. I wonder if anyone else is using mod_gearman with check_snmp --rate and has worked out a slick way to handle this issue?

Simon

[tmcdonald]

Kinda hacky, but you could *possibly* symlink all your /var/check_snmp rate files to a central network location.

[stideswell]

Yes, I thought of an NFS share or something but feel that, most probably, each worker server will still end up writing separately named files anyway or something and I wouldn't be any further forward. I suppose I need to look at the plugin source code to work out how it names the files and whether that is likely to be consistent across workers.

I managed to get this working by making a new service group for all such checks and a separate Gearman queue to process them ...
servicegroups=SNMP_Rate
... then I only have one worker processing this queue. It's not ideal because I cannot remove the worker server without messing up all of those checks. It would be really cool if the author of check_snmp would give the option of storing the state information in e.g. memcached instead, and ensured that the unique ID of the state file/key was consistent across worker servers then the solution would be totally flexible.

[sreinhardt]

That would be a pretty good idea! Since check_snmp is done by the nagios plugin guys, you could submit a feature request to them! It might also be a good idea for them to consider it with core 4 remote workers.