check_snmp & net-snmp extend OID

[gehogan3]

I am trying to utilize the ability of net-snmp extended MIB to watch the status of some home grown services. I can get "snmpget" to work perfectly, and check_snmp:

./check_snmp -H ***.****.***.*** -o NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\"homeGrownCheck\" -w 5 -c 10 -C public
SNMP OK - 3 | 'NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."homeGrownCheck"'=0;5;10;

However, I cannot get it to work within NagiosXI. I build the check command properly, and test it, but it says that it is an invalid OID. Now, if I translate it to the full dotted numeric OID....and use that in the NagiosXI check, it works fine. The only think I can come up with is the quoting. Apparently, it must be part of the snmpget call, but somewhere along the way, that's getting screwed up. I have tried every possible method of quoting, double quoting, escaping, and double escaping that I can think of. Having the check as the full OID, is ugly and makes the graphs pretty worthless in a dashboard.

$ snmptranslate -On NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\"homeGrownCheck\"
iso.3.6.1.4.1.8072.1.3.2.3.1.1.19.97.115.116.101.114.105.115.107.65.99.116.105.118.101.67.97.108.108.115

I've tried check_snmp_extend.sh, but it didn't seem to do everything I need. (i.e. it just took host and extension name, no alert levels and didn't return data for performance graphing)

I am sure that it's a PICNIC problem, but for the life of me I cannot figure this one out.
Any help would be greatly appreciated.

Thanks in advance,
Emmett

[Box293]

I've not done this with SNMP before but I do think that in XI CCM the \"homeGrownCheck\" needs to be \\"homeGrownCheck\\"

If that doesn't work, I would like to test it here in my lab. Do you have some steps to setup snmp home grown services on a linux box so I can do check_snmp tests.

[gehogan3]

Unfortunately, escaping the backslashes just results in even more escaped backslashes...this is the result of testing with \\"homeGrownCheck\\" (via the CCM GUI)...

COMMAND: /usr/local/nagios/libexec/check_snmp -H ***.***.***.*** -o NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\\\\"homeGrownCheck\\\\" -w 40 -c 50 -C public
OUTPUT: External command error: NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\\homeGrownCheck\\: Unknown Object Identifier (Index out of range: \\homeGrownCheck\\ (nsExtendToken))

Here is just a single slash to escape the quotes in CCM:

COMMAND: /usr/local/nagios/libexec/check_snmp -H ***.***.***.*** -o NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\\"homeGrownCheck\\" -w 40 -c 50 -C public
OUTPUT: External command error: NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\homeGrownCheck\: Unknown Object Identifier (Index out of range: \homeGrownCheck\ (nsExtendToken))

It looks to me like the quotes are not getting passed through at all...even when escaped. (I even tried using single and double quotes to quote the quoted string!)

To extend SNMP (using net-snmp on a centos box), you can just add this to /etc/snmp/snmpd.conf on host 111.222.333.444 (the machine you want to monitor):

Code: Select all

extend homeGrownCheck "/bin/date +%N"

(This is obviously just a test...something that returns a number which can be checked...you can replace it with anything "echo 2" works just fine.)

Restart, snmpd and now the following commands should work:

Code: Select all

/usr/local/nagios/libexec//check_snmp -H 111.222.333.444 -o NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\"homeGrownCheck\" -w 5 -c 10 -C public
SNMP CRITICAL - *414173913* | 'NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."homeGrownCheck"'=414173913;5;10;

Code: Select all

$ snmpget -v1 -c public 10.150.20.161 NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\"homeGrownCheck\"
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."homeGrownCheck" = STRING: 186053747

On the manchine being monitored:

Code: Select all

$ snmptranslate -On NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\"homeGrownCheck\"
.1.3.6.1.4.1.8072.1.3.2.3.1.1.14.104.111.109.101.71.114.111.119.110.67.104.101.99.107

Yet...the "test" from within the GUI of CCM fails as mentioned at the top.

Thanks for your help!

-Emmett

[gehogan3]

Well...I found a workaround.

If I use the full, numeric OID it works, then I just set the label via "-l homeGrownCheck" in the check_snmp command and it works fine...and the label on the perf charts is readable.

My only worry is that the numeric OID *might* not be consistent across multiple machines.

I know this is probably beyond the scope of these forums, but does anyone know how net-snmp assigns the "extended OIDs"?

For example if I use this in my snmpd.conf file:

extend homeGrownCheck /usr/local/bin/check_script.sh

$ snmptranslate -On NET-SNMP-EXTEND-MIB::nsExtendOutput1Line.\"homeGrownCheck\"
.1.3.6.1.4.1.8072.1.3.2.3.1.1.19.97.115.116.101.114.105.115.107.65.99.116.105.118.101.67.97.108.108.115

Will "homeGrownCheck" *ALWAYS* translate to the same numeric OID or does it depend on whether or not there are any other extended SNMP checks and their order in the config file?

[gehogan3]

Well...following up my own messages might be considered bad form to some, but I thought I should share my findings.

I asked on the net-snmp IRC channel and found out that net-snmp uses the name I give the extension to generate the numeric OID, so as long as I am consistent in my naming, the numeric OIDs will remain consistent across my systems.

Just in case anyone else was wondering.

[Box293]

OK So I tried to replicate your issue but for one reason or another I could not get check_snmp to work (it just timed out) however an snmpwalk worked fine (proving I had correctly configured snmpd on the remote box).

Regardless I think I know the source of your problem.

Unfortunately, escaping the backslashes just results in even more escaped backslashes...this is the result of testing with \\"homeGrownCheck\\" (via the CCM GUI)...

The problem is using the "Test Check Command" button in CCM. Due to some issues with how PHP escapes characters the "Test Check Command" does not work in these situations and should be ignored.

So for all further testing of this service you need to:

Make the changes to the service
Save the Service
Apply Confguration
Go back to the home screen and find the Service
When viewing the Service Status Details page click the Schedule a forced immediate check link

This is why you get extra escapes etc.

I do think that in XI CCM the \"homeGrownCheck\" needs to be \\"homeGrownCheck\\"

I suspect this would work if you tried it without using the test check command button.

then I just set the label via "-l homeGrownCheck" in the check_snmp command and it works fine...and the label on the perf charts is readable.

This is a great solution to your problem, so perhaps you're all good anyway