Nagios Support Forum

Posted: **Thu Apr 14, 2016 5:45 pm**

I edited my previous post so I'm now out of order... See above. :)

Posted: **Thu Apr 14, 2016 6:08 pm**

gormank wrote:Yep, it created a host named: UDP/IPv6: [2001:4888:a03c00:190

We suspect it's a combination of how snmptrapd stores the address in the spooled trap along with how snmptt interacts with it. Stay tuned.

Posted: **Thu Apr 14, 2016 6:12 pm**

Just an FYI...
After adding the odd new wizard created host to the SNMP trap service and generating a test trap, still no alert.

Posted: **Fri Apr 15, 2016 12:12 am**

I haven't had much time to delve into the real cause of the problem, however this is a quick fix that should get you working.

Edit the file /usr/local/bin/snmptraphandling.py
This is line 26:

Code: Select all

host = sys.argv[1]

Change it so it looks like this:

Code: Select all

                host = sys.argv[1]
                if host.startswith("UDP/IPv6"):
                        host = host.partition('[')[-1].rpartition(']')[0]

Then do a simple test like this:

Code: Select all

/usr/local/bin/snmptraphandling.py "UDP/IPv6: [2001:4888:a03:311f:c0:a:0:190]:32880" "SNMP Traps" "Normal" "1439269658" "" "My Test"

That should appear under Unconfigured Objects as 2001:4888:a03

c0

0:190

Let us know how that goes.
NOTE: with the snmptraphandling.py file, this is Python. I'm no Python expert but that indentation you see are tabs, I believe this is how python needs the code formatted or it will have a temper tantrum.

Edit:
I just replied to your post to the net-snmp mailing list https://sourceforge.net/p/net-snmp/mail ... nth=201604.

I believe there is a bug in net-snmp and my lengthy reply outlines it and how to replicate it. Unfortunately I do not posses the skills required to fix the issue however the workaround above should be all you need.

Posted: **Fri Apr 15, 2016 2:25 pm**

I made the change in snmptraphandling.py as advised. Now sending test traps creates the alert below as desired. Running the script as shown above results in nothing.

Code: Select all

Generic trap (11003): HP Onboard Administrator Test Trap sent from enclosure: TXSLM2MCHP7004 / sysName.0 (OCTETSTR):TXSLM2MCHP7004-OA cpqHoTrapFlags (INTEGER):1 cpqHoGenericData (OCTETSTR):HP Onboard Administrator Test Trap sent from enclosure: TXSLM2MCHP7004

Code: Select all

# diff /usr/local/bin/snmptraphandling.py /usr/local/bin/snmptraphandling.py.org
27,28d26
<               if host.startswith("UDP/IPv6"):
<                       host = host.partition('[')[-1].rpartition(']')[0]

Here are a pair of test traps from blade chassis, one w/ a v6 address and the other a v4 address just for grins. To me it looks like snmptt is not stripping the extra info off the address.

Code: Select all

2016-04-15 19:19:22 0.0.0.0(via UDP/IPv6: [2001:4888:a03:311f:c0:a:0:190]:32880) TRAP, SNMP v1, community sp1der
        CPQHOST-MIB::compaq Enterprise Specific Trap (CPQHOST-MIB::cpqHo2GenericTrap) Uptime: 3 days, 2:41:31.48
        SNMPv2-MIB::sysName.0 = STRING: TXSLM2MCHP7004-OA       CPQHOST-MIB::cpqHoTrapFlags = INTEGER: 1        CPQHOST-MIB::cpqHoGenericData = STRING: "HP Onboard Administrator Test Trap sent from enclosure: TXSLM2MCHP7004"

Code: Select all

2016-04-15 19:20:02 10.133.133.100(via UDP: [10.133.133.100]:1167->[10.133.133.74]) TRAP, SNMP v1, community sp1der
        CPQHOST-MIB::compaq Enterprise Specific Trap (CPQHOST-MIB::cpqHo2GenericTrap) Uptime: 129 days, 8:51:25.25
        SNMPv2-MIB::sysName.0 = STRING: OA-7446A0F9190B CPQHOST-MIB::cpqHoTrapFlags = INTEGER: 0        CPQHOST-MIB::cpqHoGenericData = STRING: "HP Onboard Administrator Test Trap sent from enclosure: TXLSM2MCHP7001"

And the snmptt log info...

Code: Select all

Fri Apr 15 19:19:23 2016 .1.3.6.1.4.1.232.0.11003 MAJOR "Status Events" UDP/IPv6: [2001:4888:a03:311f:c0:a:0:190]:32880 - Generic trap (11003): HP Onboard Administrator Test Trap sent from enclosure: TXSLM2MCHP7004

Code: Select all

Fri Apr 15 19:20:02 2016 .1.3.6.1.4.1.232.0.11003 MAJOR "Status Events" 10.133.133.100 - Generic trap (11003): HP Onboard Administrator Test Trap sent from enclosure: TXLSM2MCHP7001

Posted: **Fri Apr 15, 2016 5:55 pm**

I noticed another v6 feature that Nagios doesn't handle. I know, by now you're ready to strangle me...

The following addresses are the same IPv6. They're not the same to Nagios so a trap arriving as the first on a host defined as the second will be unknown.

2001:4888:a03:311f:c0:a:0:196
2001:4888:a03:311f:c0:a::196

Posted: **Mon Apr 18, 2016 3:28 am**

gormank wrote:Here are a pair of test traps from blade chassis, one w/ a v6 address and the other a v4 address just for grins. To me it looks like snmptt is not stripping the extra info off the address.

The problem is not with snmptt, it has to do with the net-snmp library and snmptrapd.

I've created a bug report about this here:
https://sourceforge.net/p/net-snmp/bugs/2704/

In short, the second line of a IPv6 trap is being formatted as:
UDP/IPv6: [::1]:50618
When it should be:
::1
Compared to an IPv4 trap.

The third line also is not correct, but this does not tie into your problem.

So because it is not existing as an address, when snmptt processes the trap it does "the best it can" which is really not much. It basically forwards on the same value to the EXEC statement, which is our /usr/local/bin/snmptraphandling.py script.

gormank wrote:To me it looks like snmptt is not stripping the extra info off the address.

To be very clear about this, the only thing snmptt does with addresses is either dns resolution / manipulation, it does not do any other type of manipulation to the address. We have a situation where we are simply getting an incorrect address in the trap.

gormank wrote:I noticed another v6 feature that Nagios doesn't handle. I know, by now you're ready to strangle me...

The following addresses are the same IPv6. They're not the same to Nagios so a trap arriving as the first on a host defined as the second will be unknown.

2001:4888:a03c00:196
2001:4888:a03c0:196

This is a valid issue you have raised.

Let me go back to how Nagios receives SNMP traps from snmptt.

SNMPTT is using the script /usr/local/bin/snmptraphandling.py which sends PASSIVE check results to the Nagios command pipe. It requires the following arguments:

<HOST> <SERVICE> <SEVERITY> <TIME> <PERFDATA> <DATA>

<HOST> = The hostname of the device that sent the trap to this server.

It's important to not confuse the "hostname" with the "address" of the Nagios object.

So in your example, you would need a host object with the name of 2001:4888:a03

c0

0:196 or 2001:4888:a03

c0

:196

Now, going back to the snmptrapd bug. IF the bug didn't exist, and snmptrapd correctly referenced the address in the trap, snmptrapd would also attempt to resolve that address to a DNS name, and if it did, it would then provide that as the address in the spooled traps. Failing that, if it couldn't resolve it, it would then put the IPv6 address into the trap.
Then SNMPTT would attempt to resolve this address to a DNS name, if it couldn't resolve it, it would submit it to snmptraphandling.py script as the IPv6 address.

As for these two addresses being the same:
200:4888:a03

c0

0:196
2001:4888:a03

c0

:196

They way they are being submitted to Nagios (with my snmptraphandling.py work around), it is a hostname that is being targeted in Nagios, not the actual address of the host.

With that in mind I don't expect that you would have hosts named 2001:4888:a03

c0

0:196 or 2001:4888:a03

c0

:196. With the current bug, we can't use snmptrapd or snmptt to resolve these to DNS names, what other options are available.

Let's go back to the EXEC statement.

/usr/local/bin/snmptraphandling.py <HOST> <SERVICE> <SEVERITY> <TIME> <PERFDATA> <DATA>

Right now, <HOST> is the $r value. This could actually be "pink_monkeys" and if there was a host object in nagios called "pink_monkeys" then it would receive the trap.

So my suggestion is that you could use a MATCH statement that matched the "incorrectly supplied IPv6 address", and then use the actual name of the Nagios object in your snmptt.conf. For example:

Code: Select all

EVENT nSvcEvent .1.3.6.1.4.1.20006.1.7 "Status Events" Normal 
FORMAT The SNMP trap that is generated as a result of an event with the service $* 
EXEC /usr/local/bin/snmptraphandling.py "pink_monkeys" "SNMP Traps" "$s" "$@" "$-*" "The SNMP trap that is generated as a result of an event with the service $*" 
MATCH $r: UDP/IPv6: [200:4888:a03:311f:c0:a:0:196]:45418
MATCH $r: 2001:4888:a03:311f:c0:a::196

Those match statements will need to be modified to a regex match as the port number at the end will always be different.
http://snmptt.sourceforge.net/docs/snmp ... CONF-MATCH
Or
use the REGEX which can modify the EXEC lines:
http://snmptt.sourceforge.net/docs/snmp ... CONF-REGEX

The problem with these solutions is you'll need duplicate EVENTS for event host that you receive traps from.

OR you could do a similar thing in the snmptraphandling.py script and convert the corrected address (using my current modifications) to match it to a valid hostname (or try resolving it to a DNS address) and submit that value instead. Thinking about it, this might be the simplest method.

Until the snmptrapd bug can be resolved, you are going to need to work out how you are going to work around this problem.

Posted: **Thu Apr 28, 2016 12:44 am**

I've been revisiting this problem and I believe I have a more permanent solution (until the Net-SNMP bug is fixed).

Simply get the /usr/local/bin/snmptraphandling.py script to do a dns resolution on the IPv6 address.

Edit /usr/local/bin/snmptraphandling.py
Change this:

Code: Select all

host = sys.argv[1]

To this:

Code: Select all

host = sys.argv[1]
if host.startswith("UDP/IPv6"):
		try:
				import socket
				host = socket.gethostbyaddr(sys.argv[1].partition('[')[-1].rpartition(']')[0])[0]
				#host = host.partition('.')[0]
		except:
				host = sys.argv[1].partition('[')[-1].rpartition(']')[0]

This will try to resolve the address, if it was not successful, then it simply submits the IPv6 address to Nagios.

This line here:

Code: Select all

host = socket.gethostbyaddr(sys.argv[1].partition('[')[-1].rpartition(']')[0])[0]

Will result in something like myserver.mydomain.local

If you uncomment this line:

Code: Select all

#host = host.partition('.')[0]

The object will be the short version myserver

Posted: **Thu Apr 28, 2016 12:55 pm**

The issue with addresses is in Nagios, not net-snmp and not snmptt. Nagios treats the address as a static string, but in IPv6 its a somewhat dynamic string. You can put pretty much anything for an address and Nagios won't care. This was fine for v4 but not v6.

Posted: **Thu Apr 28, 2016 4:13 pm**

The snmptraphandling.py script, when it submits a passive check result, targets the name of the host object, not the address directive.

The changes to the script will resolve the IPv6 address to a DNS record. Hence you can target a human friendly named address object.

Nagios Support Forum

IPv6 and SNMP: Traps arrive, but not processed by snmptrapd

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr

Re: IPv6 and SNMP: Traps arrive, but not processed by snmptr