Snmptt mangling IPv6 trap sender addresses

[gormank]

Hi,
Snmptt is mangling IPv6 trap sender addresses by removing the colons, prepending UDPIPv6, and appending 161.
Needless to say the traps are ending up in unconfigured objects rather than becoming trap alerts. IPv4 traps are of course fine.
Can you help fix this?

Thanks

/var/log/snmptrapd.log:
2021-02-10 23:43:00 0.0.0.0(via UDP/IPv6: [2001:4888:a03:311f:c0:a:0:224]:161) TRAP, SNMP v1, community public
SNMPv2-SMI::enterprises.232 Enterprise Specific Trap (11003) Uptime: 276 days, 4:26:44.48
SNMPv2-MIB::sysName.0 = STRING: ILOMXQ9490191. SNMPv2-SMI::enterprises.232.11.2.11.1.0 = INTEGER: 4 SNMPv2-SMI::enterprises.232.11.2.8.1.0 = STRING: "Remote Insight Test Trap"

/var/log/snmptt/snmptt.log:
Wed Feb 10 23:33:36 2021 .1.3.6.1.4.1.232.0.11003 WARNING "Status Events" UDPIPv620014888a03311fc0a0224161 - Generic trap (11003): Remote Insight Test Trap

NXI 5.7.3
RHEL 7.7

[cdienger]

I haven't been able to reproduce this - I haven't been able to get it to work in a desirable way with the exception of sending traps to and from ::1 either, but would like to get more information about your system while I look into this. Try running:

Code: Select all

snmptrap -v 2c -c public udp6:[2001:4888:a03:311f:c0:a:0:224]:161 42 linkDown

I also noticed traps appear to be coming in on port 161 which is usually used for polling devices and not receiving traps. Try the same command but with port 162 as well:

Code: Select all

snmptrap -v 2c -c public udp6:[2001:4888:a03:311f:c0:a:0:224]:162 42 linkDown

Check the version:

Code: Select all

/usr/sbin/snmptt -v

And what are the contents of /etc/snmp/snmptrapd.conf ?

[gormank]

Running the trap commands doesn't log anything in snmptrapd.log

# /usr/sbin/snmptt -v
SNMPTT v1.4.2

# cat /etc/snmp/snmptrapd.conf
disableAuthorization yes
traphandle default /usr/sbin/snmptthandler

# cat /etc/sysconfig/snmptrapd
OPTIONS="-a -A -Lf /var/log/snmptrapd.log -p /var/run/snmptrapd.pid udp:162 udp6:162"

The IPv6 hosts I'm looking at are ILOs. They have these SNMP port settings under network settings:
SNMP Port 161
SNMP Trap Port 162

I see traps from an IPv4 hosts show port 162 in the logs.
When I nmap both 161 and 162 show closed on the NXI hosts.

# ps -ef | grep snmptrap | grep -v grep
root 12967 1 0 Feb10 ? 00:00:08 /usr/sbin/snmptrapd -a -A -Lf /var/log/snmptrapd.log -p /var/run/snmptrapd.pid udp:162 udp6:162 -f

# nmap -6 -p 162,161 `hostname`

Starting Nmap 6.40 ( http://nmap.org ) at 2021-02-12 00:16 UTC
Nmap scan report for cocsm2mlnag001.m2m.local (2001:4888:a05:311f:e0:a:0:1f5)
Host is up (0.00019s latency).
Other addresses for cocsm2mlnag001.m2m.local (not scanned): fe80::250:56ff:fe98:26c0 fe80::d3f2:d57a:e146:4195 fe80::250:56ff:fe98:8e5
rDNS record for 2001:4888:a05:311f:e0:a:0:1f5: cocsm2mlnag001v6
PORT STATE SERVICE
161/tcp closed snmp
162/tcp closed snmptrap

Nmap done: 1 IP address (1 host up) scanned in 0.12 seconds

# nmap -p 162,161 `hostname`

Starting Nmap 6.40 ( http://nmap.org ) at 2021-02-12 00:16 UTC
Nmap scan report for cocsm2mlnag001.m2m.local (10.136.243.84)
Host is up (0.00014s latency).
rDNS record for 10.136.243.84: cocsm2mlnag001
PORT STATE SERVICE
161/tcp closed snmp
162/tcp closed snmptrap

[gormank]

This was done when the new systems were built last year. Has it been done on your test box?
https://support.nagios.com/kb/article.php?id=499

[gormank]

BTW, this started after upgrading from a 2014 version to 7.7.3 by upgrading the old version, creating a new VM with RHEL 7, installing the new NXI version 5.7.3, then restoring a backup of the NXI from the old server.

[ssax]

This is definitely /usr/sbin/snmptt that is doing it by stripping out any special characters but I'm having trouble finding where.

Code: Select all

==> /var/log/snmptt/snmptt.debug <==
Processing file: #snmptt-trap-1613179837610937
Reading trap.  Current time: Fri Feb 12 19:30:40 2021
Symbolic trap variable name detected (DISMAN-EVENT-MIB::sysUpTimeInstance).  Will attempt to translate to a numerical OID
  Translated to .1.3.6.1.2.1.1.3.0

Symbolic trap variable name detected (SNMPv2-MIB::snmpTrapOID.0).  Will attempt to translate to a numerical OID
  Translated to .1.3.6.1.6.3.1.1.4.1.0


Raw trap passed from snmptrapd:
1613179837
UDP/IPv6: [fe80::5e94:7ebb:76aa:6ab7%ens160]:36940
UDP/IPv6: [fe80::5e94:7ebb:76aa:6ab7%ens160]:36940
DISMAN-EVENT-MIB::sysUpTimeInstance 0:0:00:00.42
SNMPv2-MIB::snmpTrapOID.0 IF-MIB::linkDown

Items passed from snmptrapd:
value 0: UDPIPv6fe805e947ebb76aa6ab7ens16036940

value 1:

value 2: .1.3.6.1.2.1.1.3.0

value 3: 0:0:00:00.42

value 4: .1.3.6.1.6.3.1.1.4.1.0

value 5: IF-MIB::linkDown

Symbolic trap variable name detected (IF-MIB::linkDown).  Will attempt to translate to a numerical OID
  Translated to .1.3.6.1.6.3.1.1.5.3

Agent IP address was blank, so setting to the same as the host IP address of

Agent IP address () is the same as the host IP, so copying the host name: UDPIPv6fe805e947ebb76aa6ab7ens16036940

Trap received from UDPIPv6fe805e947ebb76aa6ab7ens16036940: IF-MIB::linkDown

[gormank]

Yes, reading that script isn't fun. This is the closest I've found with line numbers. From my reading of the regex, the example comment might be incorrect. Not to mention the a-z replacement isn't replacing the hex values in the address, etc. so I guess I'm not in the right place...

3265 chomp($tempvar[1]=<$input>); # ip address
3266 push(@rawtrap, $tempvar[1]);
3267 $tempvar[1] =~ s/[^a-zA-Z0-9.\-_:\[\]] //g; # Remove most non alphanumeric characters
3268 # Example: UDP: [127.0.0.1]:38249->[127.0.0.1]:162
3269 # Should only be IPs, but also allow for hostname characters, just in case.

[ssax]

Looks like it's this line that's doing it:

Code: Select all

   3256   $tempvar[0]  =~ s/[^a-zA-Z0-9.\-_]//g;  # Remove most non alphanumeric characters

You can try replacing that line with this:

Code: Select all

if ($tempvar[0] =~ m/IPv6/) {
  $tempvar[0] =~ s/UDP\/IPv6: \[//g;
  $tempvar[0] =~ s/\%.*$//g;
} else {
  $tempvar[0]  =~ s/[^a-zA-Z0-9.\-_]//g;  # Remove most non alphanumeric characters
}

But if SNMPTT gets upgraded the change will be reverted and you'll need to re-implement it.

Additionally, you'll need to restart SNMPTT before that change will be honored.

[gormank]

I think $tempvar[0] is the time, and according to what I pasted $tempvar[1] is the address string...

[ssax]

That is incorrect according to the code above that line:

Code: Select all

  chomp($tempvar[0]=<$input>);  # hostname

  $tempvar[0]  =~ s/[^a-zA-Z0-9.\-_]//g;  # Remove most non alphanumeric characters

I just changed what was required to make it work from a trap I sent in with this command:

Code: Select all

snmptrap -v 2c -c public udp6:[the::ipv6:address]:162 42 linkDown