Nagios XI R2.9 SNMP traps receive shows "Waiting for trap.."

[pshaw1]

I followed the instructions in http://assets.nagios.com/downloads/nagi ... ios_XI.pdf. No errors.
Created a device and added the SNMP Trap service to it. Under the status information, it only says "Waiting for trap...".
I checked /var/log/messages and /var/log/snmp/snmptt.log and I see the snmp traps coming in.

I am using snmptt-1.4-0.9.beta2.el6.noarch. I tried using snmptt-1.3-3.nagios.noarch.rpm but when I did, it didn't add the snmptt user and the script failed when trying to add that user to the nagcmd and nagios group.

[sreinhardt]

Well, let's start of with some basic information and move on from there. Also I should ask, are you presently on 1.3 or 1.4? I would suggest the latter.

Versions installed

Code: Select all

    rpm -qa | grep snmp

Looking for bins:

Code: Select all

    ls -lva /usr/local/bin | grep -i 'snmp\|addmib'
    ls -lva /usr/local/sbin | grep -i 'snmp\|addmib'
    ls -lva /usr/sbin | grep -i 'snmp\|addmib'
    cat /etc/snmp/snmptrapd.conf

snmptt\trapd settings and user perms:

Code: Select all

    grep -i 'daemon_uid\|mode =' /etc/snmp/snmptt.ini
    grep -i 'exec' /etc/snmp/snmptt.conf | tail -n 10
    grep -i 'nag' /etc/group
    grep -i 'snmp' /etc/group

Checking log and spool dirs:

Code: Select all

    ll /var/log/snmptt/
    ll -d /var/log/snmptt/
    ll /var/spool/snmptt
    ll -d /var/spool/snmptt

[pshaw1]

# rpm -qa | grep snmp

Code: Select all

net-snmp-utils-5.5-49.el6_5.1.x86_64
snmptt-1.4-0.9.beta2.el6.noarch
net-snmp-libs-5.5-49.el6_5.1.x86_64
net-snmp-perl-5.5-49.el6_5.1.x86_64
net-snmp-5.5-49.el6_5.1.x86_64
php-snmp-5.3.3-27.el6_5.x86_64

So running snmptt-1.4-0.9.beta2.el6.noarch

# ls -lva /usr/local/bin/ | grep -i 'snmp\|addmib'

Code: Select all

-rwxr-xr-x   1 root nagios      804 Apr  3 12:09 addmib
-rwxr-xr-x   1 root root       2078 Apr  3 12:09 snmptraphandling.py
-rwxr-xr-x   1 root root      30438 Apr  3 12:09 snmpttconvertmib

No bins matching in /usr/local/sbin

# ls -lva /usr/sbin | grep -i 'snmp\|addmib'

Code: Select all

-rwxr-xr-x.  1 root root       30744 Mar  6 12:51 snmpd
-rwxr-xr-x.  1 root root       30776 Mar  6 12:51 snmptrapd
-rwxr-xr-x   1 root root      177466 Oct 22  2012 snmptt
-rwxr-xr-x   1 root root        6493 Oct 22  2012 snmptthandler

# cat /etc/snmp/snmptrapd.conf

Code: Select all

disableAuthorization yes
traphandle default /usr/sbin/snmptthandler

# grep -i 'daemon_uid\|mode =' /etc/snmp/snmptt.ini

Code: Select all

mode = daemon
description_mode = 0
# A second (child) process will be started as the daemon_uid user so
daemon_uid = snmptt

# grep -i 'exec' /etc/snmp/snmptt.conf | tail -n 10

Code: Select all

#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (coldStart)"
#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (warmStart)"
#EXEC qpage -f TRAP notifygroup1 "Link down on interface $1.  Admin state: $2.  Operational state: $3"
#EXEC qpage -f TRAP notifygroup1 "Link up on interface $1.  Admin state: $2.  Operational state: $3"
#EXEC qpage -f TRAP notifygroup1 "SNMP authentication failure"

# grep -i 'nag' /etc/group

Code: Select all

nagios:x:501:nagios,apache,snmptt
nagcmd:x:502:nagios,apache,snmptt

# grep -i 'snmp' /etc/group

Code: Select all

nagios:x:501:nagios,apache,snmptt
nagcmd:x:502:nagios,apache,snmptt
snmptt:x:489:

# ll /var/log/snmptt/

Code: Select all

total 36
-rw-rw-r-- 1 snmptt snmptt  8252 Apr  3 14:07 snmptt.log
-rw-rw-r-- 1 snmptt snmptt  4679 Apr  3 13:52 snmpttsystem.log
-rw-rw-r-- 1 snmptt root   13863 Apr  3 13:58 snmpttunknown.log

# ll -d /var/log/snmptt/

Code: Select all

drwxrwxr-x 2 snmptt snmptt 4096 Apr  3 12:19 /var/log/snmptt/

# ll /var/spool/snmptt

Code: Select all

total 0

# ll -d /var/spool/snmptt

Code: Select all

drwxrwxr-x 2 snmptt snmptt 4096 Apr  3 14:07 /var/spool/snmptt

# tail /var/log/snmptt/snmptt.log

Code: Select all

Thu Apr  3 14:13:34 2014 .1.3.6.1.6.3.1.1.5.3 Critical "Status Events" testswitch - Link down on interface 10101.  Admin state: GigabitEthernet1/0/1.  Operational state: ethernetCsmacd
Thu Apr  3 14:13:49 2014 .1.3.6.1.6.3.1.1.5.4 Normal "Status Events" testswitch - Link up on interface 10101.  Admin state: GigabitEthernet1/0/1.  Operational state: ethernetCsmacd

In Nagios, the device had a domain at the end of it and I noticed in the log above it didn't, so I renamed the device in Nagios and that didn't fix it. I also added in the snmptt.ini the domains to strip just in case.

[sreinhardt]

The reason you are not getting these in the nagios interface, but they are considered known traps, is that these two traps do not have proper exec lines within snmptt.conf.

# grep -i 'exec' /etc/snmp/snmptt.conf | tail -n 10

Code: Select all

    #EXEC qpage -f TRAP notifygroup1 "Device reinitialized (coldStart)"
    #EXEC qpage -f TRAP notifygroup1 "Device reinitialized (warmStart)"
    #EXEC qpage -f TRAP notifygroup1 "Link down on interface $1.  Admin state: $2.  Operational state: $3"
    #EXEC qpage -f TRAP notifygroup1 "Link up on interface $1.  Admin state: $2.  Operational state: $3"
    #EXEC qpage -f TRAP notifygroup1 "SNMP authentication failure"

On my system that same command looks like:

Code: Select all

#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (coldStart)"
#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (warmStart)"
#EXEC qpage -f TRAP notifygroup1 "Link down on interface $1.  Admin state: $2.  Operational state: $3"
#EXEC qpage -f TRAP notifygroup1 "Link up on interface $1.  Admin state: $2.  Operational state: $3"
#EXEC qpage -f TRAP notifygroup1 "SNMP authentication failure"
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "The SNMP trap that is generated as a result of an event with the host $*"
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "The SNMP trap that is generated as a result of an event requiring $*"
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "The SNMP trap that is generated as a result of an event with the service $*"
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "The SNMP trap that is generated as a result of an event requiring $*"

A simple way to resolve this would be to add exec lines directly under the commented out ones, something like this should do:

Code: Select all

EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "Link down on interface $1.  Admin state: $2.  Operational state: $3"
AND
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "Link up on interface $1.  Admin state: $2.  Operational state: $3"

You then need to restart snmptt with: service snmptt restart

[pshaw1]

Worked!
Note: with the domain names stripped, the alarm wouldn't stay and would get a

Code: Select all

SERVICE ALERT: testswitch;SNMP Traps;OK;HARD;1;OK: TRAP RESET

without another trap received.

So I renamed the device back with the domain and turned off domain stripping in snmptt.ini and now the alarms stay.

Thank you very much for your help!