Nagios Support Forum • Nagios XI R2.9 SNMP traps receive shows "Waiting for trap.."

Page 1 of 1

Nagios XI R2.9 SNMP traps receive shows "Waiting for trap.."

Posted: Thu Apr 03, 2014 1:37 pm

by pshaw1

I followed the instructions in http://assets.nagios.com/downloads/nagi ... ios_XI.pdf. No errors.
Created a device and added the SNMP Trap service to it. Under the status information, it only says "Waiting for trap...".
I checked /var/log/messages and /var/log/snmp/snmptt.log and I see the snmp traps coming in.

I am using snmptt-1.4-0.9.beta2.el6.noarch. I tried using snmptt-1.3-3.nagios.noarch.rpm but when I did, it didn't add the snmptt user and the script failed when trying to add that user to the nagcmd and nagios group.

Re: Nagios XI R2.9 SNMP traps receive shows "Waiting for tra

Posted: Thu Apr 03, 2014 2:57 pm

by sreinhardt

Well, let's start of with some basic information and move on from there. Also I should ask, are you presently on 1.3 or 1.4? I would suggest the latter.

Versions installed

Code: Select all

    rpm -qa | grep snmp

Looking for bins:

Code: Select all

    ls -lva /usr/local/bin | grep -i 'snmp\|addmib'
    ls -lva /usr/local/sbin | grep -i 'snmp\|addmib'
    ls -lva /usr/sbin | grep -i 'snmp\|addmib'
    cat /etc/snmp/snmptrapd.conf

snmptt\trapd settings and user perms:

Code: Select all

    grep -i 'daemon_uid\|mode =' /etc/snmp/snmptt.ini
    grep -i 'exec' /etc/snmp/snmptt.conf | tail -n 10
    grep -i 'nag' /etc/group
    grep -i 'snmp' /etc/group

Checking log and spool dirs:

Code: Select all

    ll /var/log/snmptt/
    ll -d /var/log/snmptt/
    ll /var/spool/snmptt
    ll -d /var/spool/snmptt

Re: Nagios XI R2.9 SNMP traps receive shows "Waiting for tra

Posted: Thu Apr 03, 2014 3:17 pm

by pshaw1

# rpm -qa | grep snmp

Code: Select all

net-snmp-utils-5.5-49.el6_5.1.x86_64
snmptt-1.4-0.9.beta2.el6.noarch
net-snmp-libs-5.5-49.el6_5.1.x86_64
net-snmp-perl-5.5-49.el6_5.1.x86_64
net-snmp-5.5-49.el6_5.1.x86_64
php-snmp-5.3.3-27.el6_5.x86_64

So running snmptt-1.4-0.9.beta2.el6.noarch

# ls -lva /usr/local/bin/ | grep -i 'snmp\|addmib'

Code: Select all

-rwxr-xr-x   1 root nagios      804 Apr  3 12:09 addmib
-rwxr-xr-x   1 root root       2078 Apr  3 12:09 snmptraphandling.py
-rwxr-xr-x   1 root root      30438 Apr  3 12:09 snmpttconvertmib

No bins matching in /usr/local/sbin

# ls -lva /usr/sbin | grep -i 'snmp\|addmib'

Code: Select all

-rwxr-xr-x.  1 root root       30744 Mar  6 12:51 snmpd
-rwxr-xr-x.  1 root root       30776 Mar  6 12:51 snmptrapd
-rwxr-xr-x   1 root root      177466 Oct 22  2012 snmptt
-rwxr-xr-x   1 root root        6493 Oct 22  2012 snmptthandler

# cat /etc/snmp/snmptrapd.conf

Code: Select all

disableAuthorization yes
traphandle default /usr/sbin/snmptthandler

# grep -i 'daemon_uid\|mode =' /etc/snmp/snmptt.ini

Code: Select all

mode = daemon
description_mode = 0
# A second (child) process will be started as the daemon_uid user so
daemon_uid = snmptt

# grep -i 'exec' /etc/snmp/snmptt.conf | tail -n 10

Code: Select all

#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (coldStart)"
#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (warmStart)"
#EXEC qpage -f TRAP notifygroup1 "Link down on interface $1.  Admin state: $2.  Operational state: $3"
#EXEC qpage -f TRAP notifygroup1 "Link up on interface $1.  Admin state: $2.  Operational state: $3"
#EXEC qpage -f TRAP notifygroup1 "SNMP authentication failure"

# grep -i 'nag' /etc/group

Code: Select all

nagios:x:501:nagios,apache,snmptt
nagcmd:x:502:nagios,apache,snmptt

# grep -i 'snmp' /etc/group

Code: Select all

nagios:x:501:nagios,apache,snmptt
nagcmd:x:502:nagios,apache,snmptt
snmptt:x:489:

# ll /var/log/snmptt/

Code: Select all

total 36
-rw-rw-r-- 1 snmptt snmptt  8252 Apr  3 14:07 snmptt.log
-rw-rw-r-- 1 snmptt snmptt  4679 Apr  3 13:52 snmpttsystem.log
-rw-rw-r-- 1 snmptt root   13863 Apr  3 13:58 snmpttunknown.log

# ll -d /var/log/snmptt/

Code: Select all

drwxrwxr-x 2 snmptt snmptt 4096 Apr  3 12:19 /var/log/snmptt/

# ll /var/spool/snmptt

Code: Select all

total 0

# ll -d /var/spool/snmptt

Code: Select all

drwxrwxr-x 2 snmptt snmptt 4096 Apr  3 14:07 /var/spool/snmptt

# tail /var/log/snmptt/snmptt.log

Code: Select all

Thu Apr  3 14:13:34 2014 .1.3.6.1.6.3.1.1.5.3 Critical "Status Events" testswitch - Link down on interface 10101.  Admin state: GigabitEthernet1/0/1.  Operational state: ethernetCsmacd
Thu Apr  3 14:13:49 2014 .1.3.6.1.6.3.1.1.5.4 Normal "Status Events" testswitch - Link up on interface 10101.  Admin state: GigabitEthernet1/0/1.  Operational state: ethernetCsmacd

In Nagios, the device had a domain at the end of it and I noticed in the log above it didn't, so I renamed the device in Nagios and that didn't fix it. I also added in the snmptt.ini the domains to strip just in case.

Re: Nagios XI R2.9 SNMP traps receive shows "Waiting for tra

Posted: Thu Apr 03, 2014 3:41 pm

by sreinhardt

The reason you are not getting these in the nagios interface, but they are considered known traps, is that these two traps do not have proper exec lines within snmptt.conf.

# grep -i 'exec' /etc/snmp/snmptt.conf | tail -n 10

Code: Select all

    #EXEC qpage -f TRAP notifygroup1 "Device reinitialized (coldStart)"
    #EXEC qpage -f TRAP notifygroup1 "Device reinitialized (warmStart)"
    #EXEC qpage -f TRAP notifygroup1 "Link down on interface $1.  Admin state: $2.  Operational state: $3"
    #EXEC qpage -f TRAP notifygroup1 "Link up on interface $1.  Admin state: $2.  Operational state: $3"
    #EXEC qpage -f TRAP notifygroup1 "SNMP authentication failure"

On my system that same command looks like:

Code: Select all

#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (coldStart)"
#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (warmStart)"
#EXEC qpage -f TRAP notifygroup1 "Link down on interface $1.  Admin state: $2.  Operational state: $3"
#EXEC qpage -f TRAP notifygroup1 "Link up on interface $1.  Admin state: $2.  Operational state: $3"
#EXEC qpage -f TRAP notifygroup1 "SNMP authentication failure"
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "The SNMP trap that is generated as a result of an event with the host $*"
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "The SNMP trap that is generated as a result of an event requiring $*"
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "The SNMP trap that is generated as a result of an event with the service $*"
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "The SNMP trap that is generated as a result of an event requiring $*"

A simple way to resolve this would be to add exec lines directly under the commented out ones, something like this should do:

Code: Select all

EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "Link down on interface $1.  Admin state: $2.  Operational state: $3"
AND
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "Link up on interface $1.  Admin state: $2.  Operational state: $3"

You then need to restart snmptt with: service snmptt restart

Re: Nagios XI R2.9 SNMP traps receive shows "Waiting for tra

Posted: Thu Apr 03, 2014 4:00 pm

by pshaw1

Worked!
Note: with the domain names stripped, the alarm wouldn't stay and would get a

Code: Select all

SERVICE ALERT: testswitch;SNMP Traps;OK;HARD;1;OK: TRAP RESET

without another trap received.

So I renamed the device back with the domain and turned off domain stripping in snmptt.ini and now the alarms stay.

Thank you very much for your help!