Help with SNMP traps?

[scottwilkerson]

Code: Select all

# snmptranslate .1.3.6.1.4.1.17420.0.6
SNMPv2-SMI::enterprises.17420.0.6

Is this to assume that the MIB file from the manufaturer is incorrect?

I would believe that either they are incorrect or there is another MIB that is needed for these OIDs.

I'm not totally sure, but Google thinks you may be looking for DGPUPS-MIB

[jbennett]

This is what I've found as well, but I simply cannot find it to download (it's not on the provided CD). I've shot an email to the manufacturer.

In any case, this is only a few of the PDUs that I'm trying to monitor. I'm still not getting anything from any of the other devices. This MIB is only for 18 of the 190 PDUs that I am trying to monitor via traps and I still can't get anything from the 184 UPSs.

[scottwilkerson]

In any case, this is only a few of the PDUs that I'm trying to monitor. I'm still not getting anything from any of the other devices. This MIB is only for 18 of the 190 PDUs that I am trying to monitor via traps and I still can't get anything from the 184 UPSs.

Are we sure any of the 184 have sent traps? I ask because on many systems a trap is only sent when a problem is detected

[jbennett]

In any case, this is only a few of the PDUs that I'm trying to monitor. I'm still not getting anything from any of the other devices. This MIB is only for 18 of the 190 PDUs that I am trying to monitor via traps and I still can't get anything from the 184 UPSs.

Are we sure any of the 184 have sent traps? I ask because on many systems a trap is only sent when a problem is detected

At some point in the past week plus multiples should have generated traps as we have had power issues at a number of locations. I have not recieved any email notifications of these nor have any of these devices Traps check changed from an OK state when I've checked them at a known down time.

I have the same template applied to the SNMP Traps for both PDUs & UPS as I do for other devices that are sending emails.

[scottwilkerson]

Can we verify that Nagios has not changed the state for these by running a state history report for the hosts in question

[jbennett]

I sent you the statehistory report via PM for the UPS host group for the past week

Let me know if that's not what you need.

Specifically, on page 5, take a look at the critical states for the standard SNMP checks that 'I currently have running for the UPS. There are a number of unknown states as a number of these were offline due to power outages but you won't find any SNMP Traps services listed.

Here is a current example of a UPS that should be generating a trap but isn't (attached).

[lmiltchev]

Can you run the following commands and show us the output?

Code: Select all

tail -n 50 /var/log/snmptt/snmptt.log
cat /var/log/messages | grep trap

[jbennett]

Nothing but a bunch of authentication failures. Since it's just the same thing over and over again, I've snipped the output. I have also included my snmpd.conf for reference.

Code: Select all

Thu Apr 18 13:31:47 2013 .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" nagiosserver - An authenticationFailure trap signifies that the SNMP
Thu Apr 18 13:31:48 2013 .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Thu Apr 18 13:31:48 2013 .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Thu Apr 18 13:31:50 2013 .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" nagiosserver - An authenticationFailure trap signifies that the SNMP
Thu Apr 18 13:31:50 2013 .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Thu Apr 18 13:31:50 2013 .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP

Code: Select all

Apr 18 13:31:43 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:43 nagiosserver.company.local [xxx.xxx.xxx.xxx] (via UDP: [127.0.0.1]:40831) TRAP, SNMP v1, community public   .1.3.6.1.4.1.8072.3.2.10 Authentication Failure Trap (0) Uptime: 1 day, 4:54:03.25
Apr 18 13:31:44 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:44 localhost.localdomain [UDP: [127.0.0.1]:48797]: .1.3.6.1.2.1.1.3.0 = Timeticks: (10404325) 1 day, 4:54:03.25        .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.6.3.1.1.5.5      .1.3.6.1.6.3.1.1.4.3.0 = OID: .1.3.6.1.4.1.8072.3.2.10
Apr 18 13:31:44 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:44 localhost.localdomain [UDP: [127.0.0.1]:49972]: .1.3.6.1.2.1.1.3.0 = Timeticks: (10404325) 1 day, 4:54:03.25        .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.6.3.1.1.5.5      .1.3.6.1.6.3.1.1.4.3.0 = OID: .1.3.6.1.4.1.8072.3.2.10
Apr 18 13:31:45 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:45 nagiosserver.company.local [xxx.xxx.xxx.xxx] (via UDP: [127.0.0.1]:40831) TRAP, SNMP v1, community public   .1.3.6.1.4.1.8072.3.2.10 Authentication Failure Trap (0) Uptime: 1 day, 4:54:05.26
Apr 18 13:31:46 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:46 localhost.localdomain [UDP: [127.0.0.1]:48797]: .1.3.6.1.2.1.1.3.0 = Timeticks: (10404526) 1 day, 4:54:05.26        .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.6.3.1.1.5.5      .1.3.6.1.6.3.1.1.4.3.0 = OID: .1.3.6.1.4.1.8072.3.2.10
Apr 18 13:31:46 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:46 localhost.localdomain [UDP: [127.0.0.1]:49972]: .1.3.6.1.2.1.1.3.0 = Timeticks: (10404526) 1 day, 4:54:05.26        .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.6.3.1.1.5.5      .1.3.6.1.6.3.1.1.4.3.0 = OID: .1.3.6.1.4.1.8072.3.2.10
Apr 18 13:31:47 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:47 nagiosserver.company.local [xxx.xxx.xxx.xxx] (via UDP: [127.0.0.1]:40831) TRAP, SNMP v1, community public   .1.3.6.1.4.1.8072.3.2.10 Authentication Failure Trap (0) Uptime: 1 day, 4:54:07.27
Apr 18 13:31:47 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:47 localhost.localdomain [UDP: [127.0.0.1]:48797]: .1.3.6.1.2.1.1.3.0 = Timeticks: (10404727) 1 day, 4:54:07.27        .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.6.3.1.1.5.5      .1.3.6.1.6.3.1.1.4.3.0 = OID: .1.3.6.1.4.1.8072.3.2.10
Apr 18 13:31:48 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:48 localhost.localdomain [UDP: [127.0.0.1]:49972]: .1.3.6.1.2.1.1.3.0 = Timeticks: (10404727) 1 day, 4:54:07.27        .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.6.3.1.1.5.5      .1.3.6.1.6.3.1.1.4.3.0 = OID: .1.3.6.1.4.1.8072.3.2.10
Apr 18 13:31:48 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" nagiosserver - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:48 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:48 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:48 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" nagiosserver - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:49 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:49 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:49 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" nagiosserver - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:49 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:49 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:49 nagiosserver.company.local [xxx.xxx.xxx.xxx] (via UDP: [127.0.0.1]:40831) TRAP, SNMP v1, community public   .1.3.6.1.4.1.8072.3.2.10 Authentication Failure Trap (0) Uptime: 1 day, 4:54:09.28
Apr 18 13:31:50 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:50 localhost.localdomain [UDP: [127.0.0.1]:48797]: .1.3.6.1.2.1.1.3.0 = Timeticks: (10404928) 1 day, 4:54:09.28        .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.6.3.1.1.5.5      .1.3.6.1.6.3.1.1.4.3.0 = OID: .1.3.6.1.4.1.8072.3.2.10
Apr 18 13:31:50 nagiosserver snmptrapd[2912]: 2013-04-18 13:31:50 localhost.localdomain [UDP: [127.0.0.1]:49972]: .1.3.6.1.2.1.1.3.0 = Timeticks: (10404928) 1 day, 4:54:09.28        .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.6.3.1.1.5.5      .1.3.6.1.6.3.1.1.4.3.0 = OID: .1.3.6.1.4.1.8072.3.2.10
Apr 18 13:31:54 nagiosserver nagios: SERVICE ALERT: localhost;SNMP Traps;WARNING;HARD;1;An authenticationFailure trap signifies that the SNMP  /
Apr 18 13:31:54 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:54 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" nagiosserver - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:54 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:54 nagiosserver snmptt[0]: .1.3.6.1.6.3.1.1.5.5 Warning "Status Events" localhost - An authenticationFailure trap signifies that the SNMP
Apr 18 13:31:54 nagiosserver nagios: SERVICE ALERT: localhost;SNMP Traps;WARNING;HARD;1;An authenticationFailure trap signifies that the SNMP  /
Apr 18 13:31:58 nagiosserver nagios: SERVICE ALERT: localhost;SNMP Traps;WARNING;HARD;1;An authenticationFailure trap signifies that the SNMP  /
Apr 18 13:31:59 nagiosserver nagios: SERVICE ALERT: localhost;SNMP Traps;WARNING;HARD;1;An authenticationFailure trap signifies that the SNMP  /

Code: Select all

###########################################################################
#
# snmpd.conf
#
#   - created by the snmpconf configuration program
#
###########################################################################
# SECTION: Access Control Setup
#
#   This section defines who is allowed to talk to your running
#   snmp agent.

# rwuser: a SNMPv3 read-write user
#   arguments:  user [noauth|auth|priv] [restriction_oid]

rwuser  public auth

# rouser: a SNMPv3 read-only user
#   arguments:  user [noauth|auth|priv] [restriction_oid]

rouser  public auth

# rocommunity: a SNMPv1/SNMPv2c read-only access community name
#   arguments:  community [default|hostname|network/bits] [oid]

rocommunity  public

# rwcommunity: a SNMPv1/SNMPv2c read-write access community name
#   arguments:  community [default|hostname|network/bits] [oid]

rwcommunity  public

###########################################################################
# SECTION: Trap Destinations
#
#   Here we define who the agent will send traps to.

# trapsink: A SNMPv1 trap receiver
#   arguments: host [community] [portnum]

trapsink  localhost public 162

# trap2sink: A SNMPv2c trap receiver
#   arguments: host [community] [portnum]

trap2sink  localhost public 162

# informsink: A SNMPv2c inform (acknowledged trap) receiver
#   arguments: host [community] [portnum]

informsink  localhost public 162

# trapcommunity: Default trap sink community to use
#   arguments: community-string

trapcommunity  public


# authtrapenable: Should we send traps when authentication failures occur
#   arguments: 1 | 2   (1 = yes, 2 = no)

authtrapenable  1

###########################################################################
# SECTION: Monitor Various Aspects of the Running Host
#
#   The following check up on various aspects of a host.

# proc: Check for processes that should be running.
#     proc NAME [MAX=0] [MIN=0]
#
#     NAME:  the name of the process to check for.  It must match
#            exactly (ie, http will not find httpd processes).
#     MAX:   the maximum number allowed to be running.  Defaults to 0.
#     MIN:   the minimum number to be running.  Defaults to 0.
#
#   The results are reported in the prTable section of the UCD-SNMP-MIB tree
#   Special Case:  When the min and max numbers are both 0, it assumes
#   you want a max of infinity and a min of 1.

proc

# disk: Check for disk space usage of a partition.
#   The agent can check the amount of available disk space, and make
#   sure it is above a set limit.
#
#    disk PATH [MIN=100000]
#
#    PATH:  mount path to the disk in question.
#    MIN:   Disks with space below this value will have the Mib's errorFlag set.
#           Can be a raw byte value or a percentage followed by the %
#           symbol.  Default value = 100000.
#
#   The results are reported in the dskTable section of the UCD-SNMP-MIB tree

disk  / 10%
disk  /tmp 10%
disk  /var 10%
disk  /boot 10%
disk  /var/nagiosramdisk 10%

# load: Check for unreasonable load average values.
#   Watch the load average levels on the machine.
#
#    load [1MAX=12.0] [5MAX=12.0] [15MAX=12.0]
#
#    1MAX:   If the 1 minute load average is above this limit at query
#            time, the errorFlag will be set.
#    5MAX:   Similar, but for 5 min average.
#    15MAX:  Similar, but for 15 min average.
#
#   The results are reported in the laTable section of the UCD-SNMP-MIB tree

load  5.0 4.0 3.0

# file: Check on the size of a file.
#   Display a files size statistics.
#   If it grows to be too large, report an error about it.
#
#    file /path/to/file [maxsize_in_bytes]
#
#      if maxsize is not specified, assume only size reporting is needed.
#
#   The results are reported in the fileTable section of the UCD-SNMP-MIB tree

file  /var/log/maillog 209715200
file

Code: Select all

# vi /etc/snmp/snmptrapd.conf
disableAuthorization yes
traphandle default /usr/local/sbin/snmptthandler

Code: Select all

# ls -lh /etc/snmp
total 108K
-rw-r--r-- 1 root   root   3.8K Apr 16 08:44 snmpd.conf
-rw-r--r-- 1 root   root   3.9K Jan 29 10:16 snmpd.conf.new
-rw-r--r-- 1 root   root    19K Jan 29 10:17 snmpd.conf.old
-rw-r--r-- 1 root   root     74 Jan 29 13:24 snmptrapd.conf
-rw-r--r-- 1 root   root    45K Apr 18 08:24 snmptt.conf
-rw-rw-r-- 1 apache nagios  26K Mar 28 09:11 snmptt.ini

[scottwilkerson]

It looks like the only traps that are making it to the server are from the local machine

Is there a firewall that could be blocking port 162 UDP?

[jbennett]

I have asked our networking guys and was told no. I specifically asked about it again and was told that they do not block anything on 162 UDP on our network.