snmp Trap import to Nagios XI missing EXEC line

[dybnguyen]

Working with Trevor McDonald on Nagios Quickstart, we ran into an issue when trying to import a SNMP MIB file with process trap option.
Here is a quick recap:
1. Admin -> Manage MIB -> Download one of the MIB file in the list (NET-SNMP-MIB)
2. Browse and select the downloaded MIB file
3. Check "process trap" abd then click "Upload MIB" button
4. Login into Nagios XI server and check the snmptt.ini. It would have the fule path name to the "processed MIB files".
5. In our case, it's in /usr/share/snmp/mibs/processed_mibs/NET-SNMP-AGENT-MIB.txt
6. The issue here is all the events do not have the EXEC line like the following:
...
EVENT event-apias-lsnr-disabled .1.3.6.1.4.1.18328.1.2.5401 "Status Events" INFORMATIONAL
FORMAT Local listener stopped accepting connections.: APIAS disabled listener $1 on $2
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "Local listener stopped accepting connections.: APIAS disabled listener $1 on $2"
SDESC
...

We would need a work around to add the MIB with with process trap. Working with Trevor on the Quickstart and he is familiar with this issue. So I would prefer Trevor to continue working with us on this case.

We also want to confirm whether the preloaded MIB files that are listed in the GUI page would need to be reloaded for "process trap" to be enabled.

Thanks.
Dy

Ericsson dba iconectiv

[tmcdonald]

To clarify a few things:

The MIBs that shipped by default with the system were not processed for traps, so any traps coming in that would rely on them being processed were showing up as unknown. This may be expected, since most MIBs we see are for specific devices that would *not* already have a processed trap in the snmptt configs. I will have to confirm this tomorrow.

What is most certainly an issue is that MIBs we uploaded (with the Process Traps box checked) were *not* being processed properly. They were missing the EXEC line which I had to manually enter. I will confirm tomorrow if this is the case on our systems in-house and if so will file a bug report.

[kbarina]

I am having this same issue on 2014R1.5. When I use Manage MIBs via the web UI and select "process traps", it generates the file in /usr/share/snmp/mibs/processed_mibs but no EXEC line is created. If I use the addmibs command from the CLI instead, the entries are added correctly to snmptt.conf.

[tmcdonald]

I'm going to go ahead and re-submit that bug report. I'm thinking it fell through the cracks because I'm not seeing it.

EDIT: Bug report ID4139 posted below:

Code: Select all

Reproducibility: Always
Target Version: 2014R1.5 stock
Description: Admin -> Manage MIBs is not properly processing them when the "Process traps" box is checked.  The file gets created but there is no EXEC line

Steps To Reproduce: 

0.)  Make sure snmptt and the other trap-necessary packages are installed
1.)  Download a MIB you do not already have on your system (I used the akcp.mib from the SensorProbe doc)
2.)  Upload via the Manage MIBs page, making sure to check "Process trap"
3.)  The upload will be successful, but the entry in /usr/share/snmp/mibs/processed_mibs/whatever.txt will lack an EXEC line that should point to our snmptraphandler.py script

Additional Information: 

support.nagios.com/forum/viewtopic.php?f=6&t=29219&p=113663

[dybnguyen]

Hi Trevor,
I finally able to search and get back to this discussion. Look like you file a bug report within Nagios org. Do you know how we would obtain the fix for this?

I also ran into another issue related when making change to the SNMP trap setting. For the SNMP Trap service,
under the Notification Settings tab, the default setting is:

Notification Settings

Specify the parameters that determine how notifications should be sent for the service .
When a problem is detected.
o Wait [0] minutes before sending a notification
...
If we change the value from 0 to any other number, the email notification is no longer working.
Would you please help and show us how to change this values?
That is, we want to wait for 1 minutes before sending out the email notifications for incoming SNMP Trap.
The reason for this is if the SNMP trap is reset within 1 minute, then no email notification is sent out which is correct.

Thanks.
Dy Nguyen
Ericssion dba iconectiv

[sreinhardt]

This would be updated through our standard update procedures, unless there is a smaller component or patch that we can separately ship to you. As for your notification issue, after you have set a delay, has the trap service returned to an OK state prior to when the notification delay would have been completed? If so, the notification was properly not sent, as the service had recovered and nagios understands that it should not have sent anything as there was no longer an issue. However if it did not send and it had not went to an OK state, then we would want to look at logs.

[kranthi_sistla]

Hi Spenser,

I am working with Dy on this issue . Please find below the details you had asked :

To give a overview of what we are looking for. We have few links that we monitor , whenever any link goes down we want the email notification to be sent out. However, if the links are down for less than 5 mins then we don't want any notification to go out.

Scenario 1 - first_notification_delay = 0 ==> We received the email alert for this configuration.

[1412954302] SERVICE ALERT: njprdhub1;SNMP Traps;CRITICAL;HARD;1;Min inbound sessions not reached: RAISED ALARM - Barnwood Digital IN US SCB number of inbound Transceiver sessions is below minimum of 2 / enterprises.18328.1.6.18601.1 ():Barnwood Digital IN US SCB enterprises.18328.1.6.18601.2 ():Transceiver enterprises.18328.1.6.18601.3 ():2
[1412954302] SERVICE NOTIFICATION: cscc;njprdhub1;SNMP Traps;CRITICAL;xi_service_notification_handler;Min inbound sessions not reached: RAISED ALARM - Barnwood Digital IN US SCB number of inbound Transceiver sessions is below minimum of 2 / enterprises.18328.1.6.18601.1 ():Barnwood Digital IN US SCB enterprises.18328.1.6.18601.2 ():Transceiver enterprises.18328.1.6.18601.3 ():2
[1412954302] SERVICE NOTIFICATION: kranthi;njprdhub1;SNMP Traps;CRITICAL;xi_service_notification_handler;Min inbound sessions not reached: RAISED ALARM - Barnwood Digital IN US SCB number of inbound Transceiver sessions is below minimum of 2 / enterprises.18328.1.6.18601.1 ():Barnwood Digital IN US SCB enterprises.18328.1.6.18601.2 ():Transceiver enterprises.18328.1.6.18601.3 ():2
[1412954302] SERVICE NOTIFICATION: yoga;njprdhub1;SNMP Traps;CRITICAL;xi_service_notification_handler;Min inbound sessions not reached: RAISED ALARM - Barnwood Digital IN US SCB number of inbound Transceiver sessions is below minimum of 2 / enterprises.18328.1.6.18601.1 ():Barnwood Digital IN US SCB enterprises.18328.1.6.18601.2 ():Transceiver enterprises.18328.1.6.18601.3 ():2

Scenario 2 - first_notification_delay = 5 ==> No email notification was received in this condition. In general , whenever we set the delay value anything other than 0 the notifications are not received.

[1412954482] SERVICE ALERT: njprdhub1;SNMP Traps;CRITICAL;HARD;1;Min inbound sessions not reached: RAISED ALARM - Barnwood Digital IN US SCB number of inbound Transceiver sessions is below minimum of 2 / enterprises.18328.1.6.18601.1 ():Barnwood Digital IN US SCB enterprises.18328.1.6.18601.2 ():Transceiver enterprises.18328.1.6.18601.3 ():2

In you response you had asked if the trap service returned to OK before the notification was to be sent out. We did observe the below message in the logs . Is this what you are referring to :

[1412954649] SERVICE ALERT: njprdhub1;SNMP Traps;OK;HARD;1;OK: TRAP RESET

Regards
Kranthi Sistla

[tmcdonald]

It looks like the trap is in fact coming back as OK within 5 minutes. Converting those timestamps gives us the following:

Code: Select all

1412954482 = Fri, 10 Oct 2014 15:21:22 GMT = CRITICAL trap
1412954649 = Fri, 10 Oct 2014 15:24:09 GMT = OK trap

This would cause the notification to not be sent out.

[MrSteve]

Hi I worked with Spencer at the Nagios conference on this issue for over an hour and we still were not able to resolve. Spencer was able to isolate the template that was causing the reset by setting a custom message, but we couldn't figure out why it was still getting fired.

Hopefully we can figure out what the issue is and try and get it resolved soon. I know you folks put a lot of time and effort into putting on a great conference, so hopefully you are getting some rest this weekend, but we would really appreciate some help here.

FYI, I got approval to purchase Nagios Enterprise (just have to work through getting a PO).

Thanks,
Steve Thomas

[Nagios Support]

We will have to do some more digging into this and will get back to you within the next 24 hours.