Page 1 of 4

Issue with Receiving SNMP Traps

Posted: Tue Nov 01, 2016 9:34 am
by msmulpuri
Hello,

I have setup Nagios XI with SNMP Trap Receiver and having issues with none of the traps written to the snmptt.log. All the traps received are getting written to snmpttunknown.log. If I check the same traps by sending manually from command line the same can be seen written to the snmpttunknown.log which indicates the traps are working. I have the SNMP Traps for each of the hosts that are monitored in Nagios except they show only Waiting for Trap in the dashboard. I have followed the correct document from Nagios to setup SNMP Traps and also referenced the tutorial article at https://support.nagios.com/kb/article.php?id=77
Can someone please assist here with troubleshooting why the traps not getting written to the snmptt.log or show up in the dashboard.

Regards!

Re: Issue with Receiving SNMP Traps

Posted: Tue Nov 01, 2016 12:00 pm
by gormank
Did you add MIBs from admin, manage MIBs, or using addmib? I think you're missing trap info in the snmptt config so snmptt discards the traps.
Another possibility is the traps are spooled in /var/spool/snmptt. In that case I think a restart of snmptt will fix it for a while. service snmptt restart.

Re: Issue with Receiving SNMP Traps

Posted: Tue Nov 01, 2016 12:29 pm
by msmulpuri
I added the MIBs via Web GUI. There are never any traps coming to /var/spool/snmptt queue. Also my name has been added to the customer support access list and please have this case moved to the customer support queue since I submitted this originally to the generic Forum when I opened the case. I have tried snmptt service restart but no luck. Please help. Thanks!

Re: Issue with Receiving SNMP Traps

Posted: Tue Nov 01, 2016 2:15 pm
by tgriep
When you added the MIB's using the XI GUI, did you check the Process MIBs box before uploading them?
When you do that and the file has Trap Definitions, it will add the Trap settings in the /etc/snmp/snmptt.conf file.
Can you post your /etc/snmp/snmptt.conf file and one of the entries from the unknown log file?

Re: Issue with Receiving SNMP Traps

Posted: Tue Nov 01, 2016 8:21 pm
by msmulpuri
Hello,

Below you will find few entries from the snmptt.conf for the MIBs loaded and the Unknown trap entries written to snmpttunknown log file for your reference. The MIBs were loaded successfully when I added them using the wizard from XI GUI and Process MIBs box checked. There were no issues when MIBs loaded. I see all the Trap Definitions added in the snmptt.conf file. Hope this helps if needed I can attach the entire snmptt.conf file. Current scenario is there are two Nagios XI servers both of them were setup as Trap Sender/Receiver to one another both monitor the infrastructure at the same time. Our main goal here is that we need to receive SNMP traps from all the Broadsoft platform servers where the MIBs specific to Broadsoft servers are loaded. As far as the Broadsoft MIBs are concerned they have been successfully loaded and the snmptt.conf file has the correct Trap Definitions. Firewall is not an issue from the Broadsoft server perspective to send traps to the Nagios XI SNMP Receiver.

-->/etc/snmp/snmptt.conf entries
EVENT bwPMtomcatRestarted .1.3.6.1.4.1.6431.1.1.1.8 "Status Events" Normal
FORMAT This notification provides the date and time of the Tomcat server restart. $*
EXEC /usr/local/bin/snmptraphandling.py "$r" "SNMP Traps" "$s" "$@" "$-*" "This notification provide
s the date and time of the Tomcat server restart. $*"
SDESC
This notification provides the date and time of the Tomcat server restart.
@severity:informational
@subcomponent:processmonitor
@correlationrules:bwPMtomcatLaunched, bwPMtomcatShutDown, bwPMtomcatStarted, and bwPMtomcatDeath can
be correlated into a single notification. These events have to be considered as a transition of th
e state machine for the management of the Tomcat process running on some of the BroadWorks servers.
Variables:
1: identifier
2: timeStamp
3: alarmName
4: systemName
5: severity
6: component
7: subcomponent
8: problemText
9: recommendedActionsText
EDESC

--> /var/log/snmptt/snmpttunknown.log file entries
Tue Nov 1 10:01:28 2016: Unknown trap (.1.3.6.1.4.1.20006.1.7) received from 192.168.2.10 at:
Value 0: 192.168.2.10
Value 1: 192.168.2.10
Value 2: 26:19:37:07.57
Value 3: .1.3.6.1.4.1.20006.1.7
Value 4: 192.168.2.10
Value 5:
Value 6:
Value 7:
Value 8:
Value 9:
Value 10:
Ent Value 0: .1.3.6.1.4.1.20006.1.3.1.2=192.168.1.224
Ent Value 1: .1.3.6.1.4.1.20006.1.3.1.6=SSH
Ent Value 2: .1.3.6.1.4.1.20006.1.3.1.7=2
Ent Value 3: .1.3.6.1.4.1.20006.1.3.1.17=CRITICAL - Socket timeout after 10 seconds

Tue Nov 1 12:45:28 2016: Unknown trap (.1.3.6.1.4.1.20006.1.7) received from 192.168.2.10 at:
Value 0: 192.168.2.10
Value 1: 192.168.2.10
Value 2: 26:22:21:07.55
Value 3: .1.3.6.1.4.1.20006.1.7
Value 4: 192.168.2.10
Value 5:
Value 6:
Value 7:
Value 8:
Value 9:
Value 10:
Ent Value 0: .1.3.6.1.4.1.20006.1.3.1.2=192.168.1.224
Ent Value 1: .1.3.6.1.4.1.20006.1.3.1.6=SSH
Ent Value 2: .1.3.6.1.4.1.20006.1.3.1.7=0
Ent Value 3: .1.3.6.1.4.1.20006.1.3.1.17=SSH OK - OpenSSH_5.3 (protocol 2.0)

Thank you!

Re: Issue with Receiving SNMP Traps

Posted: Wed Nov 02, 2016 9:16 am
by tgriep
The 2 unknown entries in your post looks like the it is missing the NAGIOS-NOTIFY-MIB.txt file entries in the snmptt.conf file and that is why they are ending up in the unknown log.
The mib file should be in the server and you can run the addmib command to add the entries to the snmptt.conf file by running the following

Code: Select all

addmib /usr/share/snmp/mibs/NAGIOS-NOTIFY-MIB.txt
service snmptt restart
That should reload those 2 entries. If you get other entries in the unknown log, that means that the entries are missing in the snmptt.conf file and the MIB will have to be reloaded.

Re: Issue with Receiving SNMP Traps

Posted: Thu Nov 03, 2016 10:50 am
by msmulpuri
Hello,

The 2 unknown entries are fine. The actual issue is with the traps not seen/not received in Nagios XI GUI even though the snmpd is running on all the Broadsoft servers plus the MIBs exist in the /usr/share/snmp/mibs directory on both the Nagios XI trap receiver server and all the Broadsoft servers. I see only Waiting for Trap message in Nagios XI GUI. Not sure why this stays like that.

I the entries for all the Broadsoft MIBs are there in the snmptt.conf file. At this point this is what we need to address as why the Nagios XI GUI still shows Waiting for Traps all the time as opposed to recieve traps from Broadsoft servers. Please find attached the snmptt.conf file from the Trap receiver (Nagios XI). Please help.

Re: Issue with Receiving SNMP Traps

Posted: Thu Nov 03, 2016 11:19 am
by gormank
You may want to go to the start and grok the trap movement through the system.

The snmptrapd daemon listens for and receives traps, logs, then writes them to /var/spool/snmptt/. Snmptt sits around waiting for files in that dir and processes them. If there's a definition from a MIB in its config file, snmptt writes a command file for Nagios to execute, which updates the status of the host. If snmptt has no definition for a trap, it logs and discards the trap.

Since files aren't piling up in the spool dir, snmptt seems to be discarding traps, which indicates its config doesn't tell it what to do.

I'd locate a specific trap that isn't updating the status of an object, and that trap definition in snmptt and post both.
You might also look at admin, unconfigured objects.

I'm just a customer, not support so if the above is wrong, blame me.

Re: Issue with Receiving SNMP Traps

Posted: Thu Nov 03, 2016 12:42 pm
by msmulpuri
Thanks for your reply!

The way traps are working currently is by having the Broadsoft EMS forward the traps to the Nagios XI server. No issues here. However, since each of the Broadsoft servers in Nagios XI are added for SNMP Traps monitoring which stays Waiting for trap status all the time. If I disable Broadsoft EMS server trap forwarding to Nagios XI I am back to square one and have no visibility to the traps. I don't see anything queued up in /var/spool/snmptt and the unconfigured objects either. My understanding is that when the SNMP Traps are configured for a host it should recieve the taps as they happen if any. I hope customer support will help me out.

Re: Issue with Receiving SNMP Traps

Posted: Thu Nov 03, 2016 1:52 pm
by tgriep
On the XI server that is not receiving the Traps, stop the snmptt daemon by running

Code: Select all

service snmptt stop
Then either force a Trap from your devices of wait foe one to happen and look at the files in the /var/spool/snmptt folder.
That is where the snmptrapd daemon should put the files it receives and that is where the snmptt daemon gets them to process and send on to XI.
So you see any of the files in that folder that came from your devices?