Getting SNMP traps working

[nortonhealthcare]

Getting SNMP traps working

Hello,

I am running Nagios XI 5.5.1 , and am having two problems currently trying to make traps appear in the nagios web interface.

1. When I go to Admin -> Manage MIBs -> Process All Traps, no changes appear in the /etc/snmp/snmptt.conf file. The permissions / ownership are as such:

-rwxr-xr-x 1 root nagios 2.8K Aug 13 15:22 /etc/snmp/snmptt.conf

2. I am not seeing SNMP traps that appear in /var/log/snmptt/snmpttunknown.log appear in Admin -> Unconfigured Objects. For instance:

Tue Aug 14 10:15:04 2018: Unknown trap (.1.3.6.1.4.1.9.9.91.2.0.1) received from 192.168.6.42 at:
Value 0: 192.168.6.42
Value 1: 192.168.6.42
Value 2: 377:9:21:21.99
Value 3: .1.3.6.1.4.1.9.9.91.2.0.1
Value 4: 192.168.6.42
Value 5: public
Value 6: .1.3.6.1.4.1.9.9.91.2
Value 7:
Value 8:
Value 9:
Value 10:
Ent Value 0: .1.3.6.1.4.1.9.9.91.1.2.1.1.4.3072.3=-200
Ent Value 1: .1.3.6.1.4.1.9.9.91.1.1.1.1.4.3072=-217


Any help would be greatly appreciated!

================================================================================

EDIT:

I just read the guidelines on getting problems resolved faster. Here are the answers to those questions:

1. CentOS 7.4.1804
2. 64 bit
3. Manual install
4. No proxy, no ssl

Here is the output of the system profile:


System Profile
A system profile makes it easier for our support techs to understand the system that you are running on. Including a downloaded system profile with your support ticket is always recommended.


Nagios XI - System Info
System
Nagios XI version: 5.5.1
XI installed from: manual
XI UUID: 75450274-7529-466e-8ed0-8aef83bf516b
Release info: nhclnx59.campus.alliant.org 3.10.0-693.5.2.el7.x86_64 x86_64
CentOS Linux release 7.5.1804 (Core)
Gnome is not installed
Apache Information
PHP Version: 5.4.16
Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Server Name: nhclnx59
Server Address: 172.17.52.107
Server Port: 80
Date/Time
PHP Timezone: US/Eastern
PHP Time: Tue, 14 Aug 2018 12:52:36 -0400
System Time: Tue, 14 Aug 2018 12:52:36 -0400
Nagios XI Data
License ends in: MSQQRR
UUID: 75450274-7529-466e-8ed0-8aef83bf516b
Install Type: manual/unknown

Aug 14 12:49:31 nhclnx59.campus.alliant.org check_nrpe[8101]: Remote 172.17.52.242 accepted a Version 2 Packet
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
Aug 14 12:52:07 nhclnx59.campus.alliant.org ndo2db[15266]: Trimming eventhandlers.
CPU Load 15: 0.53
Total Hosts: 11
Total Services: 36

Function get_base_uri() returns: http://nhclnx59/nagiosxi/
Function get_base_url() returns: http://nhclnx59/nagiosxi/
Function get_backend_url(internal_call=false) returns: http://nhclnx59/nagiosxi/includes/compo ... rofile.php
Function get_backend_url(internal_call=true) returns: http://localhost/nagiosxi/backend/

Ping Test localhost
Running:
/bin/ping -c 3 localhost 2>&1
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.034 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.041 ms
64 bytes from localhost (127.0.0.1): icmp_seq=3 ttl=64 time=0.044 ms

--- localhost ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.034/0.039/0.044/0.008 ms
Test wget To localhost
WGET From URL: http://localhost/nagiosxi/includes/components/ccm/
Running:
/usr/bin/wget http://localhost/nagiosxi/includes/components/ccm/
--2018-08-14 12:52:38-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://localhost/nagiosxi/login.php?red ... f&noauth=1 [following]
--2018-08-14 12:52:39-- http://localhost/nagiosxi/login.php?red ... f&noauth=1
Reusing existing connection to [localhost]:80.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: '/usr/local/nagiosxi/tmp/ccm_index.tmp'

0K .......... .......... .... 6.73M=0.004s

2018-08-14 12:52:39 (6.73 MB/s) - '/usr/local/nagiosxi/tmp/ccm_index.tmp' saved [25526]

Network Settings
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

valid_lft forever preferred_lft forever

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: ens192: mtu 1500 qdisc mq state UP group default qlen 1000

link/ether 00:50:56:b7:4f:06 brd ff:ff:ff:ff:ff:ff

inet 172.17.52.107/21 brd 172.17.55.255 scope global ens192

valid_lft forever preferred_lft forever

inet6 fdfe:9042:c53d:0:250:56ff:feb7:4f06/64 scope global noprefixroute dynamic

valid_lft 2592000sec preferred_lft 604800sec

inet6 fe80::250:56ff:feb7:4f06/64 scope link

valid_lft forever preferred_lft forever

3: ens224: mtu 1500 qdisc mq state DOWN group default qlen 1000

link/ether 00:50:56:b7:01:37 brd ff:ff:ff:ff:ff:ff


default via 172.17.55.254 dev ens192 proto static metric 100

172.17.48.0/21 dev ens192 proto kernel scope link src 172.17.52.107 metric 100


Nagios XI Components
actions 2.0.1
alertcloud 1.2.1
alertstream 2.1.0
autodiscovery 2.2.5
backendapiurl 1.0.3
bandwidthreport 1.8.0
bbmap 1.2.0
birdseye 3.2.2
bulkmodifications 2.2.0
capacityplanning 2.3.0
ccm 2.7.0
custom-includes 1.0.4
customcolumn 1.0
customlogin 1.0.0
customlogo 1.2.0
deploydashboard 1.3.0
deploynotification 1.3.3
duo 1.0.0
escalationwizard 1.5.0
freevariabletab 1.0.1
globaleventhandler 1.2.2
graphexplorer 2.2.0
helpsystem 2.0.0
highcharts 4.0.1
homepagemod 1.1.7
hypermap 1.1.6
hypermap_replay 1.2.0
isms 1.2.3
latestalerts 1.2.6
ldap_ad_integration 1.1.0
massacknowledge 2.1.14
metrics 1.2.10
minemap 1.2.4
nagiosbpi 2.7.1
nagioscore
nagioscorecfg
nagiosim 2.2.6
nagiosna 1.4.0
nagiosql
nagvis 2.0.0
nocscreen 1.1.2
nrdsconfigmanager 1.6.4
nxti 1.0.0
opscreen 1.8.0
perfdata
pingaction 1.1.1
pnp
profile 1.4.0
proxy 1.1.4
rdp 1.0.3
rename 1.6.0
scheduledbackups 1.2.0
scheduledreporting
similetimeline 1.5.0
snmptrapsender 1.5.5
statusmap 1.0.2
tracerouteaction 1.1.1
usermacros 1.1.0
xicore
Nagios XI Config Wizards
ec2 1.0.0
s3 1.0.0
autodiscovery 1.4.1
bpiwizard 1.1.4
bulkhostimport 2.0.4
digitalocean 1.0.0
google-cloud 1.0.0
linode 1.0.0
microsoft-azure 1.0.0
rackspace 1.0.0
dhcp 1.1.4
dnsquery 1.1.3
docker 1.0.0
domain_expiration 1.1.4
email-delivery 2.0.4
emcclariion 2013-03-09
esensors_websensor 1.1.4
exchange 1.3.2
folder_watch 1.0.5
ftpserver 1.5.5
genericnetdevice 1.0.3
ldapserver 1.3.3
linux-server 1.5.5
linux_snmp 1.5.4
macosx 1.3.0
mailserver 1.2.4
mongodb_database 1.1.2
mongodbserver 1.1.2
mountpoint 1.0.2
mssql_database 1.6.2
mssql_query 1.6.4
mssql_server 1.9.1
mysqlquery 1.2.3
mysqlserver 1.3.3
nagioslogserver 1.0.5
nagiostats 1.2.3
nagiosxiserver 1.3.0
ncpa 2.0.0
nna 1.0.4
nrpe 1.5.2
oraclequery 1.3.3
oracleserverspace 1.5.3
oracletablespace 1.5.4
passivecheck 1.2.4
passiveobject 1.1.3
postgresdb 1.5.3
postgresquery 1.2.3
postgresserver 1.3.4
printer 1.1.3
radiusserver 2.0.1
sla 1.3.0
snmp 1.5.8
snmp_trap 1.5.3
snmpwalk 1.3.6
solaris 1.2.5
sshproxy 1.5.7
switch 2.4.0
tcpudpport 1.3.3
tftp 1.0.2
vmware 1.7.1
watchguard 1.4.5
website 1.3.0
website_defacement 1.1.5
websiteurl 1.3.7
webtransaction 1.2.5
windowsdesktop 1.4.2
windowseventlog 1.3.3
windowsserver 1.6.1
windowssnmp 1.5.1
windowswmi 2.1.0
Nagios XI Dashlets
alertcloud
bbmap
capacityplanning
graphexplorer
hypermap
latestalerts
metrics
metricsguage
minemap
xicore_xi_news_feed
xicore_getting_started
xicore_admin_tasks
xicore_eventqueue_chart
xicore_component_status
xicore_server_stats
xicore_monitoring_stats
xicore_monitoring_perf
xicore_monitoring_process
xicore_perfdata_chart
xicore_host_status_summary
xicore_service_status_summary
xicore_comments
xicore_hostgroup_status_overview
xicore_hostgroup_status_grid
xicore_servicegroup_status_overview
xicore_servicegroup_status_grid
xicore_hostgroup_status_summary
xicore_servicegroup_status_summary
xicore_available_updates
xicore_network_outages
xicore_network_outages_summary
xicore_network_health
xicore_host_status_tac_summary
xicore_service_status_tac_summary
xicore_feature_status_tac_summary
availability
custom_dashlet 1.0.5
gauges 1.2.2
googlemapdashlet 1.1.0
internettrafficreport
rss_dashlet 1.1.0
sansrisingports 2.0
sla
statusinfo 2015-12-18
worldtimeserver 2.0.0

[tgriep]

The Process All Traps button will go through all of the MIB files in the /usr/share/snmp/mibs folder and try to add the entries if possible.
But, if a MIB file requires another MIB file to be loaded first, that would cause the files to not load correctly.

In your example, the OID .1.3.6.1.4.1.9.9.91.2.0.1 comes from the CISCO-ENTITY-SENSOR-MIB mib file.
But that file requires the following MIB files to be loaded first.

Code: Select all

ENTITY-MIB
CISCO-SMI

So download those 2 MIB files and add it to the Nagios server but use the "Add to SNMPTT" option.
Then add the CISCO-ENTITY-SENSOR-MIB again using the same "Add to SNMPTT" option.
Hopefully that will fix it for you.

A couple of things you may have to do first, you may have to remove the existing bad entries in the /etc/snmp/snmptt.ini and the /etc/snmp/snmptt.conf file first before you can re-upload the MIB files.
In the bottom of the /etc/snmp/snmptt.ini it may have entries to the processed mib folder /usr/share/snmp/mibs/processed_mibs
Remove those entries as well.


If after doing this it still does not work, we would need to see the SNMP configuration files and the MIB files from the server so can you run the following 3 commands as root.

Code: Select all

tar cvfz /tmp/snmp.tgz /etc/snmp/*
tar cvfz /tmp/sharesnmp.tgz /usr/share/snmp/mibs/*
tar cvfz /tmp/snmplog.tgz /var/log/snmptt/*

Then post these 3 files so we can check the settings and the MIB files for any errors.

Code: Select all

/tmp/snmp.tgz
/tmp/sharesnmp.tgz
/tmp/snmplog.tgz

[nortonhealthcare]

So download those 2 MIB files and add it to the Nagios server but use the "Add to SNMPTT" option.

Then add the CISCO-ENTITY-SENSOR-MIB again using the same "Add to SNMPTT" option.
Hopefully that will fix it for you.

I was able to download the files, but can you elaborate on the "Add to SNMPTT" process? I can't find any reference to it on the web. Is this in the web interface somewhere?

[tgriep]

The "Add to SNMPTT" button is in the XI GUI under the Admin > Manage MIBs menu (XI 5.5.2 has it for sure).
Older versions of XI, the box was called "Process Traps".

When selecting the "Add to SNMPTT" button, the upload will go through the MIB files and add the Trap entries to the Trap Translator which then forwards the Trap to XI.

[nortonhealthcare]

OK. I'm running Nagios 5.5.1, which has the "Process Trap" button. I downloaded the 2 files, used the "Process Trap" button for both of them, and then used the "Process All Traps" option.

/etc/snmp/snmptt.conf has 66 lines, all of which are commented out.
/etc/snmp/snmptt.ini has 627 lines.

I can currently see unconfigured traps in "SNMP Trap Interface", but do not see anything under "Unconfigured Objects." How do I get Nagios to recognize the traps as coming from unconfigured objects so that I can add monitoring for those objects?

I have included the files you have requested.

[tgriep]

Thanks for the files.

With the new SNMP Trap Interface menu came with some changes.
The configurations are not stored in the snmptt.conf file but are now stored in the snmptt.conf.nxti file.

Your config file has the entries but is missing the EXEC line that forwards the received trap to the Nagios process.

Try this, make a backup of the snmptt.conf.nxti and then remove the entries from it and save it.
Login to the XI GUI and go to the Admin Manage MIBs menu.
Click on the Browse button and select the MIB file and upload the MIB file with the "Process Trap" button checked.
Do that for all of the MIB files.

Don't use the Process All Traps button or the Import All to Trap Interface button.

After the MIBs are imported, force the trap from the device and see if the traps are in the Unconfigured Objects menu.

[nortonhealthcare]

Thanks for the help. I followed the directions, and snmptt.conf.nxti remains 0 bytes in size:

[root@nhclnx59 snmp]# pwd
/etc/snmp
[root@nhclnx59 snmp]# ls -lah snmptt.conf.nxti
-rwxr-xr-x 1 root nagios 0 Aug 16 08:43 snmptt.conf.nxti

Could there be a problem with ownership / permissions?

[ssax]

I recommend doing it manually from the command line for easier troubleshooting, I think there may be a bug with the Process All Traps functionality:

Please run these commands:

Code: Select all

cd /usr/share/snmp/mibs
wget http://www.circitor.fr/Mibs/Mib/C/CISCO-ENTITY-SENSOR-MIB.mib
wget http://www.circitor.fr/Mibs/Mib/C/CISCO-SMI.mib
wget http://www.circitor.fr/Mibs/Mib/C/CISCO-TC.mib
wget http://www.circitor.fr/Mibs/Mib/E/ENTITY-MIB.mib
addmib /usr/share/snmp/mibs/CISCO-ENTITY-SENSOR-MIB.mib
service snmptt restart

The addmib command will add it to the /etc/snmp/snmptt.conf file.

If you get the error:

Code: Select all

This file looks like it has been added already!  Exiting.

Look for the line "MIB: CISCO-ENTITY-SENSOR-MIB (file:/usr/share/snmp/mibs/CISCO-ENTITY-SENSOR-MIB.mib) converted on Thu Aug 16 16:05:30 2018 using snmpttconvertmib v1.3" and remove it and rerun the addmib command again.

I'll have to talk with the developers to figure out what the process all traps button is supposed to do, it isn't putting in an EXEC line in the ones in /usr/share/snmp/mibs/processed_traps so I think that is the bug.

Let us know the results.

[nortonhealthcare]

That did indeed work for getting the switch that was sending traps to Nagios to appear in Unconfigured Objects. Thanks! Are the test traps from localhost supposed to also show up in unconfigured objects? I'll be configuring a lot of monitors via SNMP traps and would like to know if that's an easy way to test that Nagios is functioning properly. Thus far localhost has not appeared in Unconfigured Objects in two different installations.

[ssax]

They will only show up if you have the MIBs properly installed and added and you have the SNMP Trap service on localhost like the others.