NSTI - Architecture

Overview

This Article will outline and describe how Nagios SNMP Trap Interface works from the ground up. Details describe how SNMP traps are accepted in the backend, displayed in the User Interface, written to the database, and more. Much of the focus of this article will be on the back end handling and writing of traps using Python. This guide applies to NSTI version 3.0.2 or newer.

Glossary of Terms

MIB - Management Information Base: The structure that describes the management data of a device using OID’s

NSTI - Nagios SNMP Trap Interface

OID - Object Identifier: Identifies a variable that can be read or set by SNMP.

Python - [Python] is a widely used general-purpose, high-level programming language.

SNMP - Simple Network Management Protocol

SNMP Trap - Asynchronous notification from agent to manager.

Storm - Storm is an object-relational mapper (ORM) for Python developed at Canonical.

Yum - Package manager utilized by all RHEL based Linux systems.

Note about configuration files: It is assumed that when editing files such and snmptt.ini and snmptrapd.conf or any other configuration files the user will almost always have to restart the service that depends on those files so they can load them anew and the changes can take affect. Keep this in mind while navigating through this documentation.

NSTI Make UP

Nagios SNMP Trap Interface is a web interface loaded by mod_wsgi (served by Apache) and made up of a Python framework, Flask, that allows Abstraction and Object-Relational Mapping to a MySQL database. This allows NSTI to create, and read from, multiple database tables and allow for all the database information to be displayed into a speedy and attractive User Interface using Bootstrap.

Here is a list of the prerequisites that will be included during install:

- mysql
- mysql-devel
- mysql-server
- httpd
- gcc
- wget
- make
- tar
- mod_wsgi
- snmptt
- net-snmp
- python

Once python is available the install will use pip (Python package installer) to install the follow additional dependencies:

- Flask
- MySQL-python
- storm

Basic Workflow

a) Traps, Before Getting to NSTI

1. Generate a trap from an outside host. There are multiple methods to send traps, but snmptt, Net-SNMP and the Perl snmptrap() are the easiest free solutions to use. Trap senders become more sophisticated with handling custom messages, OID tree tables and views and MIB management UI’s and these will all work with NSTI as long as the configuration is correct.

2. snmptrapd receives the trap and writes it to the MySQL database using the snmptt daemon. This is where NSTI is reading, displaying and filtering traps all through the database and multiple overlaying jquery calls that in turn draw the trap table that the user views. Almost every action in the NSTI interface is going to trigger some kind of jquery combined with ajax to query the API and then display what data is returned.

This is also another great feature of NSTI which allows other applications to access the database that NSTI has created and is described in more detail in section II. Backend, (iv) NSTI Application Program Interface (API).

Here is the basic workflow of a Trap going into the NSTI host and being handled and written to the database where it is read and displayed by NSTI:

b) Traps, After Getting to NSTI

After the received traps are written to the database NSTI is able to view the traps and display them into the tables in the User Interface. This is done using jquery and AJAX calls to the API to read the trap tables and display them with a combination of JavaScript, HTML and CSS.

3. Read database and parse Traps into Normal, Unknown or Archive trap tables.

4. Display NSTI User Interface with fully loaded trap tables from live database using JSON encoded API arrays based on the URL of the query.

Configuration of Trap Handlers

There are a number of ways to handle trap sending and receiving for NSTI and we are going to go over a few methods of doing that here. First, the default method of receiving traps is set by NSTI during installation.

Sections:

1) Default Trap Receiving Method After Install

2) Alternative Trap Receiving Methods

1) Default Trap Receiving Method After Install

During installation NSTI will write to the configuration file for snmptt located here: /etc/snmp/snmptt.ini

In install.sh, the main install script for NSTI,the snmptt.sh file is called which makes 3 changes to the snmptt.ini file, shown here:

SNMPTTINI="/etc/snmp/snmptt.ini"

sed -i’.bkp’ ’s/^mode[ \t]*=[ \t]*standalone/mode = daemon/g’ "$SNMPTTINI"
sed -i’.bkp’ ’s/^dns_enable[ \t]*=[ \t]*0/dns_enable = 1/g’ "$SNMPTTINI"
sed -i’.bkp’ ’s/^mysql_dbi_enable[ \t]*=[ \t]*0/mysql_dbi_enable = 1/g’ "$SNMPTTINI"

Let’s break these down to see what they are doing.

Action: Set snmptt to daemon mode and use snmptrapd to collect traps:

sed -i’.bkp’ ’s/^mode[ \t]*=[ \t]*standalone/mode = daemon/g’ "$SNMPTTINI"

Result: mode = daemon (in line 16 of snmptt.ini)

This line of the snmptt.sh file is setting the Mode of Operation of snmptt. This will define if snmptt will work in standalone or daemon mode and since we want to continuously collect traps we want to process anything received into the database for NSTI to read so we will set daemon mode and allow snmptrapd to perform the collection of traps.

snmptrapd.conf

When in daemon mode the snmptrapd.conf file will contain a traphandle statement that will look like this: (from SNMPTT Documentation)

traphandle default /usr/sbin/snmptthandler

This will send the trap caught by snmptrapd and sent to the snmptthandler script as defined above. The handler will do the following:

Read traps passed from snmptrapd
Writes the trap in to a spool directory (commonly /var/spool/snmptt)
Then quits

Then SNMPTT (daemon mode) will run another set of tasks:

Load Config files containing trap definitions at startup
Read traps in spool directory
Search traps for any matches
Log and execute EXEC statements among other trap actions
Sleep for 5 seconds (configurable)
Start at step 2 as a loop to read any traps passed to the spool directory

Source

Here is how NSTI will constantly accept and write traps, but during step 1 above we come to the second change to the snmptt.ini file:

Action: Set DNS resolution to 1 in snmptt.ini

sed -i’.bkp’ ’s/^dns_enable[ \t]*=[ \t]*0/dns_enable = 1/g’ "$SNMPTTINI"

Result: dns_enable = 1 (in line 38 of snmptt.ini)

The first part above simply tells snmptrapd to pass the IP address of both the device sending the trap, and the IP address of the actual SNMP agent. Here is the explanation from the snmptt.ini file itself:

# If DNS is enabled, the agent IP address is converted to a host name using a DNS lookup
# (which includes the local hosts file, depending on how the OS is configured). This name
# will be used for: NODES entry matches, hostname field in logged traps (file / database),
# and the $A variable. Host names on the NODES line will be resolved and the IP address
# will then be used for comparing.
# Set to 0 to disable DNS resolution
# Set to 1 to enable DNS resolution

Source

The last change, and possibly the most important part, of this script is the mysql_dbi_enable which is when SNMPTT trap handler is on the log/execute/write trap step (step 4 above):

Action: Set mysql_dbi_enable to 1 in snmptt.ini

sed -i’.bkp’ ’s/^mysql_dbi_enable[ \t]*=[ \t]*0/mysql_dbi_enable = 1/g’ "$SNMPTTINI"

Result: mysql_dbi_enable = 1 (in line 402 of snmptt.ini)

This will tell snmptt to enable logging to the MySQL database via DBI. This is how NSTI will display, query, filter all the traps in the database so if this is working NSTI is able to show trap data. The SNMPTT Documentation describes that the Perl modules DBI:: and DBD::mysql are required to use this, but even on minimal installs it should be included with the already installed NSTI packages.

The SNMPTT Documentation describes to update the following other variables in snmptt.ini:

mysql_dbi_host
mysql_dbi_port
mysql_dbi_database
mysql_dbi_table
mysql_dbi_table_unknown
mysql_dbi_username
mysql_dbi_password

These won’t need to be changed because NSTI will use the default values for these variables. If you need to change anything in here to customize your Nagios SNMP Trap Interface implementation then make sure to change the database credentials and access variables located in the nsti.py file:

#~ Database Configuration Options
#~ Use mysql for MySQL and postgres for PostgreSQL
DB_TYPE=’mysql’
DB_HOST=’localhost’
DB_PORT=’3306’
DB_NAME=’snmptt’
DB_USER=’snmpttuser’
DB_PASS=’password’

#~ Page Display Options
TRUNCATE=100
PERPAGE=50

As you can see here these values match the defaults in snmptt.ini so if you even want to go back to them after editing them you can look at the default snmptt.ini file.

Source

2) Alternative Trap Receiving Methods

There are a number of ways that will allow a user to receive traps and write them to the database. Here is an example of another trap receiving method you could use to accomplish the same thing that the above is trying to do. Ultimately if the receiver can handle any incoming traps, write them to /var/spool/snmptt, parse traps in pool and write them to the database you can make any trap receiver work with NSTI.

The Perl Net:: SNMPTrapd works the same way as snmptrapd, but uses Perl modules to accomplish the same thing. Learn about using this at the CPAN manual page for the module:

https://metacpan.org/pod/Net::SNMPTrapd

NSTI will work with any type of trap receiver as long as the outcome is that a trap follows the a similar workflow or at least simply ends with spooled traps being written to the MySQL database.

Backend

Handling of Traps

Here is a workflow diagram of SNMP traps detailing the path of creation > transmission > reception > handling > writing of a trap coming into NSTI and finally page request, loading and drawing of pages and tables:

This is described in more detail in the (iii) Configuration of Trap Handlers and how snmptt writes traps to the database using the SNMP Trap Translator suite.

Python

The framework for NSTI is first made up of a Virtual Web Server Gateway Interface (WSGI) which is a commonly used deployment platform for Python and in NSTI is how Django is deployed, then a Python framework allowing database object-relational mapping (Flask) and WSGI libraries to allow Flask to build the web frame work for NSTI to display our trap data (Jinjga2 and Werkzeug). Below will describe how these all fit together to create the NSTI User Interface.

WSGI

The Web Server Gateway Interface is the Python standard for web servers and applications. There are a large number of frameworks that support WSGI, here is a list.

WSGI is accessed and served through the apache protocol using mod_wsgi and defined in the /nsti/nsti/dist/apache.conf:

WSGIScriptAlias /nsti /usr/local/nsti/wsgi.py

<Directory /usr/local/nsti>
Order allow,deny
Allow from all
</Directory>

This will determine allowed access for NSTI, which by default is ’all.’ This will be another way to grant deeper security for the database and the user interface. It show the directory paths and how WSGI will load the application.

In Nagios SNMP Trap Interface we are using Django, the Python framework, which will allow us to use Flask.¹ Flask is what allows NSTI to use relational mapping in our database which is a way for each trap in the database to have certain column data allowing NSTI to display and filter traps, via the API, based on the time written, ID, OID, etc.

This Server Gateway Interface is defined by the following file in NSTI, located at /nsti/wsgi.py:

import sys
import os

basedir = os.path.abspath(os.path.dirname(__file__))
nstidir = os.path.join(basedir, ’nsti’)

sys.path.append(nstidir)
from nsti import app as application

application.secret_key = ’mysecretkey’

As you can see when the /nsti is accessed WSGI will import app as application and this will allow the user to access all the pages with the /nsti domain which will also serve access to the API through /nsti/api/. This is also where the application.secret_key = ’ ’ will be defined so that users can add another layer of security and not allow access to the URL or the API which could be used to maliciously move, alter or delete data in the database.

The wsgi.py above is then going to allow the loading of the /nsti/nsti/nsti.py file which contains access to the ’/’ directive for NSTI and contains a bad request callback handler so we can find out why the framework failed to load or render a callback so we can track the problem down using the sophisticated debugger:

from flask import Flask, url_for, render_template, redirect
import os

app = Flask(__name__)
app.config.from_pyfile(os.path.join(app.root_path, ’etc’, ’nsti.py’))
app.secret_key = ’mysecretkey’
app.server_name = ’nsti.dev’
app.jinja_env.globals[’static’] = lambda filename: url_for(’static’, filename=filename)

#~ First the error handlers...
@app.errorhandler(400)
def bad_request(error):
return render_template(’system/bad_request.html’, error=error), 400

#~ Then our default landing page...
@app.route(’/’)
def landing():
return redirect(url_for(’traplist’))

#~ Now import all of our modules
import trapview
import filters

Here is where WSGI and Flask shine as you can just make a route for anything you might want. As NSTI develops a bit more we will have access to add new routes wherever they might be needed.

View the WSGI Documentation in the Resources section.

Flask, A Micro-framework

NSTI currently uses Flask 0.10. It is called a "micro"-framework because its goal is keep the core "simple but extensible." The nsti.py above shows how Flask with load the python application based on the configuration set in nsti.py. Then the application is called by the following:

app = Flask(__name__)

This is how the route system will work while __name__ in our case will be nsti. Now we can access the "/" followed by nsti which will run the following in Flask:

if __name__ == "__main__":
    app.run()

Now you are serving the application through the gateway (WSGI) and will be able to access the URL anytime the request is sent.

For debugging purposes you can catch any errors through Flask by setting the above in nsti.py to this:

app.run(debug=True)

This will show the Flask traceback and which file might have causes or returned an error. The trace back is already set up for NSTI, but this could be a method of getting very verbose debugging information.

View the Flask Documentation in the Resources section.

Django, Nothing Micro About It

Django is a ’high-level’ Python web framework. Flask uses Django to display web content and handle routes to pages as shown in the previous WSGI section. The reason Flask can do what it does with routes and accessing the API is because of the high level of the Django frame work design.

View the Django Documentation in theResources section.

Jinja2 and Werkzeug

Jinja2 is a template engine that allows NSTI to display template style HTML pages in addition to using JavaScript. It allows usage such as the header in NSTI showing a block that will be loaded in its place:

<title>{% block title %}{% endblock %}</title>
<ul>
{% for user in users %}
  <li><a href="{{ user.url }}">{{ user.username }}</a></li>
{% endfor %}
</ul>

Werkzeug is a library with utilities to create or manage a framework. One example of NSTI using Werkzeug is how it shorten URL’s and allows /nsti and other URL directives to be shortened and accessed. When NSTI is being developed it is easier to access the developer mode through the port :8080, but Werkzeug allows users to access this same directive by using the /nsti URL in production.

On the backend Werkzeug is being utilized by allowing NSTI to make actions and queries based on passed arguments in the URL, here is an example of this being used in Line 59 of the trapview.py file. This section below is used to display the main trap table using jquery and the ../read/.. protocol from the API:

@app.route(’/api/trapview/read/<tablename>’, methods=[’GET’, ’POST’, ’OPTIONS’])
@origin_access_allow_all
def read(tablename):
traptype = getattr(db, tablename)
trap_filters = filters.get_requested_filters()

where_clause = db.sql_where_query(traptype, request.args, trap_filters)

if where_clause:
results = db.DB.find(traptype, where_clause)
else:
results = db.DB.find(traptype)

result_dict = db.encode_storm_result_set(results)

json_str = json.dumps(result_dict, default=db.encode_storm_result_set, indent=4)
return Response(response=json_str, status=200, mimetype=’application/json’)

View the Jinja2 and Werkzeug Documentation in the Resources section.

Database

Nagios SNMP Trap Interface utilizes a MySQL database that is constructed by the Net-SNMP suite during the NSTI install and is called snmptt. Then, during the install, NSTI is going to add tables to the database in the /nsti/install/database.sh:

#!/bin/sh

DB_SCHEMA="${BASEPATH}/nsti/dist/nsti.sql"
INTERACTIVE="True"


if mysqlshow -u root &>/dev/null; then
	# Set the password to "nagiosxi"
	mysqlpass=nagiosxi
	mysqladmin -u root password "$mysqlpass"
	echo "MySQL root password is now set to: $mysqlpass"
else
	for i in 1 2 3; do
		if [ "$INTERACTIVE" = "True" ]; then
			# Ask for the password
			read -p "Enter MySQL Root Password: " pass
		fi

		# Test the password
		if mysqlshow -u root -p"$pass" &>/dev/null; then
			echo "Password validated."
			mysqlpass="$pass"
			break
		else
			echo "Password failed." >&2
			[ $i -eq 3 ] && exit 1
		fi
	done
fi

mysqladmin -s -uroot -p"$mysqlpass" create ${DB_NAME}
mysql -uroot -p"$mysqlpass" -e ’CREATE USER "’${DB_USER}’"@"’${DB_HOST}’" IDENTIFIED BY "’${DB_PASS}’";’
mysql -uroot -p"$mysqlpass" -e ’GRANT ALL PRIVILEGES ON snmptt.* TO "’${DB_USER}’"@"’${DB_HOST}’";’

if ! mysql -u${DB_USER} -p${DB_PASS} ${DB_NAME} < $DB_SCHEMA;
then
    echo "Unable to add database schema. Does the ${DB_NAME} database exist?"
    exit 1
fi

It is important to note above that the script will look for a root user in the database and if it doesn’t have a password the database password will be set to nagiosxi. If there is a password set for the MySQL root user the script will allow the user running the install script to interactively type the MySQL root password so the script can complete all of its tasks. If this needs to be changed make sure you modify the correct files so you won’t get an error when trying to connect.

This is going to load the new database schema we want set up for NSTI. This is set by the /nsti/nsti/dist/nsti.sql:

-- MySQL dump 10.13  Distrib 5.1.69, for redhat-linux-gnu (i386)
--
-- Host: localhost    Database: snmptt
-- ------------------------------------------------------
-- Server version       5.1.69

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE=’ 00:00’ */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0*/;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE=’NO_AUTO_VALUE_ON_ZERO’ */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;

-- MySQL dump 10.9
--
-- Host: localhost    Database: snmptt
-- ------------------------------------------------------
-- Server version   4.1.10a
--
-- Table structure for table `snmptt`
--

CREATE TABLE IF NOT EXISTS `snmptt` (
      `id` mediumint(9) NOT NULL auto_increment,
      `eventname` varchar(50) default NULL,
      `eventid` varchar(50) default NULL,
      `trapoid` varchar(100) default NULL,
      `enterprise` varchar(100) default NULL,
      `community` varchar(20) default NULL,
      `hostname` varchar(100) default NULL,
      `agentip` varchar(16) default NULL,
      `category` varchar(20) default NULL,
      `severity` varchar(20) default NULL,
      `uptime` varchar(20) default NULL,
      `traptime` varchar(30) default NULL,
      `formatline` varchar(255) default NULL,
      `trapread` int(11) default ’0’,
      `timewritten` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;


--
-- Table structure for table `snmptt_archive`
--
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE IF NOT EXISTS `snmptt_archive` (
  `id` mediumint(9) NOT NULL AUTO_INCREMENT,
  `snmptt_id` mediumint(9) NOT NULL,
  `eventname` varchar(50) DEFAULT NULL,
  `eventid` varchar(50) DEFAULT NULL,
  `trapoid` varchar(100) DEFAULT NULL,
  `enterprise` varchar(100) DEFAULT NULL,
  `community` varchar(20) DEFAULT NULL,
  `hostname` varchar(100) DEFAULT NULL,
  `agentip` varchar(16) DEFAULT NULL,
  `category` varchar(20) DEFAULT NULL,
  `severity` varchar(20) DEFAULT NULL,
  `uptime` varchar(20) DEFAULT NULL,
  `traptime` varchar(30) DEFAULT NULL,
  `formatline` varchar(255) DEFAULT NULL,
  `trapread` int(11) DEFAULT ’0’,
  `timewritten` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=8 DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;

--
-- Table structure for table `snmptt_unknown`
--
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE IF NOT EXISTS `snmptt_unknown` (
  `id` mediumint(9) NOT NULL AUTO_INCREMENT,
  `trapoid` varchar(100) DEFAULT NULL,
  `enterprise` varchar(100) DEFAULT NULL,
  `community` varchar(20) DEFAULT NULL,
  `hostname` varchar(100) DEFAULT NULL,
  `agentip` varchar(16) DEFAULT NULL,
  `uptime` varchar(20) DEFAULT NULL,
  `traptime` varchar(30) DEFAULT NULL,
  `formatline` varchar(255) DEFAULT NULL,
  `trapread` int(11) DEFAULT ’0’,
  `timewritten` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;

/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;


--
-- Table structure for table `filter_atom`
--
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE IF NOT EXISTS `filter_atom` (
  `id` mediumint(9) NOT NULL AUTO_INCREMENT,
  `column_name` varchar(100) DEFAULT NULL,
  `comparison` varchar(100) DEFAULT NULL,
  `val` varchar(100) DEFAULT NULL,
  `filter_id` mediumint(9),
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;


--
-- Table structure for table `filter`
--
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE IF NOT EXISTS `filter` (
  `id` mediumint(9) NOT NULL AUTO_INCREMENT,
  `name` varchar(100) DEFAULT NULL UNIQUE,
  `combiner` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Here you will see the .sql schema file perform table creation for any table ’IF NOT EXISTS.’ Once completed your snmptt database should look like this:

[root@nstidev01 etc]# mysql -u snmpttuser -ppassword
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 45
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ’help;’ or ’\h’ for help. Type ’\c’ to clear the current input statement.

mysql> use snmptt;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
------------------
| Tables_in_snmptt |
------------------
| filter |
| filter_atom |
| snmptt |
| snmptt_archive |
| snmptt_unknown |
------------------
5 rows in set (0.00 sec)

It is important to note that the database structure is all thanks to Storm, from their website: "Storm is an object-relational mapper (ORM) for Python developed at Canonical.", this will fit in well with how NSTI is using a simple database. Storm will allow NSTI to be able to abstract and compare the data to tailor it for a user to view and create, call, store and stop sessions to the database.

This detail of the above is described in more detail under "User Interface" and "Trap Filters: Storm, a ORM by Cannonical".

NSTI Application Program Interface (API)

Using the API in Nagios SNMP Trap Interface is one of the ways NSTI can be very useful. Here is a general read of the snmptt trap database:

http://<nsti IP>/nsti/api/trapview/read/Snmptt

This will display all of the trap currently in the snmptt database in JSON format. This could allow other software or programs to see the data in the NSTI API as long as you have the correct credentials and the API key. Access any of the tables just change the last varialbe to SnmpttArchive or SnmpttUnknown.

Here is what the above data may look like (example displaying 3 traps):

[
    {
        "eventid": ".1.3.6.1.4.1.2343.2.993.1.17", 
        "category": "Other Event", 
        "uptime": "2015-01-30 07:17:12", 
        "severity": "ok", 
        "traptime": "2015-01-30 07:17:12", 
        "timewritten": "01-30-2015 07:17:12", 
        "formatline": "Do not forget to eat.", 
        "hostname": "192.168.5.1", 
        "community": "private", 
        "agentip": "192.168.5.1", 
        "eventname": "demoTrap", 
        "trapoid": ".1.3.6.1.4.1.2343.2.993.1.17", 
        "trapread": 0, 
        "enterprise": ".1.3.6.1.4.1.2343", 
        "id": 1
    }, 
    {
        "eventid": ".1.3.6.1.4.1.9996.13.990.0.17", 
        "category": "Closure", 
        "uptime": "2015-01-30 07:17:12", 
        "severity": "warning", 
        "traptime": "2015-01-30 07:17:12", 
        "timewritten": "01-30-2015 07:17:12", 
        "formatline": "Oh no the fire hydrant blew up", 
        "hostname": "192.168.5.54", 
        "community": "private", 
        "agentip": "192.168.5.54", 
        "eventname": "demoTrap", 
        "trapoid": ".1.3.6.1.4.1.9996.13.990.0.17", 
        "trapread": 0, 
        "enterprise": ".1.3.6.1.4.1.9996", 
        "id": 2
    }, 
    {
        "eventid": ".1.3.6.1.4.1.9996.45.33.5.6", 
        "category": "Closure", 
        "uptime": "2015-01-30 07:17:12", 
        "severity": "normal", 
        "traptime": "2015-01-30 07:17:12", 
        "timewritten": "01-30-2015 07:17:12", 
        "formatline": "Another sample message", 
        "hostname": "192.168.5.41", 
        "community": "private", 
        "agentip": "192.168.5.41", 
        "eventname": "demoTrap", 
        "trapoid": ".1.3.6.1.4.1.9996.45.33.5.6", 
        "trapread": 0, 
        "enterprise": ".1.3.6.1.4.1.9996", 
        "id": 3
    }
]

You can further filter your data you get from the read directive by adding arguments to the URL:

?<data_column>_<comparison>=<value>

Here are the possible data columns that can be utilized with Nagios SNMP Trap Interface:

Column	SQL Data Type
id	mediumint(9) NOT NULL auto_increment
eventname	varchar(50) default NULL
eventid	varchar(50) default NULL
trapoid	varchar(100) default NULL
enterprise	varchar(100) default NULL
community	varchar(20) default NULL
hostname	varchar(100) default NULL
agentip	varchar(16) default NULL
category	varchar(20) default NULL
severity	varchar(20) default NULL
uptime	varchar(20) default NULL
traptime	varchar(30) default NULL
formatline	varchar(255) default NULL
trapread	int(11) default ’0’
timewritten	timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP

Then you can apply a _lt (less than), _gt (greater than), __contains, __in to the above data types and add a ’=’ that will filter the JSON returned.

You can also string multiple query delimiters together by using AND and OR after each filter or inside a filter to allow multiple types of a single column variable.

You can filter over date data types with the following date specifier keys:

API Abreviation	Value
s	Seconds
m	Minutes
h	Hours
d	Days
w	Weeks
M	Months

Let’s add this to our example in the beginning of the section:

http://<nsti IP>/nsti/api/trapview/read/Snmptt?relative_timewritten__lt=1d(boart

This will display all the traps in the snmptt table that were written to the database less than 1 day ago.

Additionally you can specify an actual date filter using the NSTI database date format here (note the single space between date and time):

MM-DD-YYYY HH:mm:ss

Month(int 2) - Day(int 2) - Year(int 4)   Hour(int 2) - Minute(int 2) - Second(int2)

Here is another example using the exact date format:

http://<nsti IP>/nsti/api/trapview/read/Snmptt?relative_timewritten__lt=01-10-2015 12:00:00

This will display any traps in the snmptt table that were written to the database before January 10th, 2015 12:00:00.

This can be done with all of the above column data types as long the the data being compared exists. If you get an empty array ’[]’ when you run a query it could either be because you aren’t querying any data that exists in that row or that the row doesn’t have data in that value range.

The code that makes these queries possible is located in /nsti/nsti/database.py. This contains all the definitions for the SQL queries that are described above.

This is described in detail in the User Interface andTrap Filterssection below and in the NSTI Documentation in the Resources section.

User Interface

Bootstrap

NSTI takes advantage of the Bootstrap suite that has a large number of JavaScript and CSS files defining the overall style of the User Interface. The following files apply to its import into NSTI:

/nsti/nsti/static/css/ contains the following files that Bootstrap uses:

- bootstrap-theme.css
- bootstrap-theme.min.css
- bootstrap.css
- bootstrap.min.css

/nsti/nsti/static/js/ contains additional JavaScript files that Bootstrap uses:

- bootstrap.js
- bootstrap.min.js

If you want to know how Bootstrap works or how these files are accessed and utilized visit the Bootstrap Documentation page (currently v3.3.2, NSTI uses v3.0.0) here:

http://getbootstrap.com/

All of the viewable HTML, CSS and other parts of the page are displayed and classified by the above bootstrap files. The color schemes are also part of the bootstrap suite.

Trap Tables

The User Interface table in NSTI are directly linked the similarly named tables in the database. To access the different tables in the interface the user will select one of the table tabs on the top left of the NSTI User Interface as shown here:

Each time your press the above buttons the Trap Display table is reloaded with new data via jquery and redrawn. The Normal table will query the snmptt database table, the Archive table will query the snmptt_archive database table and the Unknown table will query the snmptt_unknown database table.

Here is the query that is run when each button is clicked in the User Interface:

$.getJSON(api_url, pr, function(data) {
    TRAPS = data;
    load_body();
});

The api_url is what determines which database table we will get our traps from. Here is what the above variable query will look like during the request:

var api_url = BASE_URL   ’api/trapview/read/’   TABLE;

Here, TABLE will be set by the current view that is being displayed. That is how NSTI will display trap data. A common problem for users who have traps incoming to NSTI and don’t see anything in the Normal table. It is more than likely the traps were set as unknown by the trap handler and are in the Unknown table as intended.

This is all made possible by the @app.route statements that allow the table to be read, deleted, archived and viewed. To view the definitions of the routes for the Trap Table check this file /nsti/nsti/trapview.py

Trap Filters

Filtering of traps was added to NSTI 3.0.0 and was the main reason why the program was rewritten from NSTI v1.4.

When applying a filter to the Trap Table in NSTI there will be another Ajax call to the backend that will return the filtered trap data to be displayed in the same way that the above Trap Tables are loaded. This is also done by using the @app.route system in Flask that was explain above in the Trap Tables section. There are differences, but the filtering of traps is almost exactly the same as the reading of traps.

In /nsti/nsti/filters.py you will see all the defined filter routes that have actions such as list, create, edit, delete, add-active, read-active, delete-active. All these protocols are made possible by the route system using Flask and Werkzeug in the same way that trapview.py allows the user to view the traps in the database.

Atoms

Filters use atoms to apply filters to each one or multiple columns. An atom is actually a data structure for the definition of a filter so that when the data is being requested the filters can be applied and any data that still remains is the data we want to return to the Trap Table.

@app.route(’/api/filter/read’, methods=[’GET’, ’POST’, ’OPTIONS’])
@origin_access_allow_all
def read_filter():
filters = {}
json_result = {’filters’: filters}
where_clause = db.sql_where_query(db.Filter, request.args)

if where_clause:
db_filters = db.DB.find(db.Filter, where_clause)
else:
db_filters = db.DB.find(db.Filter)

for f in db_filters:
j = {’id’: f.id}
action = []
for atom in db.DB.find(db.FilterAtom, db.FilterAtom.filter_id == f.id):
action.append({’column_name’: atom.column_name,
’comparison’: atom.comparison,
’value’: atom.val})
j[’actions’] = action
filters[f.name] = j

json_str = json.dumps(json_result)
return Response(response=json_str, status=200, mimetype=’application/json’)

As you see above ’for atom in db.DB...’ will execute the combined search to be passed which will return the data we are trying to filter down to. This is done using an Ajax call using the atom and the comparison values and functions located in /nist/nsti/database.py to do the following:

- Parse relative and local timewritten (lines 128-153)
- Parse the query using prepare_query_tuple (lines 156-180), a time and query sanity check
- get_queryable_keys (lines 183-206), explained:

’’’Gets the queryable columns from a dictionary. Looks to see if they are
  valid lookups in the database, and if they are, returns all valid search
  result.

 @param arguments - The raw arguments, usually the request variables.
 @param traptype - The type of the trap
 @returns - A list containing the valid queryable columns.
 ’’’

Execute the sql_where_query (lines 229-278), this is the bread and butter of the back-end request and API querying system so seeing the code greater describes the query actions:

def sql_where_query(traptype, arguments, parsed_filters=None, force_combiner=None):
    ’’’Gets the actual query function that will be passed to find
    given the arguments we are searching for.

    @param traptype - The object that will be queried
    @param arguments - Dictionary that holds the key values for searching

    ’’’
    if parsed_filters is None:
        parsed_filters = []

    logging.debug(’Entering sql_where_query...’)
    query = []
    queryable = get_queryable_keys(traptype, arguments, parsed_filters)
    combiner = get_combiner(arguments, force_combiner)
    safe_queryable = pick_non_columns(traptype, queryable)

    logging.debug(’Filtering with: %r’, safe_queryable)

    for key, value in safe_queryable:
        attribute = key

        if key.endswith(’__contains’):
            new_key = key.replace(’__contains’, ’’)
            attribute = getattr(traptype, new_key)
            query.append(attribute.like(u’%%%s%%’ % unicode(value)))
        elif key.endswith(’__in’):
            new_key = key.replace(’__in’, ’’)
            attribute = getattr(traptype, new_key)
            query.append(attribute.is_in(value))
        elif key.endswith(’__gt’):
            new_key = key.replace(’__gt’, ’’)
            attribute = getattr(traptype, new_key)
            query.append(attribute > value)
        elif key.endswith(’__lt’):
            new_key = key.replace(’__lt’, ’’)
            attribute = getattr(traptype, new_key)
            query.append(attribute < value)
        #~ Otherwise we want to do an exact match
        else:
            attribute = getattr(traptype, key)
            if(key in [’id’]):
                query.append(attribute == int(value))
            else:
                query.append(attribute == unicode(value))

    if not query:
        return combiner(True)
    else:
        return combiner(*query)

Here you can see the actual definitions of the __contains and other comparison options and how the Python handles it in the database.

Storm

In the same file that runs the database logic, /nist/nsti/database.py, the key to the structure and database naming convention which allows NSTI to query traps with more advanced logic (one to one, one to many and many to many comparisons, also constructed queries with IN, AND, OR logic, etc. ) for example is Storm. This is the final piece that Python will need to finish the structure of Nagios SNMP Trap Interface.

from nsti import app
import logging
import datetime
import storm.locals as SL
import filters
from flask import session

from storm.tracer import debug
import storm.store
import storm.expr


debug(False)  # The flag enables or disables statement logging


LIMIT = app.config.get(’PERPAGE’, 50)

#~ Setup our DB connect string for Storm
db_connect = ’%s://%s:%s@%s:%s/%s’ % (app.config.get(’DB_TYPE’),
                                      app.config.get(’DB_USER’),
                                      app.config.get(’DB_PASS’),
                                      app.config.get(’DB_HOST’),
                                      app.config.get(’DB_PORT’),
                                      app.config.get(’DB_NAME’))

#~ Establish the database connection
DATABASE = SL.create_database(db_connect)
DB = SL.Store(DATABASE)


class Snmptt(object):
    __storm_table__ = ’snmptt’
    id = SL.Int(primary=True)
    eventname = SL.Unicode()
    eventid = SL.Unicode()
    trapoid = SL.Unicode()
    enterprise = SL.Unicode()
    community = SL.Unicode()
    hostname = SL.Unicode()
    agentip = SL.Unicode()
    category = SL.Unicode()
    severity = SL.Unicode()
    uptime = SL.Unicode()
    traptime = SL.Unicode()
    formatline = SL.Unicode()
    trapread = SL.Int()
    timewritten = SL.DateTime()

...

The last section is the class (definition) of of the Snmptt protocol that NSTI uses to allow users to connect to the API.

The above only includes the code for the Snmptt class object from storm, but there is one definition for each table in the NSTI MySQL database that we have seen before with this image:

Each will have a class db_table_name(object): definition. Flask and Storm make the API and database Ajax queries work the way they do and ultimately together connect the database to the User Interface.

Importing the proper modules and storm dependencies including the python import storm.locals as SL and import filters which allow us a lot of our API comparisons and logic based queries. Then the SL variable is acted on throughout the file as the database object:

#~ Establish the database connection
DATABASE = SL.create_database(db_connect)
DB = SL.Store(DATABASE)

Flask will import the session variable and then the database.py file will load a number of storm._ modules including storm.store, for example, at the top which will allow our database to store data in a cache to save time loading pages or tables that are being viewed over and over again.

Tags

Tags are the physical notification that a filter is being applied to the table. You will see a filter, or multiple filters, displayed to the left directly above the trap table. You can apply multiple filters to a table and the tags will allow the user to see what filters are active or to be able to remove a filter while keeping other active. It is also a good general visual cue that filters are applied.

Here is an example of filter tags being applied to the trap table:

This is a good example of tags since when the user adds the IP address filter to only contain "192.168.5.41" AND for Severity to only contain CRITICAL. As you can see the filter resulted in ONE trap that satisfies both filters in the database. Above you can see the green tags that are displayed. This means that the ’IP address’ and the ’Critical Only’ filters are active and applied to the trap table. Then, if a user wants to remove a filter they just need to click the ’X’ on the green filter tag to remove it from the table. Then the table will be redrawn.

Another good point to be made is that a user can create a filter with multiple fields in it. The above picture can be achieved by creating a single filter with BOTH the IP address and the Critical Severity, but this is up to the user to decide how the filters will be needed and if they will require separation. It should just be noted that you can create a filter with many different parts or create many filters with fewer deliminators. Either way you will get the job done. The easiest method is to build multiple filters and test to see which work with your specific needs.

Trap Data Graphing

Nagios SNMP Trap Interface has the ability to read the database and display the data in the current trap table into a graph. This graph is created using Highcharts (v3.0.9) and is another useful way that NSTI utilizes the API.

Above the trap table on any page there is an expandable bar name "SNMP Trap Data Visualizer"

Click the expand button and it will drop down a graph form that will allow the user to select a column of data to ’Graph By’, calculate and then display the data in a bar graph and a pie graph. The data scales well to a point, but when you get to thousands of traps the graphs become hard to read. This can be adjusted in the graph settings to limit the number of key: data pairs that are returned and drawn in the graph. You could do this by filtering traps by Date, ID, Severity, etc.

Here is an example view with test traps in our database. This is the Host column and good picture of the different hosts sending traps into NSTI. This could be a good way to verify quickly that you SNMP Trap Sender is successfully sending traps to our receiver and snmptrapd is writing them to the database:

This is described in more detail in the Resources section.

Planned Updates to SNMP Trap Data Visualizer

This is just one example, but the trap data visualizer is planned on being expanded over the next few versions of NSTI (v3.0.3 ). Possible updates could include adding D3 graphs for comparison over multiple data set, add multiple search fields to pair against data columns, allow users to customize their graphing parameters including: data types, vertical/horizontal graphs, graphing methods (line, circle, bar, pie, charts, histograms, etc.), custom X/Y data sets/types/formatting and possibly the most important is reports based of these graphs that can be saved imported and exported and even displayed into reports with the option to add trap tables and then have them parsed to wkhtmltopdf from the user interface. This will also be extended with the use of the Nagios SNMP Trap Interface Integration Component for Nagios XI (in development).

Troubleshooting, Common Issues and Miscellaneous

The first steps in troubleshooting Nagios SNMP Trap Interface should be the tail of Apache logs and the restarting of services. First check the apache access and error logs, if you are getting connection errors and Ajax calls that return no data it is more than likely going to show up here:

tail -f /var/log/httpd/*

These logs should point you in the right direction. Pay close attention to the files that are throwing the error, since NSTI is small it should be easy to track down the problem. More than any other issue with NSTI you will need to restart Apache and MySQL and then the User Interface will load without problems:

service httpd restart
service mysqld restart

There is also a very convenient error callback system that is designed to display the files starting and ending with the original request, here is an example from the Flask website of what it might look like:

Notice this feature also doubles as a console so you can debug the issue and get all the information at the same time.

Hardware Recommendations

Running Nagios SNMP Trap Interface does not require sophisticated resources or machines to run. A server will just need enough memory to display the trap table and read the database at an acceptable speed. The database itself and how much space or redundancy is up to the SNMP environment and how many traps you are sending/writing per day/hour/second.

It is important to note that if you are receiving large numbers of traps at a time you will need more resources to handle this. It is recommended that if you have a large trap sending environment and it is possible you should increase the resources on the NSTI machine until you get to a point where the interface loads without too long of a delay and this will reflect how fast the database is being read and displayed.

Issues

Trap Loss and Multi-Threading

It also is known that a large number of traps being sent in to the same snmptrapd receiver all at the same time has in the past been shown to lose some traps during processing where a number of them waiting to be written to the spool fail to get there so snmptt will fail to parse them and they will not reach the database. This is not very common and only present on the larger environments, but something that is noted here so there can be some research done to be able to fix this caveat.

The developers of Net-SNMP also verified that the use of multi-threading is supported in SNMPv1 and SNMPv2, but not v3 for now since support for SNMPv3 and multi-threading we added at the same time. It also claims that you should always only use one thread for processing trap data (generating traps, processing MIBs) or "doing anything SNMP-related" when using the Net-SNMP agent:

http://www.net-snmp.org/wiki/index.php/FAQ:General_19

"MySQL has gone away" - Storm Connection Error

MySQL has been known to lose it’s connection to due being idle for too long. This has been known to happen during periods of NSTI running with not interaction for trap handlers or the User Interface. One of the errors that points to this problem would be similar to the following:

[Thu Jan 08 09:16:44 2015] [error] File "/usr/lib64/python2.6/site-packages/storm/database.py", line 459, in _check_disconnect
[Thu Jan 08 09:16:44 2015] [error] raise DisconnectionError(str(exc))
[Thu Jan 08 09:16:44 2015] [error] Disconnection Error: (2006, ’MySQL has gone away’)

The key out of this error message is the DisconnectionError and ’MySQL has gone away.’ This is referring to when NSTI is opening a connection to the database using SL.Store(DATABASE) on line 28 of database.py whenever a page load or action is executed. Then this database connection will stay open, but MySQL will eventually become unavailable due to network traffic, physical connection problems, socket issues, etc. (there are many reasons why a DB connection might fail).

The developers of Storm made a note of this bug, but it still comes up from time to time in the version we use 0.20 (current version 0.21), but this was said to be fixed in a much earlier version:

https://bugs.launchpad.net/storm/ bug/94986

This might be a future fix that will be made in the database file to use a rollback that would catch any disconnects, but more examples and more testing is needed before any changes can be made.

Traps: Overall Troubleshooting

Steps to verify Trap Configuration

netstat

This tool will allow you to verify that our trap receiver, snmptrapd, is listening on the default SNMP Trap port of 162. If it is listening you know that you are able to receive traps that are coming in.

Run the following command:

netstat -ln

netstat -ln | grep 162

You should see output similar to the below (in this example snmptrapd is using allow all as you can see by the *):

This is a success/output message from running a command:

udp 0 0 0.0.0.0:162 0.0.0.0:*

This verifies that snmptrapd is listening.

tcpdump

This is the next step to ensuring your SNMP configuration is working correctly. This will alow you to dump out the snmp trap data that you are incoming to the NSTI server. This is the proof that the sender is doing its job.

Run this command:

tcpdump -i eth0 -T snmp port 162

This should output something similar to the following depending on the OID or object you are querying. Make sure to use the correct version of SNMP and ensure the community string is correct (usually ’public’ if an open network):

06:34:11.801190 IP 192.168.4.253.52240 > 192.168.4.35.snmptrap: V2Trap(105) system.sysUpTime.0=111930181 S:1.1.4.1.0=S:1.1.5.3 interfaces.ifTable.ifEntry.ifIndex=3 interfaces.ifTable.ifEntry.ifAdminStatus=2 interfaces.ifTable.ifEntry.ifOperStatus=2

This is a good example of how a trap will look coming into NSTI before it is displayed in the User Interface trap table. If you see a message like this you have just verified that traps are coming in successfully.

Unknown Traps

When you install NSTI one of the common problems people have is getting the traps into the Normal trap table, also known as the snmptt table and as Snmptt through the Storm API. Any time you get a trap into the normal table it has been read and determined that the trap is expected because it is located in the snmptt.conf file. This file will be read by snmptt.ini and is defined on line 620 at the time this article was written:

[TrapFiles]
# A list of snmptt.conf files (this is NOT the snmptrapd.conf file). The COMPLETE path
# and filename. Ex: ’/etc/snmp/snmptt.conf’
snmptt_conf_files = <<END
/etc/snmp/snmptt.conf
END

If you want to indicate multiple different conf files, which will be very common when using MIB parsers such as the Perl snmpttconvertmib, which creates a new .conf file for every time it is run if indicated to do so, you will just need to add a file path to that file after a new line between each .conf file.

snmptt.conf

The snmptt configuration file is how snmptt will know where to parse and write traps that are being sent to the /var/snmptt/spool directory. This is the ’read snmptt configuration files’ section of the NSTI workflow. This will determine if a Trap/OID was expected by matching the trap definition to the definitions written into snmptt.conf. If it was found and matched to then write that trap into the normal snmptt database table, but if the trap definition was not found then SNMP Trap Translator will write the trap to the snmptt_unknown database as long as the snmptt.ini file is inidcating it should write Unknown traps at all. On line 312 of snmptt.ini this is set by:

unknown_trap_log_enable = 1

Here is an example snmptt.conf file that will have actionable trap definitions (this is a small and simple example, just to illustrate the formatting):

#
#
#
EVENT coldStart .1.3.6.1.6.3.1.1.5.1 "Status Events" Normal
FORMAT Device reinitialized (coldStart)
#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (coldStart)"
SDESC
A coldStart trap signifies that the SNMPv2 entity, acting
in an agent role, is reinitializing itself and that its
configuration may have been altered.
EDESC
#
#
#
EVENT warmStart .1.3.6.1.6.3.1.1.5.2 "Status Events" Normal
FORMAT Device reinitialized (warmStart)
#EXEC qpage -f TRAP notifygroup1 "Device reinitialized (warmStart)"
SDESC
A warmStart trap signifies that the SNMPv2 entity, acting
in an agent role, is reinitializing itself such that its
configuration is unaltered.
EDESC
#
#
#
EVENT linkDown .1.3.6.1.6.3.1.1.5.3 "Status Events" Normal
FORMAT Link down on interface $1. Admin state: $2. Operational state: $3
#EXEC qpage -f TRAP notifygroup1 "Link down on interface $1. Admin state: $2. Operational state: $3"
SDESC
A linkDown trap signifies that the SNMP entity, acting in
an agent role, has detected that the ifOperStatus object for
one of its communication links is about to enter the down
state from some other state (but not from the notPresent
state). This other state is indicated by the included value
of ifOperStatus.
EDESC
#
#
#
EVENT linkUp .1.3.6.1.6.3.1.1.5.4 "Status Events" Normal
FORMAT Link up on interface $1. Admin state: $2. Operational state: $3
#EXEC qpage -f TRAP notifygroup1 "Link up on interface $1. Admin state: $2. Operational state: $3"
SDESC
A linkUp trap signifies that the SNMP entity, acting in an
agent role, has detected that the ifOperStatus object for
one of its communication links left the down state and
transitioned into some other state (but not into the
notPresent state). This other state is indicated by the
included value of ifOperStatus.
EDESC
#
#
#

EVENT, FORMAT, EXEC are the parts that will allow snmptt to assign the incoming traps that match to the proper database. Above you can see the #EXEC lines are commented out with the bang symbol (#), EXEC lines are only needed for SNMP traps if you plan on using another program to handle the trap after snmptt finishes its processing.

Processing MIBs

Populating this file by hand is unnecessary since there are many ways to accomplish parsing multiple MIB files into the snmptt.conf file.

In Nagios XI SNMP integration is an important to and there has been MIB management documentation written for supporting it. I recommend the addmib() script that Nagios created to do just that. Use this document to access the actual script, but do not run the automated script since it will remove the original configuration of snmptt made by NSTI during install:

Documentation - How to Integrate SNMP Traps With Nagios XI

The Installing MIBs section is where the addmib script is described. The actual script is located in the file that the document tells you to wget. Get this package and then use the script from it. This is a great tool for adding all the MIB files you require and remember that if some MIBs fail it’s because they use other MIBs as a dependency so the order you add them will matter.

The other option is the previously mentioned snmpttconvertmib Perl script created by SNMPTT. This will do largely the same thing as the addmib script in the above link and convert TRAP-TYPE and NOTIFICATION-TYPE to something that snmptt can read:

http://snmptt.sourceforge.net/docs/snmpttconvertmib.shtml

Once you are getting the traps in question into the normal table you have succeeded! Make sure to check your firewall if you are still having trouble. The Integrating SNMP Trap with Nagios XI document above has a section about the firewall and allowing data over port 162.

SELinux

In a few cases during testing there had been problems with MySQL being allowed to write to the snmptt database table due to SELinux being enabled and not allowing the proper snmptt user to write into MySQL. Here is a failed write in the /var/log/audit/audit.log:

type=AVC msg=audit(1423065923.604:1726): avc: denied { write } for pid=5259 comm="snmptt" name="mysql.sock" dev=dm-0 ino=450 scontext=unconfined_u:system_r:snmpd_t:s0 tcontext=system_u:object_r:mysqld_var_run_t:s0 tclass=sock_file

The immediate fix would be to disable SELinux and reboot the server. This might not be an option with many users so the best way to troubleshoot this problem is to troubleshoot how snmptt is being called and what user will be allowed to write to the MySQL database.

Bugs and Feature Requests

If you find a bug or a feature request that is not already present in the Github list of Issues then you can add them to Github via the link below:

https://github.com/NagiosEnterprises/nsti/issues

For bugs explain the issue, show steps to recreate the bug and then give it a red ’bug’ tag. For features explain what is desired and in as much detail as possible, how the feature would be implemented in Nagios SNMP Trap Interface and how the feature should work in the back end if there are specifics, and finally add a blue ’enhancement’ tag to the request.

Updates From Previous Versions

The previous stable version of NSTI is version 1.4 which has a number of similarities, but needed a way to be able to have saved filters inside the database. This resulted in splitting the original snmptt database into 3 separate tables and adding 2 more tables to manage filters.

There are plans to have upgrade scripts for NSTI v1.4 so that people who rely on it will be able to import all of their traps into the new NSTI database and not lose any historical data that might need to be preserved. This should also carry over any of the first generation filters that were added into NSTI 1.4 and should carry over to 3.0.2 without any issues.

Here is a direct download link from Nagios assets for Nagios SNMP Trap Interface v1.4 (It isn’t that easy to find these days): http://assets.nagios.com/downloads/nagiosti/downloads/nsti-rc1.4.tar.gz

The distinct differences between NSTI 1.4 and 3.0.2 are explained here. There are a number of changes between the two similarly Python driven versions. Here is a list: