How to monitor remote hosts by sending Traps to Nagios

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
newbe
Posts: 17
Joined: Wed Dec 09, 2015 11:10 am
Location: Germany

Re: How to monitor remote hosts by sending Traps to Nagios

Post by newbe »

this is /etc/snmp/snmpd.conf

Code: Select all

###############################################################################
# Access Control
###############################################################################
#       sec.name  source          community
com2sec disman    localhost       public
com2sec mynetwork 192.168.1.1     public
com2sec mynetwork 192.168.2.0/24  public
com2sec mynetwork 20.10.0.0/16    public
####
# Second, map the security names into group names:
#      sec.model         sec.name
group MyRWGroup v1         disman
group MyRWGroup v2c        disman
group MyRWGroup usm        disman
group MyROGroup v1         mynetwork
group MyROGroup v2c        mynetwork
group MyROGroup usm        mynetwork
####
# Third, create a view for us to let the groups have rights to:
#           incl/excl subtree                          mask
view all    included  .1                               80
####
# Finally, grant the 2 groups access to the 1 view with different
# write permissions:
#                context sec.model sec.level match  read   write  notif
access MyROGroup ""      any       noauth    exact  all    none   none
access MyRWGroup ""      any       auth      exact  all    all    none
###############################################################################
# Process checks.
#
#  The following are examples of how to use the agent to check for
#  processes running on the host.  The syntax looks something like:
#
#  proc NAME [MAX=0] [MIN=0]
#
#  NAME:  the name of the process to check for.  It must match
#         exactly (ie, http will not find httpd processes).
#  MAX:   the maximum number allowed to be running.  Defaults to 0.
#  MIN:   the minimum number to be running.  Defaults to 0.
#
#  Examples:
#
#  Make sure at least one sendmail, but less than or equal to 10 are running.
proc sendmail 10 1
proc sshd
proc cron
###############################################################################
# disk checks
#
# disk PATH [MIN=DEFDISKMINIMUMSPACE]
#
# PATH:  mount path to the disk in question.
# MIN:   Disks with space below this value will have the Mib's errorFlag set.
#        Default value = DEFDISKMINIMUMSPACE.
# Check the / partition and make sure it contains at least 10 megs.
disk / 10000000
#disk       /var  5%
#includeAllDisks  10%
###############################################################################
# load average checks
#
# load [1MAX=DEFMAXLOADAVE] [5MAX=DEFMAXLOADAVE] [15MAX=DEFMAXLOADAVE]
#
# 1MAX:   If the 1 minute load average is above this limit at query
#         time, the errorFlag will be set.
# 5MAX:   Similar, but for 5 min average.
# 15MAX:  Similar, but for 15 min average.
# Check for loads:
load 12 10 5
# -----------------------------------------------------------------------------
# Inteface up and down
# Trap server information
#trapsink    20.10.43.3 public
#trap2sink   localhost  public
informsink  localhost public
trapcommunity public
authtrapenable 1
rouser   	authOnlyUser
iquerySecName   authOnlyUser       
defaultMonitors          yes
linkUpDownNotifications  no

monitor -u disman -t -r 30 -o dskPath -o dskAvail -o dskTotal "dskTable" dskErrorFlag 0 1
#monitor -u disman -t -r 30 -o dskPath -o dskErrorMsg "dskTable" dskErrorFlag != 0
#monitor machineTooBusy hrProcessorLoad > 90
###########################################################################
# SECTION: Agent Operating Mode
#
master          agentx



#agentAddress udp:161

dontLogTCPWrappersConnects 1


/etc/snmp/snmptrapd.conf

Code: Select all

authCommunity log, net, execute public
traphandle default /usr/sbin/snmptthandler
disableAuthorization yes

/etc/snmp/snmp.conf

Code: Select all

# As the snmp packages come without MIB files due to license reasons, loading
# of MIBs is disabled by default. If you added the MIBs you can reenable
# loading them by commenting out the following line.
mibs IF-MIB:UCD-SNMP-MIB
# noTokenWarnings yes
#mibs +ALL
newbe
Posts: 17
Joined: Wed Dec 09, 2015 11:10 am
Location: Germany

Re: How to monitor remote hosts by sending Traps to Nagios

Post by newbe »

after keepin on working out, now im getting traps in syslog using -> "dskTable" - UCD-SNMP-MIB.
I guess to read the CPU Load from HOST-RESOURCES-MIB Objects will be difficult. Any recommends on how to do that?

in syslog there are still a couple of errors i couldnt fix yet. im workin with virtualbox and i dont have set up static ip.
could be this the reason why im getting errors with snmptrapd in line 20 ?? :roll:

Code: Select all

Jan 10 15:48:09 rf1-VB snmptrapd[3503]: AgentX master disconnected us, reconnecting in 15
Jan 10 15:48:09 rf1-VB snmptrapd[3503]: 2016-01-10 15:48:09 NET-SNMP version 5.7.2 Stopped.
Jan 10 15:48:09 rf1-VB snmptrapd[3503]: Stopping snmptrapd
Jan 10 15:48:46 rf1-VB snmpd[3592]: Turning on AgentX master support.
Jan 10 15:48:46 rf1-VB snmpd[3595]: NET-SNMP version 5.7.2
Jan 10 15:48:46 rf1-VB snmptrapd[3594]: getaddrinfo: execute Name or service not known
Jan 10 15:48:46 rf1-VB snmptrapd[3594]: /etc/snmp/snmptrapd.conf: line 20: Error: cannot resolve source hostname
Jan 10 15:48:46 rf1-VB snmptrapd[3594]: /etc/snmp/snmptrapd.conf: line 20: Error: Name or service not known
Jan 10 15:48:46 rf1-VB snmptrapd[3594]: /etc/snmp/snmptrapd.conf: line 20: Error: bad SUBTREE object id
Jan 10 15:48:46 rf1-VB snmptrapd[3594]: net-snmp: 3 error(s) in config file(s)
Jan 10 15:48:46 rf1-VB snmptrapd[3594]: NET-SNMP version 5.7.2 AgentX subagent connected
Jan 10 15:48:46 rf1-VB snmpd[3595]: Cannot statfs /run/user/1000/gvfs#012: Permission denied
Jan 10 15:48:46 rf1-VB snmptrapd[3596]: NET-SNMP version 5.7.2
Jan 10 15:48:47 rf1-VB snmptrapd[3596]: localhost [UDP: [127.0.0.1]:48546->[127.0.0.1]:162]: Trap , .1.3.6.1.2.1.1.3.0 = Timeticks: (5) 0:00:00.05, .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.6.3.1.1.5.1, .1.3.6.1.6.3.1.1.4.3.0 = OID: .1.3.6.1.4.1.8072.3.2.10
Jan 10 15:48:47 rf1-VB snmptrapd[3596]: localhost [UDP: [127.0.0.1]:48546->[127.0.0.1]:162]: Trap , .1.3.6.1.2.1.1.3.0 = Timeticks: (7) 0:00:00.07, .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.2.1.88.2.0.1, .1.3.6.1.2.1.88.2.1.1.0 = STRING: "process table", .1.3.6.1.2.1.88.2.1.2.0 = "", .1.3.6.1.2.1.88.2.1.3.0 = "", .1.3.6.1.2.1.88.2.1.4.0 = OID: .1.3.6.1.4.1.2021.2.1.100.1, .1.3.6.1.2.1.88.2.1.5.0 = INTEGER: 1, .1.3.6.1.4.1.2021.2.1.2.1 = STRING: sendmail, .1.3.6.1.4.1.2021.2.1.101.1 = STRING: No sendmail process running
Jan 10 15:48:48 rf1-VB snmptrapd[3596]: localhost [UDP: [127.0.0.1]:48546->[127.0.0.1]:162]: Trap , .1.3.6.1.2.1.1.3.0 = Timeticks: (7) 0:00:00.07, .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.2.1.88.2.0.1, .1.3.6.1.2.1.88.2.1.1.0 = STRING: "process table", .1.3.6.1.2.1.88.2.1.2.0 = "", .1.3.6.1.2.1.88.2.1.3.0 = "", .1.3.6.1.2.1.88.2.1.4.0 = OID: .1.3.6.1.4.1.2021.2.1.100.2, .1.3.6.1.2.1.88.2.1.5.0 = INTEGER: 1, .1.3.6.1.4.1.2021.2.1.2.2 = STRING: sshd, .1.3.6.1.4.1.2021.2.1.101.2 = STRING: No sshd process running
Jan 10 15:48:48 rf1-VB snmptrapd[3596]: localhost [UDP: [127.0.0.1]:48546->[127.0.0.1]:162]: Trap , .1.3.6.1.2.1.1.3.0 = Timeticks: (8) 0:00:00.08, .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.2.1.88.2.0.1, .1.3.6.1.2.1.88.2.1.1.0 = STRING: "dskTable", .1.3.6.1.2.1.88.2.1.2.0 = "", .1.3.6.1.2.1.88.2.1.3.0 = "", .1.3.6.1.2.1.88.2.1.4.0 = OID: .1.3.6.1.4.1.2021.9.1.100.1, .1.3.6.1.2.1.88.2.1.5.0 = INTEGER: 1, .1.3.6.1.4.1.2021.9.1.2.1 = STRING: /, .1.3.6.1.4.1.2021.9.1.101.1 = STRING: /: less than 10000000 free (= 815380)
Jan 10 15:48:48 rf1-VB snmptrapd[3596]: localhost [UDP: [127.0.0.1]:48546->[127.0.0.1]:162]: Trap , .1.3.6.1.2.1.1.3.0 = Timeticks: (8) 0:00:00.08, .1.3.6.1.6.3.1.1.4.1.0 = OID: .1.3.6.1.2.1.88.2.0.2, .1.3.6.1.2.1.88.2.1.1.0 = STRING: "dskTable", .1.3.6.1.2.1.88.2.1.2.0 = "", .1.3.6.1.2.1.88.2.1.3.0 = "", .1.3.6.1.2.1.88.2.1.4.0 = OID: .1.3.6.1.4.1.2021.9.1.100.1, .1.3.6.1.2.1.88.2.1.5.0 = INTEGER: 1, .1.3.6.1.4.1.2021.9.1.2.1 = STRING: /, .1.3.6.1.4.1.2021.9.1.7.1 = INTEGER: 815380, .1.3.6.1.4.1.2021.9.1.6.1 = INTEGER: 6108880
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: How to monitor remote hosts by sending Traps to Nagios

Post by ssax »

Please zip up and attach your /etc/snmp directory on your Core server and on your remote server and attach them please so that I can review them.
User avatar
Box293
Too Basu
Posts: 5126
Joined: Sun Feb 07, 2010 10:55 pm
Location: Deniliquin, Australia
Contact:

Re: How to monitor remote hosts by sending Traps to Nagios

Post by Box293 »

newbe wrote:@Box293

instead of a script which execute the snmptrap and the send a SNMPv3 Trap, DisMan Monitoring (internal monitoring) should be able to
report problems via SNMP notifications. Im trying to work it out in snmpd.conf file, unfortunately without success.
The first source i worked with was and is still http://www.net-snmp.org/wiki/index.php/ ... Monitoring

Any ideas where to look on this topic to get more into details ?

It would be nice to work with something the Snmp Agent offers already. If DisMan would be to complicated i still can use a script.

thank u for your time.
Sorry I've been on holidays and this thread is a bit complicated reading back on.

With traps, the snmpd.conf is not relevant. snmpd is the daemon that listens to SNMP GET requests. You're dealing with traps so you can ignore snmpd.

In your scenario, you are sending an snmptrap from the remote machine using DisMan. DisMan should be responsible for sending the trap and would have it's own SNMP settings, like the details for the trap listener (nagios).

Nagios will use snmptrapd to listen for traps and spool them to a file. snmptrapd.conf on the nagios server will need to be specifically configured for SNMP v3. If you're not needing v3, stay away from it as it over complicates things at this stage.

snmptt will look at the spooled files and process the traps. snmptt.conf will contain the traps it knows of (EVENTS) and from that will exectute a command (EXEC), in nagios that command is to submit the trap to the nagios command pipe as a check result.

I'm used to doing all of this in Nagios XI as it automates a lot of the configuration pieces. It would be easiest to use an XI trial VM and setup traps using the procedure https://assets.nagios.com/downloads/nag ... ios_XI.pdf is it most likely will help you get it working quickly. Once you understand it, then you can configure your core server with traps correctly.

I don't have any definitive guides for this on Core at this stage, something on my to do list but it's a big task. Simple it isn't, until you understand it!
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
newbe
Posts: 17
Joined: Wed Dec 09, 2015 11:10 am
Location: Germany

Re: How to monitor remote hosts by sending Traps to Nagios

Post by newbe »

ssax wrote:Please zip up and attach your /etc/snmp directory on your Core server and on your remote server and attach them please so that I can review them.
i will do very soon.
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: How to monitor remote hosts by sending Traps to Nagios

Post by tmcdonald »

Have you sent over the zip?
Former Nagios employee
newbe
Posts: 17
Joined: Wed Dec 09, 2015 11:10 am
Location: Germany

Re: How to monitor remote hosts by sending Traps to Nagios

Post by newbe »

Box293 wrote:
newbe wrote:@Box293

instead of a script which execute the snmptrap and the send a SNMPv3 Trap, DisMan Monitoring (internal monitoring) should be able to
report problems via SNMP notifications. Im trying to work it out in snmpd.conf file, unfortunately without success.
The first source i worked with was and is still http://www.net-snmp.org/wiki/index.php/ ... Monitoring

Any ideas where to look on this topic to get more into details ?

It would be nice to work with something the Snmp Agent offers already. If DisMan would be to complicated i still can use a script.

thank u for your time.
Sorry I've been on holidays and this thread is a bit complicated reading back on.

-> Sorry for that.

With traps, the snmpd.conf is not relevant. snmpd is the daemon that listens to SNMP GET requests. You're dealing with traps so you can ignore snmpd.
In your scenario, you are sending an snmptrap from the remote machine using DisMan. DisMan should be responsible for sending the trap and would have it's own SNMP settings, like the details for the trap listener (nagios).

->the only DisMan settings i found is in the snmpd.conf http://www.net-snmp.org/docs/man/snmpd.conf.html#lbAX
under DisMan Event MIB Section.


Nagios will use snmptrapd to listen for traps and spool them to a file. snmptrapd.conf on the nagios server will need to be specifically configured for SNMP v3. If you're not needing v3, stay away from it as it over complicates things at this stage.

->DisMan can report problems via SNMPD Notification in SNMPV2 but i found out that trapsess should be able to send snmpv3.
i will keep on working on this,because i need snmpv3. Unfortunaetly!


I'm used to doing all of this in Nagios XI as it automates a lot of the configuration pieces. It would be easiest to use an XI trial VM and setup traps using the procedure https://assets.nagios.com/downloads/nag ... ios_XI.pdf is it most likely will help you get it working quickly. Once you understand it, then you can configure your core server with traps correctly.

-> thank you for the link. it helps alot to understand how to receive traps.


thank you for your time.
newbe
Posts: 17
Joined: Wed Dec 09, 2015 11:10 am
Location: Germany

Re: How to monitor remote hosts by sending Traps to Nagios

Post by newbe »

tmcdonald wrote:Have you sent over the zip?
i upload the /etc/snmp in zip. In the zip there is snmp.conf, snmpd.conf, snmptrapd.conf, snmptt.conf and snmptt.ini. those are the main one, i worked with.
other files like mib2c.* are unnecessary!

at the moment, iam able to send notifications from snmpd - > snmptrapd > snmptthandler -> log files using DisMan Event with "monitor" directive like this
-> monitor -u disman -S -t -r 1 machineTooBusy hrProcessorLoad > 80 and the System Load Monitoring "load" directive like this -> load 12
once i stress my cpu and restart snmpd with (service snmpd restart) it does first fire a notification that snmpd have been reset and 1 minute later it will fire notification that the cpu Load is over 80. ;) so far , so good!

Now, it would be amazing if DisMan could do the same and check lets say every 3 hours for the cpu load of my remote host and fire again a notificaiton to snmptrapd but here comes a problem while using "monitoring" directive :
Note that the event will only be triggered once, when the expression first matches. This monitor entry will not fire again until the monitored condition first becomes false, and then matches again.
-> http://www.net-snmp.org/docs/man/snmpd.conf.html

is anybody familiar with DisMan Event and know how the monitor entry becomes false, so it will trigger the corresponding event again ?

There is also a mechanism for scheduling particular actions (SET assignment) called DisMan Schedule MIB. Anybody familiar with this one ?



I just want to thank y'all for your time and i really do appreciate it.
Attachments
snmp.zip
Snmp Folder on Virtual Box
(69.05 KiB) Downloaded 268 times
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: How to monitor remote hosts by sending Traps to Nagios

Post by tmcdonald »

Questions relating to DisMan probably would be best asked in a forum dedicated to SNMP, as the members there are going to be much more well-versed in that sort of thing.
Former Nagios employee
newbe
Posts: 17
Joined: Wed Dec 09, 2015 11:10 am
Location: Germany

Re: How to monitor remote hosts by sending Traps to Nagios

Post by newbe »

tmcdonald wrote:Questions relating to DisMan probably would be best asked in a forum dedicated to SNMP, as the members there are going to be much more well-versed in that sort of thing.

Thank you for pointing that out. By any chance, do you now a forum ?
Locked