Hi all,
I think there is a little issue with the way nagios processes checks sent using nagcmd.
Let me explain. We use SNMPPT to check for LinkDown and LinkUp traps from our switches. If we have a circuit go down and then come up very soon afterwards (like one or two seconds later), SNMPTT seems like it processes the traps correctly and in the right order, but it only spins the directory every 30 seconds. So when traps are processed it will process both a linkdown and linkup trap in rapid succession and correctly fire off the passive check script in the right order. I know this because I have put logging on the script and I can correctly see linkdown and linkup alerts in the right order and they correlate to the same order as I see in snmptrapd log. The problem is that the nagcmd does not seem to be taking the order in which it receives it from the script. Nagios seems to process them randomly, more often than not it process them in the wrong order. Sounds like something within nagios has multiple processes watching nagcmd and if one process is a little slower than the other it then gets processed slower and therefore reported in nagios in the incorrect order.
Has anyone else seen this issue and might know how it can be fixed ?
Thanks
Nagios Processing Passive Checks in wrong order
-
liquidcool
- Posts: 59
- Joined: Tue Feb 21, 2012 6:08 am
Re: Nagios Processing Passive Checks in wrong order
First, check to make sure that you do not have any multiple parents of the nagios process:
Are you using mklivestatus or any other 3rd party brokers?
Code: Select all
service nagios stop
killall nagios
ps -aef | grep nagios
service nagios status
service nagios startFormer Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
-
liquidcool
- Posts: 59
- Joined: Tue Feb 21, 2012 6:08 am
Re: Nagios Processing Passive Checks in wrong order
There are no multiple processes. Just one
-
scottwilkerson
- DevOps Engineer
- Posts: 19396
- Joined: Tue Nov 15, 2011 3:11 pm
- Location: Nagios Enterprises
- Contact:
Re: Nagios Processing Passive Checks in wrong order
abrist wrote:Are you using mklivestatus or any other 3rd party brokers?