Nagios Processing Passive Checks in wrong order

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
liquidcool
Posts: 59
Joined: Tue Feb 21, 2012 6:08 am

Nagios Processing Passive Checks in wrong order

Post by liquidcool »

Hi all,

I think there is a little issue with the way nagios processes checks sent using nagcmd.

Let me explain. We use SNMPPT to check for LinkDown and LinkUp traps from our switches. If we have a circuit go down and then come up very soon afterwards (like one or two seconds later), SNMPTT seems like it processes the traps correctly and in the right order, but it only spins the directory every 30 seconds. So when traps are processed it will process both a linkdown and linkup trap in rapid succession and correctly fire off the passive check script in the right order. I know this because I have put logging on the script and I can correctly see linkdown and linkup alerts in the right order and they correlate to the same order as I see in snmptrapd log. The problem is that the nagcmd does not seem to be taking the order in which it receives it from the script. Nagios seems to process them randomly, more often than not it process them in the wrong order. Sounds like something within nagios has multiple processes watching nagcmd and if one process is a little slower than the other it then gets processed slower and therefore reported in nagios in the incorrect order.

Has anyone else seen this issue and might know how it can be fixed ?

Thanks
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Nagios Processing Passive Checks in wrong order

Post by abrist »

First, check to make sure that you do not have any multiple parents of the nagios process:

Code: Select all

service nagios stop
killall nagios
ps -aef | grep nagios
service nagios status
service nagios start
Are you using mklivestatus or any other 3rd party brokers?
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
liquidcool
Posts: 59
Joined: Tue Feb 21, 2012 6:08 am

Re: Nagios Processing Passive Checks in wrong order

Post by liquidcool »

There are no multiple processes. Just one
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nagios Processing Passive Checks in wrong order

Post by scottwilkerson »

abrist wrote:Are you using mklivestatus or any other 3rd party brokers?
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
Locked