After a commit - "ipcs -q" messages is slow to zero out

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
SteveBeauchemin
Posts: 524
Joined: Mon Oct 14, 2013 7:19 pm

After a commit - "ipcs -q" messages is slow to zero out

Post by SteveBeauchemin »

This is not new. This is what we consider normal based on current experiences in Nagios XI. I just need to ask.

After I make changes in the Nagios XI GUI, adding or removing hosts, services, or anything, I run an "Apply Configuration."
If I am watching the ipcs queue I see the messages count go from zero to 270K or thereabouts. Then it counts back to zero slowly over the next 10 to 30 minutes.
Once again, we call this normal here as this is the behavior I have been seeing for a very long time. I usually run this to see it.

Code: Select all

watch -n 4 "ipcs -q"
But is it normal?? Why does it have to buffer all those messages when doing a commit of changes. It is as though the entire set of data has to be sent to the DB again, versus just sending a delta of the changes. Sometimes I see the messages grow to very large backlog that needs to be sent to MySQL. I have seen it grow to over a million messages in the queue one time.

Is this one of the items being addressed in future code that will make updates not stop Nagios status page from displaying while an update happens?

If not, should it be?

My system is not broken. It is working quite nicely. I just wonder about these things sometimes. Sometimes I wonder what normal is.

Thanks

Steve B
XI 5.7.3 / Core 4.4.6 / NagVis 1.9.8 / LiveStatus 1.5.0p11 / RRDCached 1.7.0 / Redis 3.2.8 /
SNMPTT / Gearman 0.33-7 / Mod_Gearman 3.0.7 / NLS 2.0.8 / NNA 2.3.1 /
NSClient 0.5.0 / NRPE Solaris 3.2.1 Linux 3.2.1 HPUX 3.2.1
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: After a commit - "ipcs -q" messages is slow to zero out

Post by tmcdonald »

SteveBeauchemin wrote:It is as though the entire set of data has to be sent to the DB again, versus just sending a delta of the changes.
That's what is going on, but it's not XI that is causing it, just a restart of Core. If you do service nagios restart on its own you will see the same thing. Basically when Core starts up it sends its config to the kernel message queue, then npcd processes it and updates the DB.

As for addressing this in the future, I would need to defer to our Core dev on that as it would be a change in Core, NDO, and ndomod.
Former Nagios employee
jfrickson

Re: After a commit - "ipcs -q" messages is slow to zero out

Post by jfrickson »

I'm hoping to implement live changes in Core5. One of the things that will avoid is all the database updates from a restart.

So yes, it's normal, and yes, we're going to try to change that behavior.
SteveBeauchemin
Posts: 524
Joined: Mon Oct 14, 2013 7:19 pm

Re: After a commit - "ipcs -q" messages is slow to zero out

Post by SteveBeauchemin »

Thanks for the information. I like knowing how things work under the covers.
I suspect that it is not npcd processing it and updating the database. Probably ndo2db.
In any case, feel free to close this.

Steve B
XI 5.7.3 / Core 4.4.6 / NagVis 1.9.8 / LiveStatus 1.5.0p11 / RRDCached 1.7.0 / Redis 3.2.8 /
SNMPTT / Gearman 0.33-7 / Mod_Gearman 3.0.7 / NLS 2.0.8 / NNA 2.3.1 /
NSClient 0.5.0 / NRPE Solaris 3.2.1 Linux 3.2.1 HPUX 3.2.1
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

Re: After a commit - "ipcs -q" messages is slow to zero out

Post by WillemDH »

I'm hoping to implement live changes in Core5. One of the things that will avoid is all the database updates from a restart.
Yes please please thank you!!!! :)
Nagios XI 5.8.1
https://outsideit.net
Locked