Post XI5 upgrade -> Apply Config -> ndo2db: queue send error
Posted: Thu Jan 28, 2016 5:22 pm
Finally got upgraded to Nagios XI5. Was on 2014 V2.7 ... Funny little story to tell...
Fixed the custom-dashboard issue by putting the older custom-dashlet in place.
Repaired my sudoers file that upgrade changes.
Made sure everything works. And it does. But...
Whenever I make a CCM change and apply it, the log starts scrolling and tells me that ndo2db has a problem.
from /var/log/messages
Jan 28 12:57:05 crulnxicore01 ndo2db: Warning: queue send error, retrying...
Jan 28 12:57:06 crulnxicore01 ndo2db: Message sent to queue.
Jan 28 12:57:06 crulnxicore01 ndo2db: Warning: queue send error, retrying...
Jan 28 12:57:07 crulnxicore01 ndo2db: Message sent to queue.
forever...
Until I run "service ndo2db restart" and then everything is fine again.
So, I checked for stuff based on my original install notes and the system changes I introduced to my setup that made things work the way I needed them to. I discovered that the /etc/sysctl.conf file happened to have a new time stamp, and my kernel.msgmnb setting had been changed. To a much smaller number. 1/3 the size actually.
I would like to make a suggestion for future installations and updates. If an existing setting in an OS configuration file is set to a number, and you need to increase that setting, go for it. If your update will decrease the value, DON'T DO IT. Please. My hair is already gray.
/tmp/nagiosxi/subcomponents/ndoutils is where the script is - found it finally...
The setting changed by post-upgrade doesn't take into account that I already fought through my SQL issues and got the system to work the way my installation needs it to. My large installation...
kernel.msgmnb = 131072000 is too small for my installation.
Anyway, bottom line is I'm on XI5 finally. Found and fixed what I needed to fix so far.
Thanks
Steve B
Fixed the custom-dashboard issue by putting the older custom-dashlet in place.
Repaired my sudoers file that upgrade changes.
Made sure everything works. And it does. But...
Whenever I make a CCM change and apply it, the log starts scrolling and tells me that ndo2db has a problem.
from /var/log/messages
Jan 28 12:57:05 crulnxicore01 ndo2db: Warning: queue send error, retrying...
Jan 28 12:57:06 crulnxicore01 ndo2db: Message sent to queue.
Jan 28 12:57:06 crulnxicore01 ndo2db: Warning: queue send error, retrying...
Jan 28 12:57:07 crulnxicore01 ndo2db: Message sent to queue.
forever...
Until I run "service ndo2db restart" and then everything is fine again.
So, I checked for stuff based on my original install notes and the system changes I introduced to my setup that made things work the way I needed them to. I discovered that the /etc/sysctl.conf file happened to have a new time stamp, and my kernel.msgmnb setting had been changed. To a much smaller number. 1/3 the size actually.
I would like to make a suggestion for future installations and updates. If an existing setting in an OS configuration file is set to a number, and you need to increase that setting, go for it. If your update will decrease the value, DON'T DO IT. Please. My hair is already gray.
/tmp/nagiosxi/subcomponents/ndoutils is where the script is - found it finally...
The setting changed by post-upgrade doesn't take into account that I already fought through my SQL issues and got the system to work the way my installation needs it to. My large installation...
kernel.msgmnb = 131072000 is too small for my installation.
Anyway, bottom line is I'm on XI5 finally. Found and fixed what I needed to fix so far.
Thanks
Steve B