Page 1 of 1

ndo2db: Warning: queue send error, retrying...

Posted: Wed Mar 11, 2015 1:54 am
by venkitesh
hi Team,

Daily nagios xi is crashing with the error message ndo2db: Warning: queue send error, retrying...

Nagiosxi Version : 2014R2.6

OS: CentOS release 6.5 64 bit

RAM: 6GB

CPU: 4Cores.

Total number of hosts: 620
Total number of services: 300

Sysctl.conf

kernel.msgmnb = 131072000

# Controls the maximum size of a message, in bytes
kernel.msgmax = 131072000

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 4294967295

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 268435456

kernel.msgmni = 256000

ulimit -a

core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 46668
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 46668
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

After making changes i did a reboot also. Daily at night it is crashing with this error message "ndo2db: Warning: queue send error, retrying..."

Kindly help

Re: ndo2db: Warning: queue send error, retrying...

Posted: Wed Mar 11, 2015 10:33 am
by abrist
Is XI actually crashing? Or is that error just present in the logs?
Does this happen at the same time every night? If so, are db backups/etc being run at that time?

Re: ndo2db: Warning: queue send error, retrying...

Posted: Thu Mar 12, 2015 2:03 am
by venkitesh
hi abrist,

the exact requirement is mentioned in this post.

http://support.nagios.com/forum/viewtop ... 9&p=106800

Please help

Re: ndo2db: Warning: queue send error, retrying...

Posted: Thu Mar 12, 2015 2:01 pm
by tgriep
Could you run this and post back the results?

Code: Select all

cat /proc/sys/kernel/msgmni
Also, could you post more of the error log?

Re: ndo2db: Warning: queue send error, retrying...

Posted: Fri Mar 13, 2015 12:07 am
by venkitesh
cat /proc/sys/kernel/msgmni

256000

Now the issue is without doing any thing load came back to normal.

but no service checks are happening.

also last service checks are showing last weeks date.

nagios: Auto-save of retention data completed successfully

Please help .

Re: ndo2db: Warning: queue send error, retrying...

Posted: Fri Mar 13, 2015 12:06 pm
by tgriep
Could you run this and post back the results?

Code: Select all

service ndo2db status

Re: ndo2db: Warning: queue send error, retrying...

Posted: Wed Mar 18, 2015 10:54 am
by tmcdonald
This issue is similar to one that has been affecting several users recently. We are not yet sure of the cause, so we are gathering information to try and determine a common attribute across all affected systems.

Please send us an updated system profile if possible. Go to Admin -> System Profile and click the blue "Download Profile" button. Then attach that profile.zip file to a PM to myself.

Please also run the following commands:

Code: Select all

ipcs -q >> /tmp/nagios.txt
/usr/local/nagios/bin/ndo2db --version >> /tmp/nagios.txt
/usr/local/nagios/bin/nagios --version >> /tmp/nagios.txt
ps -ef | grep ndo >> /tmp/nagios.txt
ps -ef | grep nagios.cfg >> /tmp/nagios.txt
yum list installed >> /tmp/nagios.txt
And also send me the /tmp/nagios.txt file.

Re: ndo2db: Warning: queue send error, retrying...

Posted: Thu May 28, 2015 11:55 am
by mkbansal0588
We are also experiencing the same issue. Do we have any ETA on fix?

Re: ndo2db: Warning: queue send error, retrying...

Posted: Thu May 28, 2015 3:22 pm
by tmcdonald
No ETA, but I have replied to the ticket you have open.