Nagios XI keeps crashing post upgrade to XI 2014

[abrist]

I presume you have restarted the server so that the new kernel limits are loaded, correct?
run the following command to potentially fix the postgres vacuum issue:

Code: Select all

echo "vacuum;vacuum analyze;vacuum full;"|psql nagiosxi postgres

[lmiltchev]

What is the Nagios XI version that you upgraded from?

[chriscamm]

I have run the vacuum and rebooted again.

I upgraded from XI 2012 - 2012R2.9

Thanks

Chris

[lmiltchev]

Did you see any errors, while running the vacuum on the postgres? Let us know if the problem resurfaces.

[chriscamm]

No errors at all while running the vacuum commands

Code: Select all

Jun 10 22:04:35 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:35 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:36 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:36 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:37 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:37 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:38 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:38 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:39 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:39 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:40 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:40 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:41 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:41 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:42 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:42 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:43 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:43 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:44 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:44 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:45 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:45 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:46 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:46 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:47 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:47 qualngs ndo2db[2294]: Warning: queue send error, retrying...
Jun 10 22:04:48 qualngs ndo2db[2294]: Message sent to queue.
Jun 10 22:04:48 qualngs ndo2db[2294]: Warning: queue send error, retrying...

Errors are back almost straight away.

What is the maximum Kernel settings I can use does anyone know?

Thanks

Chris

[sreinhardt]

Before we keep increasing your kernel message queues, how about we try setting less worker processes. Considering message queues should only be used for inter-process communication, if we limit the number of workers which left to up to core could be as high as 18 ((6 physical cores + 6 hyper-threaded cores) X 1.5 = 18). Plus each worker spawns processes to run the actual checks, thats a whole lot of processes. Try altering the nagios.cfg setting check_workers to check_workers=6. Do a service nagios restart, and see how that gets along.

[chriscamm]

Hi,

I have decreased the

Code: Select all

core_workers=6
max_concurrent=20

Still get

Code: Select all

Jun 12 11:24:00 qualngs ndo2db[2321]: Message sent to queue.
Jun 12 11:24:00 qualngs ndo2db[2321]: Warning: queue send error, retrying...
Jun 12 11:24:01 qualngs ndo2db[2321]: Message sent to queue.
Jun 12 11:24:01 qualngs ndo2db[2321]: Warning: queue send error, retrying...

Thanks

Chris

[lmiltchev]

It seems like we may need to schedule a remote session to further troubleshoot this issue. You can send us an email at [email protected] to open a new ticket in our ticketing system.

BTW, can you run the following command and show us the output?

Code: Select all

/usr/local/nagios/bin/ndo2db | head -2

[chriscamm]

Code: Select all

[root@qualngs ~]# /usr/local/nagios/bin/ndo2db | head -2

NDO2DB 2.0.0
[root@qualngs ~]# 

[lmiltchev]

Is a remote session an option for you?