Acknowldgement is not working

[absarcompass]

Hi Team,
I have updated Xi to 5.2.2 version
i have come across this issue where i am unable to acknowledge a problem. It takes the command but acknowledgement is not showing in
and no notification of acknowledgment fires in, it doesn't show on service or Acknowledgements tab under incident Management
I have tried multiple browsers with multiple login on NagiosXI
has anyone come across this? what needs to be check here?

[scottwilkerson]

Can you run the following and try the Acknowledgement again

Code: Select all

service nagios stop
killall -9 nagios
service nagios start

[absarcompass]

Thanks for that,
it didn't work.
I don't have any config error when I verify before restart nagios from Gui. Now i have notice that service check time is not incrementing any more.

cmdsubsys.php is not commented at crontab and log has commands in it.
there is something else which is making it fail to process.
any help will be appreciated

[absarcompass]

I am getting these in /var/log/messeages

Code: Select all

Aug  8 11:43:32 nagiosxi ndo2db: Trimming timedevents.
Aug  8 11:43:32 nagiosxi ndo2db: Trimming systemcommands.
Aug  8 11:43:32 nagiosxi ndo2db: Trimming servicechecks.
Aug  8 11:43:32 nagiosxi ndo2db: Trimming hostchecks.
Aug  8 11:43:32 nagiosxi ndo2db: Trimming eventhandlers.
Aug  8 11:43:32 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:32 nagiosxi ndo2db: Warning: queue send error, retrying...
Aug  8 11:43:33 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:33 nagiosxi ndo2db: Warning: queue send error, retrying...
Aug  8 11:43:34 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:34 nagiosxi ndo2db: Warning: queue send error, retrying...
Aug  8 11:43:35 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:35 nagiosxi ndo2db: Warning: queue send error, retrying...
Aug  8 11:43:36 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:36 nagiosxi ndo2db: Warning: queue send error, retrying...
Aug  8 11:43:37 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:37 nagiosxi ndo2db: Warning: queue send error, retrying...
Aug  8 11:43:38 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:38 nagiosxi ndo2db: Warning: queue send error, retrying...
Aug  8 11:43:38 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:38 nagiosxi ndo2db: Warning: queue send error, retrying...
Aug  8 11:43:39 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:39 nagiosxi ndo2db: Warning: queue send error, retrying...
Aug  8 11:43:40 nagiosxi ndo2db: Message sent to queue.
Aug  8 11:43:40 nagiosxi ndo2db: Warning: queue send error, retrying...

[absarcompass]

and this is in /usr/local/nagiosxi/var/cmdsubsys.log

Code: Select all

FATAL:  terminating connection due to administrator command</pre></p>
    <p><pre>SQL Error [nagiosxi] : FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command</pre></p>
    <p><pre>SQL Error [nagiosxi] : FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command</pre></p>
.    <p><pre>SQL Error [nagiosxi] : FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command</pre></p>
    <p><pre>SQL Error [nagiosxi] : FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command</pre></p>
    <p><pre>SQL Error [nagiosxi] : FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command</pre></p>
.PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
...........................................................PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
..
PROCESSED 0 COMMANDS
..........................................................
PROCESSED 0 COMMANDS
PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
...........................................................
PROCESSED 0 COMMANDS
PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
...........................................................PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491

PROCESSED 0 COMMANDS
............................................................PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491

[absarcompass]

this has affected performance graphs too
no service or host is plotting graph

[scottwilkerson]

Can you send us a profile.zip form Admin -> System Profile
If it will not download, lets start with your/usr/local/nagiosxi/html/config.inc.php

[absarcompass]

Attach is system profile

ndo2db config file is below

#####################################################################
# NDO2DB DAEMON CONFIG FILE
#####################################################################


lock_file=/usr/local/nagios/var/ndo2db.lock

ndo2db_user=nagios
ndo2db_group=nagios

socket_type=unix

socket_name=/usr/local/nagios/var/ndo.sock

tcp_port=5668


db_servertype=mysql
db_host=localhost
db_port=3306

db_name=nagios
db_prefix=nagios_

db_user=ndoutils
db_pass=n@gweb



## TABLE TRIMMING OPTIONS
# Several database tables containing Nagios event data can become quite large
# over time. Most admins will want to trim these tables and keep only a
# certain amount of data in them. The options below are used to specify the
# age (in MINUTES) that data should be allowd to remain in various tables
# before it is deleted. Using a value of zero (0) for any value means that
# that particular table should NOT be automatically trimmed.

# Keep timed events for 24 hours
max_timedevents_age=1440

# Keep system commands for 1 week
max_systemcommands_age=10080

# Keep service checks for 1 week
max_servicechecks_age=10080

# Keep host checks for 1 week
max_hostchecks_age=10080

# Keep event handlers for 31 days
max_eventhandlers_age=44640





# DEBUG LEVEL
# This option determines how much (if any) debugging information will
# be written to the debug file. OR values together to log multiple
# types of information.
# Values: -1 = Everything
# 0 = Nothing
# 1 = Process info
# 2 = SQL queries

debug_level=0



# DEBUG VERBOSITY
# This option determines how verbose the debug log out will be.
# Values: 0 = Brief output
# 1 = More detailed
# 2 = Very detailed

debug_verbosity=1



# DEBUG FILE
# This option determines where the daemon should write debugging information.

debug_file=/usr/local/nagios/var/ndo2db.debug



# MAX DEBUG FILE SIZE
# This option determines the maximum size (in bytes) of the debug file. If
# the file grows larger than this size, it will be renamed with a .old
# extension. If a file already exists with a .old extension it will
# automatically be deleted. This helps ensure your disk space usage doesn't
# get out of control when debugging.

max_debug_file_size=1000000

although parameters are set to weeks and hours but I am getting trimming messeges in
/var/log/messages

Aug 9 09:11:28 nagiosxi ndo2db: Trimming timedevents.
Aug 9 09:11:28 nagiosxi ndo2db: Trimming systemcommands.
Aug 9 09:11:28 nagiosxi ndo2db: Trimming servicechecks.
Aug 9 09:11:28 nagiosxi ndo2db: Trimming hostchecks.
Aug 9 09:11:28 nagiosxi ndo2db: Trimming eventhandlers.


Aug 9 09:12:35 nagiosxi ndo2db: Trimming timedevents.
Aug 9 09:12:35 nagiosxi ndo2db: Trimming systemcommands.
Aug 9 09:12:35 nagiosxi ndo2db: Trimming servicechecks.
Aug 9 09:12:35 nagiosxi ndo2db: Trimming hostchecks.
Aug 9 09:12:35 nagiosxi ndo2db: Trimming eventhandlers.

[absarcompass]

In addition to above post i did run database recovery script /usr/local/nagiosxi/scripts/repairmysql.sh nagio
but system load average is not coming down and fluctuating around higer values
load average: 10.20, 11.32, 11.65

[scottwilkerson]

Lets run the following as a maintenance effort on the nagiosxi postgresql database

Code: Select all

echo "vacuum;vacuum analyse;vacuum full;"|psql nagiosxi postgres

I don't see any further queue send error's but if they re-appear, you should follow this doc
https://support.nagios.com/kb/article.php?id=139