Acknowldgement is not working

[absarcompass]

Hi ssax,
thanks for your help, have PMd you the recent profile file.
we are customer with enterprise unlimited node licence but we don't have support contract.

[ssax]

Your /usr/local/nagios/etc/ndo2db.cfg file still hasn't been updated, did you follow that guide from page 10 onwards to make the changes?

You current configuration will not work properly without updating that file first with your new DB server settings.

Please make those changes and restart NDO2DB/nagios:

Code: Select all

service ndo2db restart
service nagios restart

[absarcompass]

Below is my ndo2db.cfg file
the only difference is instead of removing values i have left the older values and commented them out rest of the file is as it was mentioned in the DB offloading document

#####################################################################
# NDO2DB DAEMON CONFIG FILE
#####################################################################


lock_file=/usr/local/nagios/var/ndo2db.lock

ndo2db_user=nagios
ndo2db_group=nagios

socket_type=unix

socket_name=/usr/local/nagios/var/ndo.sock

tcp_port=5668


db_servertype=mysql
#db_host=localhost
db_host=cmps-cluster-db
#db_port=3306
db_port=3307

db_name=nagios
db_prefix=nagios_

#db_user=ndoutils
db_user=nagios
#db_pass=n@gweb
db_pass=XXXXXXXXXXX


## TABLE TRIMMING OPTIONS
# Several database tables containing Nagios event data can become quite large
# over time. Most admins will want to trim these tables and keep only a
# certain amount of data in them. The options below are used to specify the
# age (in MINUTES) that data should be allowd to remain in various tables
# before it is deleted. Using a value of zero (0) for any value means that
# that particular table should NOT be automatically trimmed.

# Keep timed events for 24 hours
max_timedevents_age=1440

# Keep system commands for 1 week
max_systemcommands_age=10080

# Keep service checks for 1 week
max_servicechecks_age=10080

# Keep host checks for 1 week
max_hostchecks_age=10080

# Keep event handlers for 31 days
max_eventhandlers_age=44640





# DEBUG LEVEL
# This option determines how much (if any) debugging information will
# be written to the debug file. OR values together to log multiple
# types of information.
# Values: -1 = Everything
# 0 = Nothing
# 1 = Process info
# 2 = SQL queries

debug_level=0



# DEBUG VERBOSITY
# This option determines how verbose the debug log out will be.
# Values: 0 = Brief output
# 1 = More detailed
# 2 = Very detailed

debug_verbosity=1



# DEBUG FILE
# This option determines where the daemon should write debugging information.

debug_file=/usr/local/nagios/var/ndo2db.debug



# MAX DEBUG FILE SIZE
# This option determines the maximum size (in bytes) of the debug file. If
# the file grows larger than this size, it will be renamed with a .old
# extension. If a file already exists with a .old extension it will
# automatically be deleted. This helps ensure your disk space usage doesn't
# get out of control when debugging.

max_debug_file_size=1000000

[ssax]

That looks fine, did you restart nagios and ndo2db? Did it fix your issues?

Code: Select all

service nagios stop
service ndo2db restart
service nagios start

What are you seeing in the nagios.log now?

[absarcompass]

Thanks ssax,
no it didn't
nagios log stops generating logs and /var/log/messages were generating same logs

Oct 12 23:29:02 nagiosxi ndo2db: Warning: queue send error, retrying...
Oct 12 23:29:03 nagiosxi ndo2db: Message sent to queue.
Oct 12 23:29:03 nagiosxi ndo2db: Warning: queue send error, retrying...
Oct 12 23:29:06 nagiosxi ndo2db: Message sent to queue.
Oct 12 23:29:06 nagiosxi ndo2db: Warning: queue send error, retrying...

and then after more than 4 hours the nagios log start generating the normal logs of checking hosts and services.

[1539225504] SERVICE ALERT: VHOST - MANAGEMENT-MARVIN;Server Work Queues;CRITICAL;SOFT;1;CRITICAL - Socket timeout
[1539225550] SERVICE ALERT: PHOST - MAIL-MAILBACKUP-NAS;CHECK-LOAD-AVERAGE;OK;SOFT;1;SNMP OK - 511 538 500
[1539225610] SERVICE ALERT: VHOST - MANAGEMENT-MARVIN;Server Work Queues;OK;SOFT;1;Current work queue (an indication of processing load) is 0
[1539225739] Auto-save of retention data completed successfully.

Previously I even tried rebooting the server itself
even after starting receving alert email and sms the ipcs -q message values don't go to 0

key msqid owner perms used-bytes messages
0xfd000002 131072 nagios 600 291280896 284454

I am quouting my previous finding that nagios deamon was writing alot on nagios.cfg. Can you guys please help me out with this??

Hi ssax,
yup have go through the doc again and have double check configs. I can run NagiosXI web very fine
its just that when ever nagios service is restarted from web or command line or new settings applied from web it stops service check and greyed out the hosts assigned to the services on web. and it was not happening before upgrade to 5.5.2

can you please confirm if nagios deamon writes to nagios.cfg all the time
below is the output of iotop for last 5 minutes

Total DISK READ: 0.00 B/s | Total DISK WRITE: 215.33 M/s
TID PRIO USER DISK READ DISK WRITE> SWAPIN IO COMMAND
1822 be/4 nagios 0.00 B 18.51 G 0.00 % 16.58 % nagios -d /usr/local/nagios/etc/nagios.cfg

I can not figure out why every time I restart nagios service or apply settings it stops working for at least more than an hour.

Thanks

[ssax]

Please send me a fresh copy of your profile, I'm going to import it into my system with your latest configs, you can download it from Admin > System Profile > Download Profile and upload it to the ticket by clicking the "choose item" link at the bottom of the menu.​ Make sure to wait until the file is finished uploading before clicking the Post Reply button.

The nagios daemon doesn't write to the nagios.cfg file, what you are seeing is the nagios process writing information (not to nagios.cfg), it's showing you the command that the nagios process was started with (which requires you to specify the nagios.cfg path). It's not directly writing to the nagios.cfg. It's essentially the output of this:

Code: Select all

[root@xid scripts]# ps aux | grep nagios.cfg
nagios   11670  0.3  0.1  21748  2032 ?        Ss   14:32   0:02 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   11686  0.0  0.0  21108   832 ?        S    14:32   0:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg

But it doesn't show the full path to the nagios binary.

It shows you the binary that is writing with the commands that it was started with.

[absarcompass]

Hi ssax,
I have PMd you the system profile, let me know if you require further information from my end.

Thanks again.

[ssax]

Thanks, received. I sent you another PM.

[absarcompass]

HI ssax,
I have downloaded the system profile today as well, but ndo2db modification date is 24/12/2014 looks like system profile is not pulling newly edit file from location /usr/local/nagios/etc/
my ndo2db file in location /usr/local/nagios/etc/ is the one that i pasted on this thread.

[ssax]

Let's do this, edit your /usr/local/nagios/etc/ndo2db.cfg and set these:
- NOTE: Make sure you note what they are currently set to so you that you can revert the changes afterwards

Code: Select all

debug_level=-1
debug_verbosity=2

Then edit your /usr/local/nagios/etc/nagios.cfg file and set:

Code: Select all

debug_level=-1
debug_verbosity=2

Then run these commands:

Code: Select all

service nagios stop
service ndo2db restart
service nagios start

Then test it out briefly and then PM me these files:

Code: Select all

/usr/local/nagios/var/ndo2db.debug
/usr/local/nagios/var/nagios.debug

Then revert your changes to the two files and run these commands again so it doesn't fill up your disk space:

Code: Select all

service nagios stop
service ndo2db restart
service nagios start