nagios.cfg not being updated

[Fred Kroeger]

I've disabled the Notifications & event handlers via the Monitoring Engine Status Admin page and they show disabled - although the red dot for event handler doesn't display (it stays grey).
Checked the nagios.cfg file and it hasn't been updated - they are both still defined as enabled. This means that the next time I restart Nagios (apply configuration) , the notifications & event handler revert back to enabled.
I'm running NagiosXI 2014R2.4. Previous to this version it has worked as expected - If you disable those functions in the GUI - they stay disabled.
I know this, because I always have the notifications disabled on my Test servers.

Permissions on the file seem OK
-rwxrwxr-x 1 apache nagios 6364 Jan 22 10:03 nagios.cfg

Has something changed in how this is supposed to operate or is it a bug?

regards... Fred

[cmerchant]

I don't know why event handlers is greyed out, it still says disabled if you hover over the grey circle. Could you check in your retentions.dat for the notifications and event handlers flag:

Code: Select all

grep "enable_event" /usr/local/nagios/var/retention.dat
grep "enable_notif" /usr/local/nagios/var/retention.dat

Are you using ramdisk? Are the files retained/preserved between system restart?

[Fred Kroeger]

Entries in retention.dat are correct

Code: Select all

# grep "enable_event" /usr/local/nagios/var/retention.dat
enable_event_handlers=0
# grep "enable_notif" /usr/local/nagios/var/retention.dat
enable_notifications=0

The entries in nagios.cfg show

Code: Select all

# grep "enable_event" /usr/local/nagios/etc/nagios.cfg
enable_event_handlers=1
# grep "enable_notif" /usr/local/nagios/etc/nagios.cfg
enable_notifications=1

I then run Apply Configuration which restarts Nagios, The Notifications & Event Handler become Enabled again.
Checking the retention.dat file - those entries are still showing disabled.

Yes I am using RAMDisk. Not sure what you mean are the files preserved between system restart? If you mean a Nagios restart, they all get updated on a restart. The only ones that remain are in the spool directory.

Are you saying that on a restart that the nagios.cfg entries get applied and then the entries in retention.dat are applied? If that is the case, then it is not happening.

This is an excerpt from the messgaes log during the Nagios restart - no errors relating to not being able to apply retention.dat

Code: Select all

Jan 29 08:41:58 pernag02 nagios: Caught SIGTERM, shutting down...
Jan 29 08:41:58 pernag02 nagios: Successfully shutdown... (PID=10649)
Jan 29 08:41:58 pernag02 nagios: Event broker module 'NERD' deinitialized successfully.
Jan 29 08:41:58 pernag02 ndo2db: Error: mysql_query() failed for 'UPDATE nagios_conninfo SET disconnect_time=NOW(), last_checkin_time=NOW(), data_end_time=FROM_UNIXTIME(0), bytes_processed='0', lines_processed='0', entries_processed='0' WHERE conninfo_id='0''
Jan 29 08:41:58 pernag02 ndo2db: mysql_error: 'MySQL server has gone away'
Jan 29 08:41:58 pernag02 ndo2db: Error: Connection to MySQL database has been lost!
Jan 29 08:41:58 pernag02 nagios: ndomod: Shutdown complete.
Jan 29 08:41:58 pernag02 nagios: Event broker module '/usr/local/nagios/bin/ndomod.o' deinitialized successfully.
Jan 29 08:41:59 pernag02 nagios: Nagios 4.0.8 starting... (PID=8839)
Jan 29 08:41:59 pernag02 nagios: Local time is Thu Jan 29 08:41:59 AWST 2015
Jan 29 08:41:59 pernag02 nagios: LOG VERSION: 2.0
Jan 29 08:41:59 pernag02 nagios: qh: Socket '/usr/local/nagios/var/rw/nagios.qh' successfully initialized
Jan 29 08:41:59 pernag02 nagios: qh: core query handler registered
Jan 29 08:41:59 pernag02 nagios: nerd: Channel hostchecks registered successfully
Jan 29 08:41:59 pernag02 nagios: nerd: Channel servicechecks registered successfully
Jan 29 08:41:59 pernag02 nagios: nerd: Channel opathchecks registered successfully
Jan 29 08:41:59 pernag02 nagios: nerd: Fully initialized and ready to rock!
Jan 29 08:41:59 pernag02 nagios: wproc: Successfully registered manager as @wproc with query handler
Jan 29 08:41:59 pernag02 nagios: wproc: Registry request: name=Core Worker 8844;pid=8844
Jan 29 08:41:59 pernag02 nagios: wproc: Registry request: name=Core Worker 8842;pid=8842
Jan 29 08:41:59 pernag02 nagios: wproc: Registry request: name=Core Worker 8841;pid=8841
Jan 29 08:41:59 pernag02 nagios: wproc: Registry request: name=Core Worker 8843;pid=8843
Jan 29 08:41:59 pernag02 nagios: ndomod: NDOMOD 2.0.0 (02-28-2014) Copyright (c) 2009 Nagios Core Development Team and Community Contributors
Jan 29 08:41:59 pernag02 nagios: ndomod: Successfully connected to data sink.  0 queued items to flush.
Jan 29 08:41:59 pernag02 nagios: ndomod registered for process data
Jan 29 08:41:59 pernag02 nagios: ndomod registered for log data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for system command data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for event handler data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for notification data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for comment data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for downtime data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for flapping data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for program status data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for host status data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for service status data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for adaptive program data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for adaptive host data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for adaptive service data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for external command data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for aggregated status data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for retention data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for contact data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for contact notification data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for acknowledgement data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for state change data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for contact status data'
Jan 29 08:41:59 pernag02 nagios: ndomod registered for adaptive contact data'
Jan 29 08:41:59 pernag02 nagios: Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.
Jan 29 08:42:02 pernag02 nagios: Successfully launched command file worker with pid 8943

I'm not really worried about the eventhandler dot not being red as it is definitely disabled - however it is something that should be fixed.

regards... Fred

[scottwilkerson]

When you click these items through the UI it changes their runtime value, which overrides the nagios.cfg value.

These are stored in retention.dat to persist across service restarts.

[Fred Kroeger]

Thanks Scott - so that would mean that the retention.dat file is not being applied after a restart?

[scottwilkerson]

I would think it means it is being applied because the setting from you clicking on items in the UI are sticking across reboots (as they should), but you cannot verify it by looking at the nagios.cfg, you have to verify it by looking in the UI to see if they are active or not.

[Fred Kroeger]

Not following you Scott ?

I disable Notifications & Event Handling from the GUI
They show they are disabled in the GUI & they show they are disabled in retention.dat

I apply a Configuration & Nagios restarts
After the restart, The GUI then shows that Notifications & Event Handling are now enabled - but they still show disabled in retention.dat.

I have tried this on two diffferent Nagios servers (both running NagiosXI 2014 R2.3) and both ignore those settings in retention.dat when Nagios restarts.

regards... Fred

[lmiltchev]

After the restart, The GUI then shows that Notifications & Event Handling are now enabled - but they still show disabled in retention.dat.

Hm-m, I was not able to recreate this issue. Can you run the following command and show us the output?

Code: Select all

grep retain_state_information /usr/local/nagios/etc/nagios.cfg

[tgriep]

If your retention.dat is in RAM disk and you reboot, all of the settings will be lost.
If you move it back to the disk, your settings will restore after a reboot.

[Fred Kroeger]

Note: I am not rebooting the computer - the steps I have listed just show that I run "Apply Configuration" which restarts the Nagios process.
And no the retention.dat file is *not* on the RAM Disk - it wouldn't make sense to have it there?

Code: Select all

# grep enable_notification /usr/local/nagios/var/retention.dat
enable_notifications=0
# grep enable_notification /usr/local/nagios/etc/nagios.cfg
enable_notifications=1
# grep retain_state_information /usr/local/nagios/etc/nagios.cfg
retain_state_information=1

Also - as mentioned already here, I have tried this on two different Nagios installations (both 2014R2.3) , so I have just tested this on a third Nagios Server (also 2014R2.3)
And as you found with yours, it works correctly. So now I'm at a loss to explain what is happening.
I'll run a diff later on the nagios.cfg files and see if I can pick any differences.

regards.. Fred