livestatus not picking up comments after nagios restart

[GavinG]

So .. you're not allowed to laugh at us here .. OK?

Anyway, you know this particular entry in the nagios.cfg file:

Code: Select all

# USE RETAINED PROGRAM STATE
# This setting determines whether or not Nagios will set
# program status variables based on the values saved in the
# retention file.  If you want to use retained program status
# information, set this value to 1.  If not, set this value
# to 0.

use_retained_program_state=1

Well, this was set to 0 before yesterday (for some reason we changed it in mid December, no idea why) but since changing it back to 1 yesterday our next_comment_id didn't reset back to 1 overnight.

So what I'm thinking is that if Nagios is restarted with this is set to 0, next_comment_id is not always set properly based on the highest comment_id in the retention.dat file (I'm assuming it should do that) but instead will be set back to 1. What makes it more odd is that I'm positive next_comment_id wasn't always resetting when we restarted Nagios which is why we were having issues tracking it down.

I'm going to keep it running for another ~24 hours and if it still doesn't reset then I think that's what the problem was. I feel kind of stupid now after finding that, although I guess it's all worth it if this has identified a small bug.

[GavinG]

And ~24 hours later next_comment_id still hasn't reset so I am more confident that's what was causing it.

After talking to somebody here, we now know why we set this to 0 (it was so that when we synced the config and retention information to DR we could set enable_notifications to 0 in nagios.cfg and not worry about it being overwritten by values in the retention.dat file).

But this brings up that next_comment_id is potentially being reset to 1 when use_retained_program_state is set to 0 instead of calculating it based on the highest comment_id in the retention.dat file. This wouldn't be an issue if *find_comment was searching for an available comment_id globally but the search restriction based on comment_type means a host and a service can currently share a comment_id. I'll try to write up a bug report for this sometime today.

[abrist]

Let us know what you find. This is very interesting.