Last State Change is in the Future

[morissette]

Nagios Core 4.1.1

status.dat and retention.log is writing last_state_date in the future by 5 years

Code: Select all

servicestatus {
	host_name=domain.com
	service_description=Vmalloc address space
	modified_attributes=1
	check_command=check_mk-mem.vmalloc
	check_period=24X7
	notification_period=24X7
	check_interval=1.000000
	retry_interval=1.000000
	event_handler=
	has_been_checked=1
	should_be_scheduled=0
	check_execution_time=0.000
	check_latency=0.000
	check_type=1
	current_state=0
	last_hard_state=0
	last_event_id=0
	current_event_id=0
	current_problem_id=0
	last_problem_id=0
	current_attempt=1
	max_attempts=1
	state_type=1
	last_state_change=1587361955
	last_hard_state_change=1587361955
	last_time_ok=1446069278
	last_time_warning=0
	last_time_unknown=0
	last_time_critical=0
	plugin_output=OK - total 112.0 MB, used 3.3 MB, largest chunk 108.6 MB
	long_plugin_output=
	performance_data=used=3.316406;89.59375;100.792969;0;111.992188 chunk=108.550781;64;32;0;111.992188
	last_check=1446069278
	next_check=0
	check_options=0
	current_notification_number=0
	current_notification_id=0
	last_notification=0
	next_notification=0
	no_more_notifications=0
	notifications_enabled=0
	active_checks_enabled=0
	passive_checks_enabled=1
	event_handler_enabled=0
	problem_has_been_acknowledged=0
	acknowledgement_type=0
	flap_detection_enabled=1
	process_performance_data=1
	obsess=1
	last_update=1446069296
	is_flapping=0
	percent_state_change=0.00
	scheduled_downtime_depth=0
	}

[root@mon01 var]# date
Wed Oct 28 16:56:18 CDT 2015
[root@mon01 var]# hwclock
Wed 28 Oct 2015 04:56:21 PM CDT -0.844102 seconds


[root@mon01 etc]# grep timezone nagios.cfg
# This option is used to override the default timezone that this
# the system configured timezone.
# NOTE: In order to display the correct timezone in the CGIs, you
# to include your timezone. Example:
#use_timezone=US/Mountain
#use_timezone=Australia/Brisbane
#use_timezone=CST6CDT
use_timezone=US/Central

The only thing that has changed is that the box was off for one day and was migrated from internal server room to colo datacenter.

[Box293]

What output do you get from:

Code: Select all

hwclock

[morissette]

This was provided in the initial post. Thanks!

[jolson]

Hey morissette, we spoke in IRC yesterday - I just wanted to include information relevant to this port that we discussed.

Screenshot of status information:

Timezone: US/Central

Example service:

Code: Select all

define service {
  use                           check_mk_passive_perf
  host_name                     domain.com
  service_description           Kernel Process Creations
  check_interval                1
  check_command                 check_mk-kernel
}

Check_mk is being used to perform these checks.

Command definition:

Code: Select all

command name check_mk-kernel
command_line echo "ERROR - you did an active check on this service - please disable active checks" && exit 1

@morissette, did you ever perform the packet capture that we talked about? It would be valuable to know if the remote device is reporting improper information.

[jolson]

Client re-installed Nagios Core which resolved the issue - communicated this to me via IRC. Closing this thread.