NLS Dashboard shows logs with a several hours delay.
Posted: Mon Apr 04, 2016 7:50 am
Yesterday I started sending logs from multiple hosts to NLS. I couldn't see the logs appear in the dashboard at the time. This morning, without me doing anything, the logs started appearing. Now, I have powered down one of the hosts which was sending multiple logs a minute. I have confirmed using tcpdump that NLS is not receiving any traffic from that host. However NLS dashboard keeps showing new logs with recent timestamps from that host.My current guess is that the logs are delayed because of time zone mismatch. It sounds similar to "trial issue, setup multiple sources, none showing up" thread.
In case this is related to time zones, here is my console output.The system which is the source of the logs shows the same console output as above.
These show time and timezone which are correct for me. However after reboot of NLS server, the delayed logs are still appearing as if they are recent.
Code: Select all
2016-04-04T13:47:02.000+01:00 YY.YY.YY.YY syslog Failed password for root from ZZ.ZZ.ZZ.ZZ port PPPP ssh2In case this is related to time zones, here is my console output.
Code: Select all
[ec2-user@ip-XX-XX-XX-XX ~]$ ls -al /etc/localtime
lrwxrwxrwx 1 root root 33 Apr 4 09:15 /etc/localtime -> /usr/share/zoneinfo/Europe/London
[ec2-user@ip-XX-XX-XX-XX ~]$ date
Mon Apr 4 13:40:53 BST 2016
[ec2-user@ip-XX-XX-XX-XX ~]$ hwclock
Cannot access the Hardware Clock via any known method.
Use the --debug option to see the details of our search for an access method.
[ec2-user@ip-XX-XX-XX-XX ~]$ cat /etc/php.ini | grep date.time
; http://www.php.net/manual/en/datetime.configuration.php#ini.date.timezone
date.timezone = Europe/London
[ec2-user@ip-XX-XX-XX-XX ~]$ cat /etc/sysconfig/clock
ZONE="Europe/London"
UTC=TrueThese show time and timezone which are correct for me. However after reboot of NLS server, the delayed logs are still appearing as if they are recent.