Page 1 of 1
Back to the Future Indexes?
Posted: Mon Apr 23, 2018 4:09 pm
by vAJ
Randomly get these issues where indexes are created for days far in the past or in the future.
Flux capacitor looks OK... but I still can't figure this out:
Index # Docs Primary Size # Shards # Replicas Action
logstash-2018.12.16 26,003 3.7MB 5 1
logstash-2018.12.15 32,326 4.8MB 5 1
logstash-2018.12.14 32,786 4.7MB 5 1
logstash-2018.12.13 40,788 5.8MB 5 1
Re: Back to the Future Indexes?
Posted: Mon Apr 23, 2018 4:50 pm
by cdienger
This will occur if the time is off on a client sending logs to NLS. I recently responded to a similar thread:
https://support.nagios.com/forum/viewto ... 38&t=48245
The long short of it:
-use the dashboard or command line to find out which hosts are sending the bad data
-filters can be created to prevent "old" or "future" data from even getting into the database
Re: Back to the Future Indexes?
Posted: Mon Apr 23, 2018 5:22 pm
by vAJ
Date / time on the host it's coming from is spot on.
It's a syslog input that I'm not passing any other filtering or timestamp modification on.
I thought the NLS logstash config wrote indexes based on received time... huh.
Re: Back to the Future Indexes?
Posted: Tue Apr 24, 2018 10:26 am
by cdienger
If the message doesn't contain a timestamp then it will place it in the current index.
Is it just a single host creating these indexes? How frequently are they created? Do they come back if you delete them? Can you share the contents of one of the indexes(PM it to me if contains sensitive info).