Nagios Support Forum

Hello,

Our Nagios log server, some Indices shows with the wrong date. The present date and previous dates Indices are showing correctly. But lots of indices with future dates are seen. If we deleted it, it will generate again. The size of that Indices should be less than 1 MB. The system date and PHP date all are fine. I have attached the screenshot with this ticket. Kindly check and let me know.

This is usually caused by a system sending logs dated into the future.

Sometimes you can find out what system it is by going to the dashboard and setting a custom date to match the timeframe where the future indexes are

Hello,

I have tried to recreate this issue in our local Nagios server by changing the date of one client machine to future date and it will not generate any new indices with a future date.

Also, I have searched for the future date logs in the server which have the issue and I couldn't see any logs related to future logs.

Please check and let me know.

How quickly are they regenerated if you delete them? Do you know where the my-index index came from? Are you able to delete that? my-index is not a NLS index and NLS should only be creating indices when it receives data for those dates, but it's possible to create indices on the command line using the Elasticsearch API. Does anyone else have access to the NLS command line?

How quickly are they regenerated if you delete them?

If we delete, it will regenerate on very next day

Do you know where the my-index index came from?

Location will be as below

# /syslog_data/data/8df86e3a-8739-43ab-8a7f-110b464cae2d/nodes/0/indices/

Are you able to delete that?

# Yes, we are able to delete

my-index is not a NLS index and NLS should only be creating indices when it receives data for those dates, but it's possible to create indices on the command line using the Elasticsearch API. Does anyone else have access to the NLS command line?

# No one else have access to NLS command line

In settings, we have mentioned 90 days to delete older indexes, but we could see more than 90 days indexes in index overview page at a time.

Delete the indices and then PM me a profile after they are recreated the next day. The profile can be gathered under Admin > System > System Status > Download System Profile or from the command line with:

/usr/local/nagioslogserver/scripts/profile.sh

This will create /tmp/system-profile.tar.gz.

Note that this file can be very large and may not be able to be uploaded through the system. This is usually due to the logs in the Logstash and/or Elasticseach directories found in it. If it is too large, please open the profile, extract these directories/files and send them separately.

Hello,

I have sent Nagios log server system profile information to you separately.

I have deleted all the future indices and within a few minutes, the new future indices are generated.

I have attached the screenshot of the index menu.

Please check the Nagios log server profile.

If you go to the dashboard, and in the time selection choose "Custom"

In the fields enter You should see the 15 or more documents on this day, you can then look at the "host" field to see which of your hosts are sending future dated logs

The new index appears to have data so the dashboard, as @scottwilkerson pointed out, should give you more details about what is sent data in.

Hello,

I have tried to search the mentioned date in the custom date setting in the dashboard menu and I am not able to set the future date. It shows invalid dates.

But when I search the future date in the query. It shows one device having message contain the future date and the timestamp is correct date, not the future date.

This date in the indices matched with the date in the message of the device.

Please see the below message from the device.

++++++++++++++++++++++++++++++

<189>date=2019-07-23 time=13:24:26 devname="FG800C3912801619" devid="FG800C3912801619" logid="0317013312" type="utm" subtype="webfilter" eventtype="ftgd_allow" level="notice" vd="root" eventtime=1563873866 policyid=40 sessionid=228957280 user="PKUMAR" group="Level-1" srcip=10.100.143.119 srcport=54811 srcintf="port2" srcintfrole="lan" dstip=157.240.25.35 dstport=443 dstintf="port12" dstintfrole="wan" proto=6 service="HTTPS" hostname="www.facebook.com"; profile="LEVEL-1" action="passthrough" reqtype="referral" url="/tr/?id=551295824981249&ev=Search&dl=https://www.cleartrip.ae/flights/intern ... cd[b][i][b][departing_departure_date]=2019-10-25[/b][/i][/b]&cd[origin_airport]=SHJ&cd[destination_airport]=JAI&" referralurl="https://www.cleartrip.ae/flights/intern ... dults=1&ch"; sentbyte=18553 rcvdbyte=6275 direction="outgoing" msg="URL belongs to an allowed category in policy" method="domain" cat=37 catdesc="Social Networking"


++++++++++++++++++++++++++++++

Kindly check why the message contains future date can create the future date Indices.