logstash warning Failed Parsing Date Invalid format, again
Posted: Tue Jan 21, 2020 11:48 am
Nagios Log Server Cluster:
Have added System Profiles from both systems as attachments.
iganagioslog - CentOS release 6.10 (Final)
This is the first system setup when we started using NLS, I believe it was a VMware image from Nagios.
NLS 2.1.3
iganagioslog01 - Red Hat Enterprise Linux Server release 7.7 (Maipo)
Install from downloaded tar file.
I've had this problem before, but seems to have cropped up again. I have searched and can't find what has changed on the systems being logged.
In logstash.log from iganagioslog:
{:timestamp=>"2020-01-21T11:08:19.843000-0500", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"21/Jan/2020:11:08:18 -0500", :exception=>"Invalid format: \"21/Jan/2020:11:08:18 -0500\"", :config_parsers=>"MMM dd HH:mm:ss", :config_locale=>"en", :level=>:warn}
{:timestamp=>"2020-01-21T11:08:19.844000-0500", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"21/Jan/2020:11:08:19 -0500", :exception=>"Invalid format: \"21/Jan/2020:11:08:19 -0500\"", :config_parsers=>"MMM dd HH:mm:ss", :config_locale=>"en", :level=>:warn}
{:timestamp=>"2020-01-21T11:08:19.845000-0500", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"21/Jan/2020:11:08:19 -0500", :exception=>"Invalid format: \"21/Jan/2020:11:08:19 -0500\"", :config_parsers=>"MMM dd HH:mm:ss", :config_locale=>"en", :level=>:warn}
{:timestamp=>"2020-01-21T11:08:19.846000-0500", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"21/Jan/2020:11:08:19 -0500", :exception=>"Invalid format: \"21/Jan/2020:11:08:19 -0500\"", :config_parsers=>"MMM dd HH:mm:ss", :config_locale=>"en", :level=>:warn}
Not seeing them in the logstash.log on iganagiosls01
Many moons ago you helped me get a similar problem fixed, by adding custom syslog and apache log input filters I have found the entries and see where they are coming from, just not how to fix it...
The other strange thing, right now I am concentrating on the 3 webcache servers, igapubwebcache01/02/03. Configured the same, yet I can't get any apache_access logs or apache_error logs to show up in a search for igapubwebcache01. I get syslog and sudo logs, but neither of the apache logs. In addition, I'm only seen the data parse failure in only one of the logstash files. The one on iganagioslog. I have a couple more screen shots, but can only attach 3.
Thanks
Mitch
Have added System Profiles from both systems as attachments.
iganagioslog - CentOS release 6.10 (Final)
This is the first system setup when we started using NLS, I believe it was a VMware image from Nagios.
NLS 2.1.3
iganagioslog01 - Red Hat Enterprise Linux Server release 7.7 (Maipo)
Install from downloaded tar file.
I've had this problem before, but seems to have cropped up again. I have searched and can't find what has changed on the systems being logged.
In logstash.log from iganagioslog:
{:timestamp=>"2020-01-21T11:08:19.843000-0500", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"21/Jan/2020:11:08:18 -0500", :exception=>"Invalid format: \"21/Jan/2020:11:08:18 -0500\"", :config_parsers=>"MMM dd HH:mm:ss", :config_locale=>"en", :level=>:warn}
{:timestamp=>"2020-01-21T11:08:19.844000-0500", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"21/Jan/2020:11:08:19 -0500", :exception=>"Invalid format: \"21/Jan/2020:11:08:19 -0500\"", :config_parsers=>"MMM dd HH:mm:ss", :config_locale=>"en", :level=>:warn}
{:timestamp=>"2020-01-21T11:08:19.845000-0500", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"21/Jan/2020:11:08:19 -0500", :exception=>"Invalid format: \"21/Jan/2020:11:08:19 -0500\"", :config_parsers=>"MMM dd HH:mm:ss", :config_locale=>"en", :level=>:warn}
{:timestamp=>"2020-01-21T11:08:19.846000-0500", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"21/Jan/2020:11:08:19 -0500", :exception=>"Invalid format: \"21/Jan/2020:11:08:19 -0500\"", :config_parsers=>"MMM dd HH:mm:ss", :config_locale=>"en", :level=>:warn}
Not seeing them in the logstash.log on iganagiosls01
Many moons ago you helped me get a similar problem fixed, by adding custom syslog and apache log input filters I have found the entries and see where they are coming from, just not how to fix it...
The other strange thing, right now I am concentrating on the 3 webcache servers, igapubwebcache01/02/03. Configured the same, yet I can't get any apache_access logs or apache_error logs to show up in a search for igapubwebcache01. I get syslog and sudo logs, but neither of the apache logs. In addition, I'm only seen the data parse failure in only one of the logstash files. The one on iganagioslog. I have a couple more screen shots, but can only attach 3.
Thanks
Mitch