Page 1 of 2

Timestamp error

Posted: Tue Jun 21, 2016 7:49 am
by Zaqen
Hi, i 'm new user with nagioslogserver.

I install it (.ova), and he run successfull, but i have this error in /var/log/logstach

Can you help if it's possible to resolv it ?

Re: Timestamp error

Posted: Tue Jun 21, 2016 9:44 am
by hsmith
Can you show me the error?

Re: Timestamp error

Posted: Wed Jun 22, 2016 7:05 am
by Zaqen
Hi,
yes i have this error in logstash log :
at: "Jun 22 08:29:23", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:25.473000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:25", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:25", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.205000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.218000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.893000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.895000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:35.642000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:35", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:35", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:38.483000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:38", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:38", :level=>:warn}

Re: Timestamp error

Posted: Wed Jun 22, 2016 9:16 am
by hsmith
What kind of device are you getting logs from that is generating this error? This generally happens when the syslog input cannot parse the timestaps of the logs being sent to it.

Re: Timestamp error

Posted: Mon Jun 27, 2016 3:18 am
by Zaqen
Hi,

Is in logstach log, i have this errors when i have install nagioslogserver. I try to add machine but impossible to see it in dashboard, i think she send correctly log to nagioslogserver but timestamps is not good.

what must i do to resolv it ?

Re: Timestamp error

Posted: Mon Jun 27, 2016 9:48 am
by hsmith
One thing you can try is switching to using raw TCP/UDP for your inputs instead of the syslog input.

Go to Administration > Global > Global Configuration and expand the syslog input.

Replace what is there with this:

Code: Select all

tcp {
    port => 5544
    type => syslog
  }
  udp {
    port => 5544
    type => syslog
  }
  
This will stop Logstash from dropping logs with a syslog format that it doesn't like, which is the default behavior.

Your syslogs will no longer be broken down in to nice fields, because we're not using the syslog input anymore, but we can have that process done with this grok filter:

Code: Select all

  if [type] == "syslog" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
  }
Give this a shot.

Re: Timestamp error

Posted: Wed Jun 29, 2016 1:43 am
by Zaqen
Hi, thank you.
I put the first code in global, i do add input and paste first code :

Code: Select all

tcp {
    port => 5544
    type => syslog
  }
  udp {
    port => 5544
    type => syslog
  }
i restart logstash but i have the same message in /var/log/logstash/logstash.log

impossible to see another server who i had.

what must i do with this ?

Code: Select all

 if [type] == "syslog" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
  }

Re: Timestamp error

Posted: Wed Jun 29, 2016 1:49 am
by Zaqen
I found this in view > edit inputs

i add, it's ok ?

Code: Select all

# 
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Wed, 29 Jun 2016 08:47:14 +0200
#

#
# Global inputs
#

input {
    syslog {
        type => 'syslog'
        port => 5544
    }
    tcp {
        type => 'eventlog'
        port => 3515
        codec => json {
            charset => 'CP1252'
        }
    }
    tcp {
        type => 'import_raw'
        tags => 'import_raw'
        port => 2056
    }
    tcp {
        type => 'import_json'
        tags => 'import_json'
        port => 2057
        codec => json
    }
}

#
# Local inputs
#

tcp {
    port => 5544
    type => syslog
  }
  udp {
    port => 5544
    type => syslog
  }

Re: Timestamp error

Posted: Wed Jun 29, 2016 1:58 am
by Zaqen
first, i edit Syslog entry in global > configuration

I stop (click on green activate button) syslog, click on + to add :

Code: Select all

tcp {
    port => 5544
    type => syslog
  }
  udp {
    port => 5544
    type => syslog
  }
I save,

second i add new filter, her name is filter, i past this :

Code: Select all

 if [type] == "syslog" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
  }
I save, and apply configuration, but i have message :

Code: Select all

 The configuration could not be verified.
Ten minutes after :
I found this :

Code: Select all

The error reported is:
  Couldn't find any input plugin named 'grok'. Are you sure this is correct? Trying to load the grok input plugin resulted in this error: no such file to load -- logstash/inputs/grok

Re: Timestamp error

Posted: Wed Jun 29, 2016 2:12 am
by Zaqen
I have same problem :

Code: Select all

{:timestamp=>"2016-06-29T09:11:08.592000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:08", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:08", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:11.904000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:11", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:11", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:11.928000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:11", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:11", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:12.196000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:12", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:12", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:12.199000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:12", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:12", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:17.071000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:17", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:17", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:17.088000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:17", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:17", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:22.499000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:22", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:22", :level=>:warn}
and again and again