Timestamp error

[Zaqen]

Hi, i 'm new user with nagioslogserver.

I install it (.ova), and he run successfull, but i have this error in /var/log/logstach

Can you help if it's possible to resolv it ?

[hsmith]

Can you show me the error?

[Zaqen]

Hi,
yes i have this error in logstash log :
at: "Jun 22 08:29:23", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:25.473000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:25", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:25", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.205000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.218000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.893000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.895000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:35.642000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:35", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:35", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:38.483000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:38", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:38", :level=>:warn}

[hsmith]

What kind of device are you getting logs from that is generating this error? This generally happens when the syslog input cannot parse the timestaps of the logs being sent to it.

[Zaqen]

Hi,

Is in logstach log, i have this errors when i have install nagioslogserver. I try to add machine but impossible to see it in dashboard, i think she send correctly log to nagioslogserver but timestamps is not good.

what must i do to resolv it ?

[hsmith]

One thing you can try is switching to using raw TCP/UDP for your inputs instead of the syslog input.

Go to Administration > Global > Global Configuration and expand the syslog input.

Replace what is there with this:

Code: Select all

tcp {
    port => 5544
    type => syslog
  }
  udp {
    port => 5544
    type => syslog
  }
  

This will stop Logstash from dropping logs with a syslog format that it doesn't like, which is the default behavior.

Your syslogs will no longer be broken down in to nice fields, because we're not using the syslog input anymore, but we can have that process done with this grok filter:

Code: Select all

  if [type] == "syslog" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
  }

Give this a shot.

[Zaqen]

Hi, thank you.
I put the first code in global, i do add input and paste first code :

Code: Select all

tcp {
    port => 5544
    type => syslog
  }
  udp {
    port => 5544
    type => syslog
  }

i restart logstash but i have the same message in /var/log/logstash/logstash.log

impossible to see another server who i had.

what must i do with this ?

Code: Select all

 if [type] == "syslog" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
  }

[Zaqen]

I found this in view > edit inputs

i add, it's ok ?

Code: Select all

# 
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Wed, 29 Jun 2016 08:47:14 +0200
#

#
# Global inputs
#

input {
    syslog {
        type => 'syslog'
        port => 5544
    }
    tcp {
        type => 'eventlog'
        port => 3515
        codec => json {
            charset => 'CP1252'
        }
    }
    tcp {
        type => 'import_raw'
        tags => 'import_raw'
        port => 2056
    }
    tcp {
        type => 'import_json'
        tags => 'import_json'
        port => 2057
        codec => json
    }
}

#
# Local inputs
#

tcp {
    port => 5544
    type => syslog
  }
  udp {
    port => 5544
    type => syslog
  }

[Zaqen]

first, i edit Syslog entry in global > configuration

I stop (click on green activate button) syslog, click on + to add :

Code: Select all

tcp {
    port => 5544
    type => syslog
  }
  udp {
    port => 5544
    type => syslog
  }

I save,

second i add new filter, her name is filter, i past this :

Code: Select all

 if [type] == "syslog" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
  }

I save, and apply configuration, but i have message :

Code: Select all

 The configuration could not be verified.

Ten minutes after :
I found this :

Code: Select all

The error reported is:
  Couldn't find any input plugin named 'grok'. Are you sure this is correct? Trying to load the grok input plugin resulted in this error: no such file to load -- logstash/inputs/grok

[Zaqen]

I have same problem :

Code: Select all

{:timestamp=>"2016-06-29T09:11:08.592000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:08", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:08", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:11.904000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:11", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:11", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:11.928000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:11", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:11", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:12.196000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:12", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:12", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:12.199000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:12", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:12", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:17.071000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:17", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:17", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:17.088000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:17", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:17", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:22.499000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:22", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:22", :level=>:warn}

and again and again