Page 1 of 2
Timestamp error
Posted: Tue Jun 21, 2016 7:49 am
by Zaqen
Hi, i 'm new user with nagioslogserver.
I install it (.ova), and he run successfull, but i have this error in /var/log/logstach
Can you help if it's possible to resolv it ?
Re: Timestamp error
Posted: Tue Jun 21, 2016 9:44 am
by hsmith
Can you show me the error?
Re: Timestamp error
Posted: Wed Jun 22, 2016 7:05 am
by Zaqen
Hi,
yes i have this error in logstash log :
at: "Jun 22 08:29:23", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:25.473000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:25", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:25", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.205000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.218000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.893000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:31.895000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:31", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:31", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:35.642000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:35", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:35", :level=>:warn}
{:timestamp=>"2016-06-22T08:29:38.483000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 22 08:29:38", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 22 08:29:38", :level=>:warn}
Re: Timestamp error
Posted: Wed Jun 22, 2016 9:16 am
by hsmith
What kind of device are you getting logs from that is generating this error? This generally happens when the syslog input cannot parse the timestaps of the logs being sent to it.
Re: Timestamp error
Posted: Mon Jun 27, 2016 3:18 am
by Zaqen
Hi,
Is in logstach log, i have this errors when i have install nagioslogserver. I try to add machine but impossible to see it in dashboard, i think she send correctly log to nagioslogserver but timestamps is not good.
what must i do to resolv it ?
Re: Timestamp error
Posted: Mon Jun 27, 2016 9:48 am
by hsmith
One thing you can try is switching to using raw TCP/UDP for your inputs instead of the syslog input.
Go to Administration > Global > Global Configuration and expand the syslog input.
Replace what is there with this:
Code: Select all
tcp {
port => 5544
type => syslog
}
udp {
port => 5544
type => syslog
}
This will stop Logstash from dropping logs with a syslog format that it doesn't like, which is the default behavior.
Your syslogs will no longer be broken down in to nice fields, because we're not using the syslog input anymore, but we can have that process done with this grok filter:
Code: Select all
if [type] == "syslog" {
grok {
match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
}
}
Give this a shot.
Re: Timestamp error
Posted: Wed Jun 29, 2016 1:43 am
by Zaqen
Hi, thank you.
I put the first code in global, i do add input and paste first code :
Code: Select all
tcp {
port => 5544
type => syslog
}
udp {
port => 5544
type => syslog
}
i restart logstash but i have the same message in /var/log/logstash/logstash.log
impossible to see another server who i had.
what must i do with this ?
Code: Select all
if [type] == "syslog" {
grok {
match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
}
}
Re: Timestamp error
Posted: Wed Jun 29, 2016 1:49 am
by Zaqen
I found this in view > edit inputs
i add, it's ok ?
Code: Select all
#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Wed, 29 Jun 2016 08:47:14 +0200
#
#
# Global inputs
#
input {
syslog {
type => 'syslog'
port => 5544
}
tcp {
type => 'eventlog'
port => 3515
codec => json {
charset => 'CP1252'
}
}
tcp {
type => 'import_raw'
tags => 'import_raw'
port => 2056
}
tcp {
type => 'import_json'
tags => 'import_json'
port => 2057
codec => json
}
}
#
# Local inputs
#
tcp {
port => 5544
type => syslog
}
udp {
port => 5544
type => syslog
}
Re: Timestamp error
Posted: Wed Jun 29, 2016 1:58 am
by Zaqen
first, i edit Syslog entry in global > configuration
I stop (click on green activate button) syslog, click on + to add :
Code: Select all
tcp {
port => 5544
type => syslog
}
udp {
port => 5544
type => syslog
}
I save,
second i add new filter, her name is filter, i past this :
Code: Select all
if [type] == "syslog" {
grok {
match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
}
}
I save, and apply configuration, but i have message :
Code: Select all
The configuration could not be verified.
Ten minutes after :
I found this :
Code: Select all
The error reported is:
Couldn't find any input plugin named 'grok'. Are you sure this is correct? Trying to load the grok input plugin resulted in this error: no such file to load -- logstash/inputs/grok
Re: Timestamp error
Posted: Wed Jun 29, 2016 2:12 am
by Zaqen
I have same problem :
Code: Select all
{:timestamp=>"2016-06-29T09:11:08.592000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:08", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:08", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:11.904000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:11", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:11", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:11.928000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:11", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:11", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:12.196000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:12", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:12", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:12.199000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:12", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:12", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:17.071000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:17", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:17", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:17.088000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:17", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:17", :level=>:warn}
{:timestamp=>"2016-06-29T09:11:22.499000+0200", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"Jun 29 09:11:22", :exception=>java.lang.IllegalArgumentException: Invalid format: "Jun 29 09:11:22", :level=>:warn}
and again and again