Adding tags with grok

[WillemDH]

Hello,

As some of my syslogs keep getting the tag grokparsefailures, I was thinking of adding a custom tag, like 'grokked' to the syslog messages. it seems i'm not able to make it work though.
They keep getting the 'tag on failure' _grokparsefailure tag http://logstash.net/docs/1.4.2/filters/ ... on_failure

I've tried like this:

Code: Select all

if [type] == "syslog-f5" {
    grok {     
      add_tag => "grokked"
    }   
  }

and of course saved and applied. What am I doing wrong to get rid of these grookparsefailures?

Grtz

Willem

[jolson]

Hello,

First thing - let's make sure that your filter is applying properly to your nodes. Please run the following command on each of your nodes:

Code: Select all

cat /usr/local/nagioslogserver/logstash/etc/conf.d/500_filters.conf

I just want to make sure we're not missing something obvious. Once you have those results back, we'll take it from there. Thanks!

[WillemDH]

Jolson:

Code: Select all

filter {
    if [program] == 'apache_access' {
        grok {
            match => [ 'message', '%{COMBINEDAPACHELOG}']
        }
        date {
            match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z' ]
        }
        mutate {
            replace => [ 'type', 'apache_access' ]
             convert => [ 'bytes', 'integer' ]
             convert => [ 'response', 'integer' ]
        }
    }

    if [program] == 'apache_error' {
        grok {
            match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] %{GREEDYDATA:errmsg}']
        }
        mutate {
            replace => [ 'type', 'apache_error' ]
        }
    }
    if [type] == "syslog-brocade" {
        grok {
          match => { "message" => "<[\d]+>[a-z]+ [\d]+ [\d\:]+ %{IPV4:logsource}%{GREEDYDATA:program}: %{YEAR}\/%{MONTHNUM}\/%{MONTHDAY}-%{TIME}%{GREEDYDATA}WWN %{IPV6:wwn}%{GREEDYDATA}%{LOGLEVEL}\, %{HOSTNAME:hostname}" }
         add_tag => "grokked"
        }
      }

    if [type] == "syslog-f5" {
        grok {
          add_tag => "grokked"
        }
      }

}

#
# Local filters
#

It seems I maanged to add the tag grokked to syslog-brocade type. But the _grokparsefailure is still there. Do I ahve to do a remove_tag? (see my other post...)

[jolson]

Willem,

Please see my response here: http://support.nagios.com/forum/viewtop ... 10#p130715

I believe that my response located there is appropriate for this thread as well. Quoted below:


The _grokparsefailure tag:

Append values to the ‘tags’ field when there has been no successful match

If it is helpful, you can also add the following tag to each Grok to not add any tag on failure.

Code: Select all

tag_on_failure => []

Looking around the internet, this is often caused by unescaped double quotes or slightly improper characters.
http://stackoverflow.com/questions/2240 ... eing-happy

Use the Grok debugger tool to figure out what might be the cause:
https://grokdebug.herokuapp.com/
If you cannot find out what might be causing the parsefailure tag, I would like you to post your current input filter and an example log - I would be happy to give it a shot.

[WillemDH]

Jolson, Ok you can close this thread. W'll continue in the other thread. Tx