drop logs with empty message

[WillemDH]

Hello,

We have a logsource which sometimes sends empty logs. We would like to drop these if the message field is empty. How would we do this?

Code: Select all

if [type] == "syslog-crushftp" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}:%{GREEDYDATA:syslog_message}" }
    }
    mutate {
        lowercase => [ "program" ]
        add_field => { 
          "logsource" => "%{syslog_hostname}"
        }
    }
    grok {
      match => { "syslog_message" => "\A%{DATESTAMP}\|\[%{HOSTNAME}:%{HTTPDUSER:ftp_session}:%{HTTPDUSER:ftp_user}:%{IP:source_ip}%{GREEDYDATA:rest_message}" }
      add_tag => "grokked_syslog_crushftp"
    }
   grok {
      match => { "syslog_message" => "" }
      add_tag => "grokked_syslog_crushftp_empty"
    }
}

As you can see, I tried with

Code: Select all

match => { "syslog_message" => "" }

but I noticed some logs which do contain a message also get the grokked_syslog_crushftp_empty tag...

Grtz

Willem

[mcapra]

This worked for me:

Code: Select all

if [message] =~ /^\s*$/ {
  drop{}
}
else {
 #process the logs
}
 

Obviously your idea of tagging instead of dropping while doing testing is a good idea. drop{} can be a bit intimidating.

[WillemDH]

Thank you very much. It works perfect. Please close this thread.