how to add more filter or modify filter

[pccwglobalit]

we need to modify filter and add response on apache combined log. where and how can we do that? thanks.

[eloyd]

Filters can be found under Administration | Global Configuration.

[hsmith]

Eric is right. Did you have anything specific you were trying to do?

[pccwglobalit]

we have added response time on apache combined log. that means we have a new log fomat combinedlog + %D. now we can see there is combinedlog in filter but we don't know how to add %D to log format.

[hsmith]

I'm sorry, but I'm having a little trouble understanding what you're trying to accomplish. Do you think you could show me a screenshot of what you have, and then an example of what you want?

[pccwglobalit]

please check this article http://unicolet.blogspot.hk/2014/09/ind ... h-elk.html


LogFormat "%h %l %u \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" extendedcombined


in nagios log server, the combined log without %D

we want to change combined log as the following:
EXTENDEDAPACHELOG %{SYSLOGTIMESTAMP:timestamp} %{GREEDYDATA:source} %{IPORHOST:clientip} %{USER:ident} %{USER:auth} "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) "%{GREEDYDATA:referer}" "%{GREEDYDATA:agent}" %{NUMBER:responsetime}

that is we add %{NUMBER:responsetime} to combined format.

[hsmith]

Can you show me one of the Apache logs that is coming in to your server after you modified your Apache host? This will assist with the filter creation process. (Please copy and paste from the message field on NLS on one of your apache logs)

Thanks!

[pccwglobalit]

192.168.99.145 - - [05/May/2016:00:02:03 +0000] "GET /healthCheck HTTP/1.1" 200 1 "-" "Java/1.7.0_79" "xxx.domin.com" "192.168.99.146" 45493

you can find the last column is response time.

[hsmith]

Would you be willing to do a remote so we can look at this? It's likely this will take much less time to do over remote than over the forums.

[eloyd]

I'm pretty sure you can just add what you want to the existing grok filter.

Go to Administration | Global Configuration and click on the Apache filter. Then change:

Code: Select all

if [program] == 'apache_access' {
    grok {
        match => [ 'message', '%{COMBINEDAPACHELOG}']
    }
    date {
        match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z' ]
    }
    mutate {
        replace => [ 'type', 'apache_access' ]
         convert => [ 'bytes', 'integer' ]
         convert => [ 'response', 'integer' ]
    }
}

to

Code: Select all

if [program] == 'apache_access' {
    grok {
        match => [ 'message', '%{COMBINEDAPACHELOG} %{INT:responseTime}']
    }
    date {
        match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z' ]
    }
    mutate {
        replace => [ 'type', 'apache_access' ]
         convert => [ 'bytes', 'integer' ]
         convert => [ 'response', 'integer' ]
    }
}

Note the line that changed is the MATCH line for the GROK filter.