Page 1 of 2
how to add more filter or modify filter
Posted: Fri Apr 29, 2016 3:57 am
by pccwglobalit
we need to modify filter and add response on apache combined log. where and how can we do that? thanks.
Re: how to add more filter or modify filter
Posted: Fri Apr 29, 2016 7:27 am
by eloyd
Filters can be found under Administration | Global Configuration.
Re: how to add more filter or modify filter
Posted: Fri Apr 29, 2016 9:11 am
by hsmith
Eric is right. Did you have anything specific you were trying to do?
Re: how to add more filter or modify filter
Posted: Tue May 03, 2016 3:56 am
by pccwglobalit
we have added response time on apache combined log. that means we have a new log fomat combinedlog + %D. now we can see there is combinedlog in filter but we don't know how to add %D to log format.
Re: how to add more filter or modify filter
Posted: Tue May 03, 2016 9:24 am
by hsmith
I'm sorry, but I'm having a little trouble understanding what you're trying to accomplish. Do you think you could show me a screenshot of what you have, and then an example of what you want?
Re: how to add more filter or modify filter
Posted: Tue May 03, 2016 9:51 am
by pccwglobalit
please check this article
http://unicolet.blogspot.hk/2014/09/ind ... h-elk.html
LogFormat "%h %l %u \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" extendedcombined
in nagios log server, the combined log without %D
we want to change combined log as the following:
EXTENDEDAPACHELOG %{SYSLOGTIMESTAMP:timestamp} %{GREEDYDATA:source} %{IPORHOST:clientip} %{USER:ident} %{USER:auth} "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) "%{GREEDYDATA:referer}" "%{GREEDYDATA:agent}" %{NUMBER:responsetime}
that is we add %{NUMBER:responsetime} to combined format.
Re: how to add more filter or modify filter
Posted: Tue May 03, 2016 10:49 am
by hsmith
Can you show me one of the Apache logs that is coming in to your server after you modified your Apache host? This will assist with the filter creation process. (Please copy and paste from the message field on NLS on one of your apache logs)
Thanks!
Re: how to add more filter or modify filter
Posted: Thu May 05, 2016 10:44 am
by pccwglobalit
192.168.99.145 - - [05/May/2016:00:02:03 +0000] "GET /healthCheck HTTP/1.1" 200 1 "-" "Java/1.7.0_79" "xxx.domin.com" "192.168.99.146" 45493
you can find the last column is response time.
Re: how to add more filter or modify filter
Posted: Thu May 05, 2016 10:59 am
by hsmith
Would you be willing to do a remote so we can look at this? It's likely this will take much less time to do over remote than over the forums.
Re: how to add more filter or modify filter
Posted: Thu May 05, 2016 11:06 am
by eloyd
I'm pretty sure you can just add what you want to the existing grok filter.
Go to Administration | Global Configuration and click on the Apache filter. Then change:
Code: Select all
if [program] == 'apache_access' {
grok {
match => [ 'message', '%{COMBINEDAPACHELOG}']
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}
to
Code: Select all
if [program] == 'apache_access' {
grok {
match => [ 'message', '%{COMBINEDAPACHELOG} %{INT:responseTime}']
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}
Note the line that changed is the MATCH line for the GROK filter.