apache log combined log

[pccwglobalit]

the below is our apache combined log

"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{Host}i\" \"%{X-Forwarded-For}i\" %D"

the original combined log without \"%{Host}i\" \"%{X-Forwarded-For}i\" %D

how can i add filter in logstash so that i can see the three parameters.

Host is ip or hostname
X-Forwarded-For is ip
%D is response time and it is INT

thanks.

[hsmith]

Can you please show me the entire grok filter?

[pccwglobalit]

if [program] == 'apache_access' {
grok {
match => [ 'message', '%{COMBINEDAPACHELOG} %{INT:responseTime}']
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}

if [program] == 'apache_error' {
grok {
match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] %{GREEDYDATA:errmsg}']
}
mutate {
replace => [ 'type', 'apache_error' ]
}
}

[hsmith]

All right, thanks for posting that.

I'm still a little bit confused about exactly what you're trying to achieve.

how can i add filter in logstash so that i can see the three parameters.

Host is ip or hostname
X-Forwarded-For is ip
%D is response time and it is INT

You want these to show up as a field in your log?

[pccwglobalit]

yes, i want show host and x-forward-for

[hsmith]

I want to see exactly what is happening. I think we'll resolve this one faster if we do a remote session.

Please send a ticket in to [email protected]. I will take ownership of the ticket, and we can schedule a remote.

Thanks!