Geo IP + Apache + Error Log + Bettermaps

[alucas]

Hello, I'm new in the forum.
I would like to ask if someone knows how to configure bettermaps with geoIP over "error logs", I already have the configuration over "access log" and it works great.
Thanks for your help.
Agustin.

[scottwilkerson]

Do your error_logs have an IP address field mapped to it's own geoIP address field?

[alucas]

Hi Scott, thanks for your answer.
Yes the client side apache error log has the "client ip" configured, then in the filters of LG we have these configured:
#geoip
geoip {
source => 'clientip'
}

#geoip_access
if [geoip_access] == 'apache_access' {
geoip {
source => 'clientip'
}
}

#geoip_error
if [geoip_error] == 'apache_error' {
geoip {
source => 'clientip'
}
}

Best regards,
agustin.

[scottwilkerson]

Are you seeing all the geoip fields for the error log entries?

[alucas]

No, that's strange, because I see this structure in the error log: timestamp + class + pid + client + message.

[Mon Sep 21 16:28:56.074781 2020] [autoindex:error] [pid 154366] [client xxx:xxx] AH01276: Cannot serve directory /WWW/html/: No matching DirectoryIndex

But the LS doesn't show the client ip in the error_log, the client ip is correctly shoed in the access_log. Because of that I think it doesn't show any geoip data.

Thanks.

[scottwilkerson]

Can you show the grok filter you have setup for the apache_error logs?

[alucas]

Yes, is the apache_default:

if [program] == 'apache_access' {
grok {
match => [ 'message', '%{COMBINEDAPACHELOG}']
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z', 'MMM d HH:mm:ss', 'MM M dd HH:mm:ss', 'ISO8601' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}

if [program] == 'apache_error' {
grok {
match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHD AY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] % {GREEDYDATA:errmsg}']
}
mutate {
replace => [ 'type', 'apache_error' ]
}
}

Regards

[scottwilkerson]

Code: Select all

grok {
match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHD AY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] % {GREEDYDATA:errmsg}']
}

This doesn't look like it is valid for your message

Code: Select all

[Mon Sep 21 16:28:56.074781 2020] [autoindex:error] [pid 154366] [client xxx:xxx] AH01276: Cannot serve directory /WWW/html/: No matching DirectoryIndex

I would check with the grok debugger
http://grokdebug.herokuapp.com/

I'm not 100% sure, but it may be because you have the microseconds decimal in the message and the %{TIME} pattern doesn't match that

Code: Select all

(?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])