Page 1 of 1
Geo IP + Apache + Error Log + Bettermaps
Posted: Wed Sep 16, 2020 1:56 pm
by alucas
Hello, I'm new in the forum.
I would like to ask if someone knows how to configure bettermaps with geoIP over "error logs", I already have the configuration over "access log" and it works great.
Thanks for your help.
Agustin.
Re: Geo IP + Apache + Error Log + Bettermaps
Posted: Thu Sep 17, 2020 1:59 pm
by scottwilkerson
Do your error_logs have an IP address field mapped to it's own geoIP address field?
Re: Geo IP + Apache + Error Log + Bettermaps
Posted: Mon Sep 21, 2020 11:16 am
by alucas
Hi Scott, thanks for your answer.
Yes the client side apache error log has the "client ip" configured, then in the filters of LG we have these configured:
#geoip
geoip {
source => 'clientip'
}
#geoip_access
if [geoip_access] == 'apache_access' {
geoip {
source => 'clientip'
}
}
#geoip_error
if [geoip_error] == 'apache_error' {
geoip {
source => 'clientip'
}
}
Best regards,
agustin.
Re: Geo IP + Apache + Error Log + Bettermaps
Posted: Mon Sep 21, 2020 11:28 am
by scottwilkerson
Are you seeing all the geoip fields for the error log entries?
Re: Geo IP + Apache + Error Log + Bettermaps
Posted: Mon Sep 21, 2020 2:50 pm
by alucas
No, that's strange, because I see this structure in the error log: timestamp + class + pid + client + message.
[Mon Sep 21 16:28:56.074781 2020] [autoindex:error] [pid 154366] [client xxx:xxx] AH01276: Cannot serve directory /WWW/html/: No matching DirectoryIndex
But the LS doesn't show the client ip in the error_log, the client ip is correctly shoed in the access_log. Because of that I think it doesn't show any geoip data.
Thanks.
Re: Geo IP + Apache + Error Log + Bettermaps
Posted: Mon Sep 21, 2020 3:15 pm
by scottwilkerson
Can you show the grok filter you have setup for the apache_error logs?
Re: Geo IP + Apache + Error Log + Bettermaps
Posted: Tue Sep 22, 2020 9:33 am
by alucas
Yes, is the apache_default:
if [program] == 'apache_access' {
grok {
match => [ 'message', '%{COMBINEDAPACHELOG}']
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z', 'MMM d HH:mm:ss', 'MM M dd HH:mm:ss', 'ISO8601' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}
if [program] == 'apache_error' {
grok {
match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHD AY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] % {GREEDYDATA:errmsg}']
}
mutate {
replace => [ 'type', 'apache_error' ]
}
}
Regards
Re: Geo IP + Apache + Error Log + Bettermaps
Posted: Tue Sep 22, 2020 10:29 am
by scottwilkerson
Code: Select all
grok {
match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHD AY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] % {GREEDYDATA:errmsg}']
}
This doesn't look like it is valid for your message
Code: Select all
[Mon Sep 21 16:28:56.074781 2020] [autoindex:error] [pid 154366] [client xxx:xxx] AH01276: Cannot serve directory /WWW/html/: No matching DirectoryIndex
I would check with the grok debugger
http://grokdebug.herokuapp.com/
I'm not 100% sure, but it may be because you have the microseconds decimal in the message and the %{TIME} pattern doesn't match that
Code: Select all
(?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])