Page 2 of 2
Re: Filter Help
Posted: Thu Apr 09, 2015 4:05 pm
by OptimusB
So I have been playing around with this and it has been a huge help. I am trying to set a couple IF statements in the filter....
I also found a pattern the match the initiate code that the log contains....
<164> = ^%{SYSLOG5424PRI}
Code: Select all
{
"SYSLOG5424PRI": [
[
"<164>"
]
],
"syslog5424_pri": [
[
"164"
]
so I am trying to set an IF statement that says if [syslog5424_pri] == '164' then do this.... but I am not able to get this working? I added an add_tag test statement to verify whether it went through which it does.... here's the code.... guess my If statement needs work?
Code: Select all
if [host] =~ /10\.242\.30\.33/ {
grok {
match => [ 'message', '%{SYSLOG5424PRI}' ]
add_tag => ['Firewall']
}
if [syslog5424_pri] == "164" {
grok {
match => [ 'message', '^%{SYSLOG5424PRI}Original Address=%{IP} %{CISCOTIMESTAMP} %{DATA:Rule} : %{DATA:garbage1}: %{WORD:Action} %{DATA:protocol} %{DATA:src_segment}:%{IP:src_IP} %{DATA:dst_segment}:%{IP:dst_IP} %{GREEDYDATA:garbage2} access-group "%{DATA:accessgroup}"' ]
add_tag => ['Type164']
}
mutate {
remove_field => [ 'garbage1', 'garbage2' ]
}
}
if [type] == "syslog" {
mutate{
add_tag => ['test']
}
}
}
Re: Filter Help
Posted: Thu Apr 09, 2015 5:04 pm
by jolson
Can you please give me your whole logstash config chain? It's hard for me to work with a few filters, being able to see the whole thing would be great:
Code: Select all
cat /usr/local/nagioslogserver/logstash/etc/conf.d/*
In addition to this, please supply an example log that you are matching.
Thanks OptimusB!
Re: Filter Help
Posted: Thu Apr 09, 2015 5:04 pm
by OptimusB
I think I found my issue... so <164> messages has two formats. The IPs can be IP or IP/Port.... is there a filter that I can use to determine this?
Code: Select all
Deny udp src inside:10.172.124.124/57970 dst outside:17.173.255.223/16386
Deny icmp src outside:142.52.142.33 dst dmz-AMN:10.5.24.7
Logs:
Code: Select all
<164>Original Address=172.30.253.153 Apr 09 2015 12:43:28 SIO-VFW-MPLS : %FWSM-4-106023: Deny icmp src outside:142.52.142.33 dst dmz-AMN:10.5.24.7 (type 8, code 0) by access-group "outside-in" [0x0, 0x0]
<164>Original Address=10.242.23.196 Apr 08 2015 12:47:30 APP-1619 : %ASA-4-106023: Deny udp src inside:10.172.124.124/57970 dst outside:17.173.255.223/16386 by access-group "inside-in" [0x46668482, 0x0]
Re: Filter Help
Posted: Thu Apr 09, 2015 5:26 pm
by jolson
If I understand you correctly, the following regex:
Code: Select all
^%{SYSLOG5424PRI}Original Address=%{IP} %{CISCOTIMESTAMP} %{DATA:Rule} : %{DATA:garbage1}: %{WORD:Action} %{DATA:protocol} %{DATA:src_segment}:%{IP:src_IP} %{DATA:dst_segment}:%{IP:dst_IP} %{GREEDYDATA:garbage2} access-group "%{DATA:accessgroup}"
Does not accommodate IP/Port combinations. We can fix that with some additional regex. For example:
Code: Select all
Original Address=%{IP}\/?\d* %{CISCOTIMESTAMP}
\/? will optionally match '/' and \d* will match 0 or more 'decimals'.
Re: Filter Help
Posted: Thu Apr 09, 2015 5:40 pm
by OptimusB
Thanks again!
Can I make that optional 'decimal' into a field? ie %{NUMBER} if it is there?
Re: Filter Help
Posted: Fri Apr 10, 2015 9:11 am
by jolson
I have not done this before, but I found this online:
https://groups.google.com/forum/#!topic ... UY3gCggtAI
It looks like he's had luck escaping a field with the 'pipe' symbol. In that case, it could look something like:
Code: Select all
Original Address=%{IP}\/?\|%{NUMBER:port} %{CISCOTIMESTAMP}
Re: Filter Help
Posted: Fri Apr 10, 2015 11:44 am
by OptimusB
I could not get that to work and I think the pipe does not apply in our case....
I was testing this using the grok debugger and couldn't get one that worked for both scenarios.
scenario 1: <ip>
scenario 2: <ip>/<port>
So I modified the filter as follows:
Code: Select all
grok {
match => [ 'message', '%{IP}\/%{NUMBER}']
match => ['message', '%{IP}']
}
This way if there's a port it will match first statement and break. If it doesn't have a port on the IP, it will match the second statement.
Not sure if there's a more efficient way but this worked perfectly.
Re: Filter Help
Posted: Fri Apr 10, 2015 11:53 am
by jolson
Perfect - It seems like you're getting the hang of this.
Do you have any further questions that I could help you with?
Re: Filter Help
Posted: Fri Apr 10, 2015 12:08 pm
by OptimusB
I am ok for now. Thanks again for the assistance. This had really helped us build the dashboards and get better visualization.
Thread can be locked. Cheers.
Re: Filter Help
Posted: Fri Apr 10, 2015 12:19 pm
by jolson
You are welcome - closing now.