Filter Help

[OptimusB]

So I have been playing around with this and it has been a huge help. I am trying to set a couple IF statements in the filter....

I also found a pattern the match the initiate code that the log contains....

<164> = ^%{SYSLOG5424PRI}

Code: Select all

{
  "SYSLOG5424PRI": [
    [
      "<164>"
    ]
  ],
  "syslog5424_pri": [
    [
      "164"
    ]

so I am trying to set an IF statement that says if [syslog5424_pri] == '164' then do this.... but I am not able to get this working? I added an add_tag test statement to verify whether it went through which it does.... here's the code.... guess my If statement needs work?

Code: Select all

if [host] =~ /10\.242\.30\.33/  {
    grok {
        match => [ 'message', '%{SYSLOG5424PRI}' ]
		add_tag => ['Firewall']
    }
	if [syslog5424_pri] == "164" {
		grok {
			match => [ 'message', '^%{SYSLOG5424PRI}Original Address=%{IP} %{CISCOTIMESTAMP} %{DATA:Rule} : %{DATA:garbage1}: %{WORD:Action} %{DATA:protocol} %{DATA:src_segment}:%{IP:src_IP} %{DATA:dst_segment}:%{IP:dst_IP} %{GREEDYDATA:garbage2} access-group "%{DATA:accessgroup}"' ]
			add_tag => ['Type164']
		}
		mutate {
			remove_field => [ 'garbage1', 'garbage2' ]
		}
	}
	if [type] == "syslog" {
		mutate{
			add_tag => ['test']
			}
		}
}

[jolson]

Can you please give me your whole logstash config chain? It's hard for me to work with a few filters, being able to see the whole thing would be great:

Code: Select all

cat /usr/local/nagioslogserver/logstash/etc/conf.d/*

In addition to this, please supply an example log that you are matching.

Thanks OptimusB!

[OptimusB]

I think I found my issue... so <164> messages has two formats. The IPs can be IP or IP/Port.... is there a filter that I can use to determine this?

Code: Select all

Deny udp src inside:10.172.124.124/57970 dst outside:17.173.255.223/16386
Deny icmp src outside:142.52.142.33 dst dmz-AMN:10.5.24.7

Logs:

Code: Select all

<164>Original Address=172.30.253.153 Apr 09 2015 12:43:28 SIO-VFW-MPLS : %FWSM-4-106023: Deny icmp src outside:142.52.142.33 dst dmz-AMN:10.5.24.7 (type 8, code 0) by access-group "outside-in" [0x0, 0x0]

<164>Original Address=10.242.23.196 Apr 08 2015 12:47:30 APP-1619 : %ASA-4-106023: Deny udp src inside:10.172.124.124/57970 dst outside:17.173.255.223/16386 by access-group "inside-in" [0x46668482, 0x0]

[jolson]

If I understand you correctly, the following regex:

Code: Select all

^%{SYSLOG5424PRI}Original Address=%{IP} %{CISCOTIMESTAMP} %{DATA:Rule} : %{DATA:garbage1}: %{WORD:Action} %{DATA:protocol} %{DATA:src_segment}:%{IP:src_IP} %{DATA:dst_segment}:%{IP:dst_IP} %{GREEDYDATA:garbage2} access-group "%{DATA:accessgroup}"

Does not accommodate IP/Port combinations. We can fix that with some additional regex. For example:

Code: Select all

Original Address=%{IP}\/?\d* %{CISCOTIMESTAMP}

\/? will optionally match '/' and \d* will match 0 or more 'decimals'.

[OptimusB]

Thanks again!
Can I make that optional 'decimal' into a field? ie %{NUMBER} if it is there?

[jolson]

I have not done this before, but I found this online: https://groups.google.com/forum/#!topic ... UY3gCggtAI

It looks like he's had luck escaping a field with the 'pipe' symbol. In that case, it could look something like:

Code: Select all

Original Address=%{IP}\/?\|%{NUMBER:port} %{CISCOTIMESTAMP}

[OptimusB]

I could not get that to work and I think the pipe does not apply in our case....

I was testing this using the grok debugger and couldn't get one that worked for both scenarios.

scenario 1: <ip>
scenario 2: <ip>/<port>

So I modified the filter as follows:

Code: Select all

grok {
  match => [ 'message', '%{IP}\/%{NUMBER}']
  match => ['message', '%{IP}']
}

This way if there's a port it will match first statement and break. If it doesn't have a port on the IP, it will match the second statement.
Not sure if there's a more efficient way but this worked perfectly.

[jolson]

Perfect - It seems like you're getting the hang of this.

Do you have any further questions that I could help you with?

[OptimusB]

I am ok for now. Thanks again for the assistance. This had really helped us build the dashboards and get better visualization.
Thread can be locked. Cheers.

[jolson]

You are welcome - closing now.