need help filtering syslogs

[benhank]

Hey guys!

Guys I promise (you know something wrong when I start off a post like this...=) ) that I won't ask this again, but can you guys show me how (by doing it for me) to take these logs and turn the terms ive listed into fields? The logs are sent as syslogs.

Code: Select all

<14>1 2019-02-07T11:15:29.550942-05:00 vidyoprtl2 java - - - VidyoPortal [audit result="SUCCESS" ip="1.1.4.7" action="Login" params="Username: super" user="super" tenant="Default" timestamp="Thu Feb 07 11:15:29 EST 2019"]

<14>1 2019-02-06T08:41:55.632196-05:00 vidyoprtl2 java - - - VidyoPortal [audit result="FAILURE" ip="1.2.1.1" action="Login" params="Username: rjones" user="rjones" tenant="Connect" timestamp="Wed Feb 06 08:41:55 EST 2019"]

<14>1 2019-02-07T13:30:41.905970-05:00 vidyoprtl2 java - - - VidyoPortal [audit result="SUCCESS" ip="1.1.4.7" action="Logout" params="Username=admin" user="admin" tenant="Default" timestamp="Thu Feb 07 13:30:41 EST 2019"]

I want to make the following into Fields and if possible use geo ips so that I can make a dashboard using geo maps's:
audit result=
ip=
action=
params="Username:
params=
user=
tenant=

Thanks fellas!

[npolovenko]

Hello, @benhank. Are these logs going through the Syslog input on the Log Server? Because if they are there should be some automatic parsing. In that case, I'd like to see how the message is currently parsed. Please go to the Dashboards menu, click on the event, take a screenshot and upload it in this thread.

But to satisfy the given criteria, you may try using the following pattern:

result="%{WORD:result}" ip="%{IP:IP}" action="%{WORD:action}" params="Username: %{USER:params}" user="%{USER:user}" tenant="%{WORD:tenant}"

[benhank]

here you go:
The log server does a good job of parsing the data, I just wanted to add to what is being done already:

Capture.PNG

So do I just copy what you created into the filters section like so?:

Capture.PNG

How do I tell the logserver to convert the syslog ip's to geo locations?

After we are done this will go a looong way towards me learning how to do this myself. I truly appreciate the help.

[npolovenko]

@benhank, Because the syslog filter doesn't parse these logs correctly I suggest forwarding these logs to the port 2056 instead (import_raw).
Then create a custom filter:

if [host] == '172.30.100.226' {

grok {

match => { "message" => "result=\"%{WORD:result}\" ip=\"%{IP:IP}\" action=\"%{WORD:action}\" params=\"Username: %{USER:params}\" user=\"%{USER:user}\" tenant=\"%{WORD:tenant}\""}
match => { "message" => "result=\"%{WORD:result}\" ip=\"%{IP:IP}\" action=\"%{WORD:action}\" params=\"Username=%{USER:params}\" user=\"%{USER:user}\" tenant=\"%{WORD:tenant}\""}
}

}

[benhank]

Should I add:

Code: Select all

geoip {
  source => "IP"

to get the geoip?

[benhank]

Hey thanks adding this to the custom filter :

Code: Select all

if [host] == '172.30.100.226' {

grok {

match => { "message" => "result=\"%{WORD:result}\" ip=\"%{IP:IP}\" action=\"%{WORD:action}\" params=\"Username: %{USER:params}\" user=\"%{USER:user}\" tenant=\"%{WORD:tenant}\""}
match => { "message" => "result=\"%{WORD:result}\" ip=\"%{IP:IP}\" action=\"%{WORD:action}\" params=\"Username=%{USER:params}\" user=\"%{USER:user}\" tenant=\"%{WORD:tenant}\""}
}

}\

sorted the data the way I needed. Now I can use this as a template for future filters!
thanks man!

[npolovenko]

@benhank, You're welcome. And yes, your suggested syntax for the geoip is correct.

if [host] == '172.30.100.226' {

grok {

match => { "message" => "result=\"%{WORD:result}\" ip=\"%{IP:IP}\" action=\"%{WORD:action}\" params=\"Username: %{USER:params}\" user=\"%{USER:user}\" tenant=\"%{WORD:tenant}\""}
match => { "message" => "result=\"%{WORD:result}\" ip=\"%{IP:IP}\" action=\"%{WORD:action}\" params=\"Username=%{USER:params}\" user=\"%{USER:user}\" tenant=\"%{WORD:tenant}\""}
}
geoip {
source => "IP"
}
}

[benhank]

thanks man you can lock it up I REALLY appreciate this . I learned alot!

[npolovenko]

@benhank, No problem!
Closing the thread.