pfsense 2.4.5

[tim.kroeger]

Does anyone have any filters that work with pfsense 2.4.5 to help make sense of the firewall logs?

I've been trying to use the filter I found on the pfsense forums, but I keep having problems actually seeing anything in the log after I enable this filter. The filter was written for pfsense 2.2 logs and I'm guessing the format has changed just enough to break it, but I cant find the issue.

This is my first go at a filter in NLS and any help would be appreciated.

[cdienger]

https://assets.nagios.com/downloads/nag ... ilters.pdf has some guidance.

We can assist as well if you'd like to post a sample of a log you're trying to parse.

[tim.kroeger]

I've spent a lot of time looking at that document. Thanks for putting it together. I'm still no closer than I was when I started on this.

Below is the raw log I'm trying to parse. I'm specifically trying to parse the message field so I can filter on the following fields: protocol, source IP, destination IP, source port and destination port. These logs are always going to come from the same IP address.


{
"_index": "logstash-2020.10.16",
"_type": "syslog",
"_id": "AXUyz2BwfiO4RVdgdHFh",
"_score": null,
"_source": {
"message": "<134>Oct 16 19:07:32 filterlog: 5,,,1000000103,lagg0.4090,match,block,in,4,0x0,,243,27854,0,none,6,tcp ,40,192.168.1.1,192.168.101.31,44064,60826,0,S,1850831513,,1024,,",
"@version": "1",
"@timestamp": "2020-10-16T19:07:32.535Z",
"type": "syslog",
"host": "172.16.254.2",
"tags": [
"_grokparsefailure_sysloginput"
],
"priority": 0,
"severity": 0,
"facility": 0,
"facility_label": "kernel",
"severity_label": "Emergency"
},
"highlight": {
"host.raw": [
"@[email protected]@end-highlight@"
],
"host": [
"@[email protected]@end-highlight@"
]
},
"sort": [
1602875252535,
1602875252535
]
}

Thanks a ton!

[cdienger]

I found these links which were extremely helpful and time saving:

https://docs.netgate.com/pfsense/en/lat ... ormat.html
https://raw.githubusercontent.com/patri ... se2-4.grok

The github link in particular saved me lot of work. I wasn't able to figure out the entire log line, but did get a majority parsed with this input and filter:

Code: Select all

tcp {
    type => 'pfsense'
    port => 2099
}

Code: Select all

if [type] == 'pfsense' {
grok {
patterns_dir => '/usr/local/nagioslogserver/etc/patterns'
        match => { "message" => "%{SYSLOG5424PRI}%{CISCOTIMESTAMP} filterlog: %{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}" }
      }
}

The patterns found in the github link were added to a file(I created one called pfsense.patterns) under /usr/local/nagioslogserver/etc/patterns/. I did have to make a couple small changes to it to get it to parse correctly - I changed:

Code: Select all

PFSENSE_IPv4_SPECIFIC_DATA (%{BASE16NUM:tos}),,(%{INT:ttl}),(%{INT:id}),(%{INT:offset}),(%{WORD:flags}),(%{INT:proto_id}),(%{WORD:proto}),

to:

Code: Select all

PFSENSE_IPv4_SPECIFIC_DATA (0x%{BASE16NUM:tos}),,(%{INT:ttl}),(%{INT:id}),(%{INT:offset}),(%{WORD:flags}),(%{INT:proto_id}),(%{WORD:proto}) ,

[tim.kroeger]

Thanks a ton! Appreciate the help.

[cdienger]

Let us know once you've had a chance to test and if there is anything further we can help with on this.