pfSense > NLS no logs

[nagiacct]

So I'm testing out this logging software (in esxi environment) and I created a lab and so far only thing I can't get data back from is the pfSense box. I ran a few TCPdumps and everything looks like its working on Nagios Logging Server (NLS) but when I run TCPdump from pfSense I get different output. I want to say it's a firewall thing but I can't corner it.

TCPdumps: (unable to copy/paste output at the moment)
1. on NLS, traffic looks like its correct syslog data is being sent to NLS (e.g. pfSense > NLS: SYSLOG local7.info, length: 64)
2. on pfSense, it shows syslog data passing to NLS...however, I also get this for example (e.g. NLS > pfSense: ICMP host <NLS IP> unreachable - admin prohibited, length 186)

What I've done:
1. Both boxes can ping each other and show routes
2. NLS is in the local LAN with firewall rule of LAN NET to any (Basically all boxes can talk to anyone, again lab environment)
3. I checked iptables on NLS and I don't see anything that would prohibit data to port 5544.
4. I noticed pfSense was sending data over port 514 to NLS:5544 so I changed the logstash.conf to be "root" per guide on Nagios.com.

[jolson]

I used pfSense in my home lab, and I got it working with Nagios Log Server with no problems.

By default pfSense will log using UDP, not TCP - and it uses a strange syslog format. What this means for us is that we should make a bare UDP input to accept pfSense logs.

2015-10-13 10_18_07-.png

Press 'Save and Apply' to apply this new input. Now, at the pfSense side of things, you'll want to redirect all of your logs to Nagios Log Server.

From pfSense:
Navigate to Status -> System Logs -> Settings. Once there, ensure that the proper IP address of Nagios Log Server is entered under the 'remote logging' section, and that the new appropriate UDP port follows. If you used port 1234 above as I did, your pfSense entry might look something like:

192.168.1.1:1234

That should be all she wrote. If I might recommend an excellent filter, you could start here (straight from the pfSense forum):

Code: Select all

if [host] =~ /192\.168\.1\.(1|2)/ {
    grok {
      match => [ 'message', '.* %{WORD:program}:%{GREEDYDATA:rest}' ]
    }
	
	if [program] == "filterlog" {
	  # Grab fields up to IP version. The rest will vary depending on IP version.
	  grok {  
		match => [ 'rest', '%{INT:rule_number},%{INT:sub_rule_number},,%{INT:tracker_id},%{WORD:interface},%{WORD:reason},%{WORD:action},%{WORD:direction},%{WORD:ip_version},%{GREEDYDATA:rest2}' ] 
	  }
	  
	  mutate {
		replace => [ 'message', '%{rest2}' ]
	  }
	  
	  if [ip_version] == "4" {
	    # IPv4. Grab field up to dest_ip. Rest can vary.
		grok {
		  match => [ 'message', '%{WORD:tos},(\d+)?,%{INT:ttl},%{INT:id},%{INT:offset},%{WORD:flags},%{INT:protocol_id},%{WORD:protocol},%{INT:length},%{IP:src_ip},%{IP:dest_ip},%{GREEDYDATA:rest3}' ]
		}
		
		if [protocol_id] != 2 {
		  # Non-IGMP has more fields.
		  grok {
		    match => [ 'rest3', '%{WORD:src_port},%{WORD:dest_port}' ]
		  }
		  
		}
	  } else {
	    # IPv6. Grab field up to dest_ip. Rest can vary.
		grok {
		  match => [ 'message', '%{WORD:class},%{WORD:flow_label},%{INT:hop_limit},%{WORD:protocol},%{INT:protocol_id},%{INT:length},%{IPV6:src_ip},%{IPV6:dest_ip},%{GREEDYDATA:rest3}' ]		
		}
		
		mutate {
		  replace => [ 'message', '%{rest3}' ]
		  lowercase => [ 'protocol' ]
		}
	  
		if [message] {
		  # Non-ICMP has more fields
  		  grok {
		    match => [ 'message', '%{INT:src_port},%{INT:dest_port},%{INT:data_length}' ]
		  }
		}
	  }
	  
	  mutate {
        	remove_field => [ 'message' ]
		remove_field => [ 'rest' ]
		remove_field => [ 'rest2' ]
		remove_field => [ 'rest3' ]
		remove_tag => [ '_grokparsefailure' ]
		add_tag => [ 'packetfilter' ]
	  }
	}
}

[nagiacct]

I tried the steps you provided but I'm still not getting any logs...I appreciate the help and filter can't wait to try it once its functional.

[jolson]

One step that I forgot: you'll need to open up that new port on your Nagios Log Server instance(s). For example if you set up UDP/1234 to listen on Nagios Log Server, you'll need to open up that port in the firewall. You can do that like so:

Code: Select all

iptables -I INPUT -p udp --dport 1234 -j ACCEPT
iptables-save

[nagiacct]

That did it! As for the filter I copied that in the filter section of the "Global Config" and saved it. How do I view/use this filter? Haven't had a lot of time to dive into the documentation yet. Thanks again for the help.

[jolson]

It all has to do with the first line:

Code: Select all

if [host] =~ /192\.168\.1\.(1|2)/ {

You can change it to match your input:

Code: Select all

if [type] == pfsense-logs {

When logs enter your input, they're tagged with 'pfsense-logs'. Then you define your filter with the same tag, which sucks logs of that type up.