Apache Filter stopped working when adding TLS

This board serves as an open discussion and support collaboration point for Nagios Log Server. NOTE: Nagios Log Server customers should use the Customer Support forum to obtain expedited support.

Apache Filter stopped working when adding TLS

Postby tke395 » Fri Jul 13, 2018 2:35 pm

I set up a CentOS 7 server with apache to forward access log and error log files to Nagios Log Server using rsyslog. I used the scripts from NLS to configure rsyslog on the client. Everything worked as expected. Log entries are showing up as apache_access and apache_error.

When I set up TLS, I added another input for syslog over tcp:
Code: Select all
tcp {
    port => 8888
    type => 'syslog'
    ssl_extra_chain_certs => ['/etc/pki/tls/certs/ca.pem']
    ssl_cert => '/etc/pki/tls/certs/device-nls.crt'
    ssl_key =>  '/etc/pki/tls/private/device-nls.key'
    ssl_enable => true
    ssl_verify => false
}


I added the following to the client config:
Code: Select all
#tls setup
$DefaultNetstreamDriverCAFile /etc/rsyslog.d/ca.pem
$ActionSendStreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name


Now, the log messages are still loaded into NLS, but they don't appear to be running through the Apache filter. They are all identified as "syslog" and the program value for apache_access or apache_error is not present.

As far as I can tell, I didn't do anything to change the format of the messages, only the transport. However, something must be different. Any ideas?

Thanks,
Eric
tke395
 
Posts: 3
Joined: Fri Jul 13, 2018 2:24 pm

Re: Apache Filter stopped working when adding TLS

Postby cdienger » Fri Jul 13, 2018 4:33 pm

We can enable debug logging on logstash to hopefully determine why it isn't triggering filters the way we'd expect. To do this edit /etc/init.d/logstash and change line 64 from:

DAEMON_OPTS="agent -f ${LS_CONF_DIR} -l ${LS_LOG_FILE} ${LS_OPTS}"

to:

DAEMON_OPTS="agent -f ${LS_CONF_DIR} -l ${LS_LOG_FILE} ${LS_OPTS} --debug"

and then run:

systemctl daemon-reload
service logstash restart


/var/log/logstash/logstash.log will grow pretty quickly after this so only have it enabled just long enough to have some logs be sent over and then revert the change and restart again. Feel free to PM me the logstash.log if needed.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
cdienger
Support Tech
 
Posts: 2089
Joined: Tue Feb 07, 2017 11:26 am

Re: Apache Filter stopped working when adding TLS

Postby tke395 » Mon Jul 16, 2018 2:04 pm

Thank you for your suggestion. I enabled debug logging and was able to verify that the filter was not triggered. However, I was not able to figure out why.

We stripped the TLS configuration out to eliminate that as a possible cause. At this point the only difference is that the working input uses syslog:
Code: Select all
syslog {
    type => 'syslog'
    port => 5544
}


The non-working input uses syslog over TCP:
Code: Select all
tcp {
    port => 7778
    type => 'syslog'
}


Is there any documentation on how the syslog message handling for these inputs differs? It appears that the "<%{POSINT:priority}>%{SYSLOGLINE}" pattern is not matched on the tcp syslog input. Then the [program] == 'apache_access' match fails on the Apache filter because program is empty.
tke395
 
Posts: 3
Joined: Fri Jul 13, 2018 2:24 pm

Re: Apache Filter stopped working when adding TLS

Postby cdienger » Mon Jul 16, 2018 3:54 pm

Ahh, somehow I missed the input type as being tcp. In this case then you'll want to account for the syslog formatting. The syslog input expects data following rfc3164: https://www.ietf.org/rfc/rfc3164.txt.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
cdienger
Support Tech
 
Posts: 2089
Joined: Tue Feb 07, 2017 11:26 am

Re: Apache Filter stopped working when adding TLS

Postby tke395 » Wed Jul 18, 2018 1:06 pm

Thanks. We got it working by adding a filter. For some reason I assumed that adding type=>'syslog' to the tcp input would process the inputs as syslog messages.
tke395
 
Posts: 3
Joined: Fri Jul 13, 2018 2:24 pm

Re: Apache Filter stopped working when adding TLS

Postby scottwilkerson » Thu Jul 19, 2018 8:18 am

tke395 wrote:Thanks. We got it working by adding a filter. For some reason I assumed that adding type=>'syslog' to the tcp input would process the inputs as syslog messages.


Glad to hear you got it working
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 12331
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises


Return to Nagios Log Server

Who is online

Users browsing this forum: No registered users and 4 guests