Page 1 of 1
TLS syslog messages not seen on Dashboard
Posted: Wed May 20, 2020 3:57 pm
by acwong
I've recently configured TLS between the nagios log server and a client linux box. TCP syslog messages sent from client are seen on the nagios dashboard. However, TLS syslog messages are not being received on the dashboard of nagios log server. I've used tcpdump and confirmed that TLS traffic is being received on the nagios log server, but not seen on the dashboard. Where should I start troubleshooting?
Re: TLS syslog messages not seen on Dashboard
Posted: Thu May 21, 2020 12:50 pm
by cdienger
I'd enable logstash debugging to troubleshoot this.
Edit /etc/init.d/logstash and change line 64 from:
Code: Select all
DAEMON_OPTS="agent -f ${LS_CONF_DIR} -l ${LS_LOG_FILE} ${LS_OPTS}"
to:
Code: Select all
DAEMON_OPTS="agent -f ${LS_CONF_DIR} -l ${LS_LOG_FILE} ${LS_OPTS} --debug"
and restart the service with:
Code: Select all
systemctl daemon-reload
service logstash restart
Let this run just long enough to allow NLS to process some TLS events from the host and then collect the /var/log/logstash/logstash.log file before reverting the config back.
Re: TLS syslog messages not seen on Dashboard
Posted: Thu May 21, 2020 4:02 pm
by acwong
Looks like a CA issue.

I've configured nagios syslog server as the CA. I followed the "Sending syslog With SSL/TLS" nagios documentation.
https://assets.nagios.com/downloads/nag ... th-SSL.pdf
Code: Select all
{:timestamp=>"2020-05-21T13:27:50.820000-0700", :message=>"SSL Error", :exception=>#<OpenSSL::SSL::SSLError: Received fatal alert: unknown_ca>, :backtrace=>["org/jruby/ext/openssl/SSLSocket.java:272:in `accept'", "/usr/local/nagioslogserver/logstash/vendor/jruby/lib/ruby/shared/jopenssl19/openssl/ssl-internal.rb:106:in `accept'", "/usr/local/nagioslogserver/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-tcp-3.0.6/lib/logstash/inputs/tcp.rb:112:in `run_server'", "/usr/local/nagioslogserver/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-tcp-3.0.6/lib/logstash/inputs/tcp.rb:84:in `run'", "/usr/local/nagioslogserver/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:342:in `inputworker'", "/usr/local/nagioslogserver/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:336:in `start_input'"], :level=>:debug, :file=>"logstash/inputs/tcp.rb", :line=>"117", :method=>"run_server"}
Re: TLS syslog messages not seen on Dashboard
Posted: Fri May 22, 2020 9:43 am
by cdienger
Double check the configuration on the sending device to make sure it is pointing to the correct syslog-ca.pem and also that the syslog-ca.pem on the sending device matches the one in /etc/pki/tls/certs/syslog-ca.pem on the NLS machine.
Re: TLS syslog messages not seen on Dashboard
Posted: Wed May 27, 2020 7:35 pm
by acwong
After double checking the certs, there was a mismatch on a cert credentials which I fixed. Now I I have another issue.
Code: Select all
{:timestamp=>"2020-05-27T17:13:04.723000-0700", :message=>"SSL Error", :exception=>#<OpenSSL::SSL::SSLError: Received fatal alert: decrypt_error>, :backtrace=>["org/jruby/ext/openssl/SSLSocket.java:272:in `accept'", "/usr/local/nagioslogserver/logstash/vendor/jruby/lib/ruby/shared/jopenssl19/openssl/ssl-internal.rb:106:in `accept'", "/usr/local/nagioslogserver/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-tcp-3.0.6/lib/logstash/inputs/tcp.rb:112:in `run_server'", "/usr/local/nagioslogserver/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-tcp-3.0.6/lib/logstash/inputs/tcp.rb:84:in `run'", "/usr/local/nagioslogserver/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:342:in `inputworker'", "/usr/local/nagioslogserver/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:336:in `start_input'"], :level=>:debug, :file=>"logstash/inputs/tcp.rb", :line=>"117", :method=>"run_server"}
Re: TLS syslog messages not seen on Dashboard
Posted: Thu May 28, 2020 10:56 am
by cdienger
Please attach a copy of the syslog config as well as the NLS input's config.
I'd also like to see a tcpdump taken on the NLS machine:
Code: Select all
yum -y install tcpdump
tcpdump -s 0 -i any host w.x.y.z -w output.pcap
where w.x.y.z is the IP of the sending machine. Let this run just long enough to reproduce the 'SSL Error' message that you see in the logs then use CTRL+C to stop it. Please PM me the output.pcap that this creates.