Nagios Support Forum

Are you able to run a tcpdump on the Nagios Log Server machine to verify traffic on port 2056 is being received from the host with the catalina logs? It might look something like this:
If there's absolutely nothing in the Logstash logs, and you can't see the events via the GUI, I question whether or not the traffic is making it to the Nagios Log Server machine. You might also check the Elasticsearch logs
(/var/log/elasticsearch/*.log) to make sure the data isn't having issues being inserted into the database.

Great point on connections.

I can see the host connected on port 5544 but not on 2056

And on the host it only shows connected to 5544.

So for some reason rsyslog isn't wanting to use the config file.

Just to make sure... here is my current conf from /etc/rsyslog.d/

/etc/rsyslog.d/90-nagioslogserver_opt_tomcat_logs_catalina.out.conf

$ModLoad imfile
$InputFilePollInterval 10
$PrivDropToGroup adm
$WorkDirectory /var/spool/rsyslog

# Input for CatalinaOut
$InputFileName /opt/tomcat/logs/catalina.out
$InputFileTag CatalinaOut:
$InputFileStateFile nls-state-opt_tomcat_logs_catalina.out # Must be unique for each file being polled
# Uncomment the folowing line to override the default severity for messages
# from this file.
#$InputFileSeverity info
$InputFilePersistStateInterval 20000
$InputRunFileMonitor

# Forward to Nagios Log Server and then discard, otherwise these messages
# will end up in the syslog file (/var/log/messages) unless there are other
# overriding rules.
if $programname == 'CatalinaOut' then @@logs.isonasnet.com:2056
if $programname == 'CatalinaOut' then ~


And the last line of the rsyslog.conf does have:

$IncludeConfig /etc/rsyslog.d/*.conf

No firewall on the host blocking outbound and the NLS has the port open.

Can you also share the full contents of your main rsyslog file? As well as the version being used:

For the benefit of others....

The issue is resolved.

When you install tomcat and configure it according to the Apache docs part of that process is to give ownership of the tomcat directory to the tomcat user..... well that means that the rsyslog service can't get in there. chown the tomcat/logs dir recursively so it's owned by user syslog group tomcat and the entries started to flood right in.

knew it was going to be something stupid

Thanks for the update! I'll go ahead and lock the thread.