Nagios Support Forum

Posted: **Thu May 04, 2017 9:34 am**

Hello,

We are adding some application log files to NagiosLog that are of the following format:

INFO  [2017-05-04 14:30:25,044][ListenerContainer-10][ID:NNNNN-NNNNN-NNNNNN-NNNNN] - Forward.COMPONENT- Message sent to [QUEUE_NAME] Variant [RAW_TEXT] Message body [<PreScanResponse>
    <PreScanID>111111</PreScanID>
    <CreatedDateTime>2017-05-04T14:30:25</CreatedDateTime>
    <StatusCode>YYY</StatusCode>
</PreScanResponse>] Size [178] Send Time [12]

Each of these lines show up as a separate entry right now, but I would like to see just one entry, INFO etc etc that contains all 5 lines in one, so when I search for the prescan ID, I will see all 5 lines and see all the other related lines at the same time

Is there a way to do this? We would want to do the same for Java stack traces as well, as right now it pollutes our logs with excessive lines.

Thank you,
Alex

Posted: **Thu May 04, 2017 9:54 am**

There's quite a few different options each with their own benefits depending on the specific logs. Logstash has a native multiline codec which I typically recommend for environments using rsyslog. nxlog has it's own multiline plugin which works better (in my opinion) than the one Logstash has, but running nxlog on a Linux system is a pain in the butt.

Here's our docs for how to do it in nxlog:
https://support.nagios.com/kb/article.php?id=546

We don't have a proper document for rsyslog (yet), but I did a write-up recently for another customer on this topic. I'll attach it below (compressed zip with a pdf).

Posted: **Thu May 04, 2017 10:05 am**

Thank you, I will try to follow the same for logstash and let you know if it works or not.

Alex

Posted: **Thu May 04, 2017 11:04 am**

Based on what I am reading in the PDF, I tried to create the following input filter and modify the rsyslog config, but I am no longer seeing any events at all coming into NagiosLog at all. Can you please advise what I'm doing incorrectly?

Code: Select all

tcp {
    codec => multiline
    { pattern => 'INFO [0-9]{4}-[0-9]{2}-[0-9]{2}'
    negate => true
    what => previous
}
    type => 'log_forwarder'
    port => 5545
}

Code: Select all

 cat 90-nagioslogserver_log_forwarder1.conf
$ModLoad imfile
$InputFilePollInterval 10
$PrivDropToGroup adm
$WorkDirectory /var/lib/rsyslog

# Input for log_forwarder1
$InputFileName /mnt/host005p/logs/Forwarder.log
$InputFileTag log_forwarder1:
$InputFileStateFile nls-state-mnt_host005p_logs_Forwarder.log # Must be unique for each file being polled
# Uncomment the folowing line to override the default severity for messages
# from this file.
#$InputFileSeverity info
$InputFilePersistStateInterval 20000
$InputRunFileMonitor
$template clean,"%rawmsg%"

# Forward to Nagios Log Server and then discard, otherwise these messages
# will end up in the syslog file (/var/log/messages) unless there are other
# overriding rules.
if $programname == 'log_forwarder1' then @@10.10.10.10:5545;clean
if $programname == 'log_forwarder1' then ~

Posted: **Thu May 04, 2017 1:59 pm**

If iptables or firewalld is running on this machine, did you create a rule to allow traffic on port 5545?

Is there anything in the Logstash log indicating parse failures? Was the configuration applied after the rule was created?

Posted: **Thu May 25, 2017 9:36 am**

Did @mcapra's post help you? Please let us know if you have any additional questions.

Nagios Support Forum

Multi-line log entries to show up as one line in NagiosLog?

Multi-line log entries to show up as one line in NagiosLog?

Re: Multi-line log entries to show up as one line in NagiosL

Re: Multi-line log entries to show up as one line in NagiosL

Re: Multi-line log entries to show up as one line in NagiosL

Re: Multi-line log entries to show up as one line in NagiosL

Re: Multi-line log entries to show up as one line in NagiosL