Syslog Event Format

[Ivica Mustapic]

Guys,

My syslog event forwarded to NLS looks like this:

Nov 16 12:38:29 <local0.info> 127.0.0.2 11/16/2015:12:38:29 LINUX 0-PPE-0 : default APPFW APPFW_STARTURL 1342247 0 : 203.59.98.46 588799-PPE0 65Odty851TdGwG7Ti3lokT+lfm40000 app_fw_khg Disallow Illegal URL: https://www.kleen-new.com.au/natural-gas <not blocked>

This event in NLS looks like this:

<134> 16/11/2015:12:38:29 LINUX 0-PPE-0 : default APPFW APPFW_STARTURL 1342247 0 : 203.59.98.46 588799-PPE0 65Odty851TdGwG7Ti3lokT+lfm40000 app_fw_khg Disallow Illegal URL: https://www.kleen-new.com.au/natural-gas

My questions are:
* What is <134> at the beginning of the line? And how to remove it?
* What happened to the "<not blocked>" text from the syslog event, it is no longer in the NLS event?
* How do I get the host name from the event into the NLS's hostname field?

Thanks for your help.

Cheers,
Ivica

[jolson]

When your log enters Nagios Log Server, it's entering through the Logstash program. Your example log is being manipulated by the 'syslog' input. To learn more about what the syslog input is doing, I highly recommend checking out this blog post:
http://kartar.net/2014/09/when-logstash ... -go-wrong/

* How do I get the host name from the event into the NLS's hostname field?

What I recommend is setting up your own inputs/filters and processing the log as you see fit. I've done many writeups on how to do this, here are a few of them:
http://support.nagios.com/forum/viewtop ... 37&t=32221
http://support.nagios.com/forum/viewtop ... 68#p134768
http://support.nagios.com/forum/viewtop ... 28#p137728

[Ivica Mustapic]

Thanks for that.

As suggested, I have created a Logstash filter and it looks to work fine.

It looks like Netscaler is not forwarding the same data to NLS that is logging locally but I was able to get what I needed with this filter:

'^<\d+>\s+%{DATE_EU}:%{TIME}\s+%{HOSTNAME:Hostname}\s+%{DATA:Source}\s+:\s+%{DATA:facility_label}\s+%{DATA:Feature}\s+%{DATA:FeatureMessage}\s+%{DATA}:\s+%{GREEDYDATA}$'

I am still mystified about the "<not blocked>" text at the end of forwarded event as it never makes it to the NLS. I can see that text is part of the forwarded event (via tcpdump) but I don't know where it goes. It looks as if never makes it to the filter stage. Everything else is working fine.

Any further suggestions about troubleshooting?

Cheers,
Ivica

[jolson]

As for <not blocked>, I bet it's being hidden by your web browser (interpreted as HTML).

Try viewing the raw log, this will show you whether or not that string actually exists:

2015-11-17-091631_1046x631_scrot.png

[Ivica Mustapic]

Good call jolson. It was hidden by my browser.

Thanks for your help.

Cheers,
Ivica

[tmcdonald]

Are we all set to close this up?

[Ivica Mustapic]

Yes please.

Cheers,
Ivica

[jolson]

Locking it up! Thanks!