Windows Hostname and Linux Logsource

[krobertson71]

I was curious to know if there was anyway to display these in one panel?

With a Windows event, the server name is in the "Hostname" field, while in Linux, the name is in the Logsource field. I wanted to create a panel to show my top event producers showing there hostname. I can use ipaddress but hostnames look better.

Is there a way to change this in NLS for syslog events to change the field name from logsource to Hostname? Or, is there a way they can both be in the same panel where the panel using the term type/table with the top 10 hosts by their name versus their IP?

Thanks for any feedback on this.

[Box293]

I believe what is needed here is a custom input filter. it allows you to take the incoming data and place it in the appropriate fields.

Have a read of this document and let us know if this is what you are after:

http://assets.nagios.com/downloads/nagi ... ilters.pdf

NOTE: input filters will not apply to logs already received.

[krobertson71]

It's a great example if you wanted to change the value of a field.

I need to do some research on maybe added a field to the syslog filter to include Hostname and populate that. The more I think about it, I don't want to change the fields that syslog brings in. Sounds like it might just be better to add a field that has the same information as 'LogSource'.

How does that sound to you as a resolution to my issue?

[sreinhardt]

I actually fully agree with this idea (for once ). Having an alternative name field would be very benefitial as you could match other devices or apps logs to the original name, but still display your pretty hostname field instead. It also adds very minor storage and processor overhead, opposed to repeatedly collecting, filtering, and changing names each time before displaying. Great idea!

As for changing names of existing entries in nls, I honestly can't say. I have to imagine it's possible, just haven't had an opportunity to do so. Probably the easiest route is essentially what you described, adding a second field and using that instead, especially considering attempting to correlate logs with mismatched names would be a nightmare.

[krobertson71]

Yeah I just need to find a way to add a field to the type "syslog" called Hostname and make it equal the same value as 'Logsource'.

Brain explodes here.

[slansing]

See, the way I do something similar to this is just create a custom query, for instance:

Code: Select all

logsource:localhost

Then re-name the query to localhost, or whatever the address is, or I'll do:

Code: Select all

logsource:localhost and logsource:test.testingstuff12.com

Or some chain combination like that, then I can use quite a few of the panels and filter them down to only selected queries, and choose those ones. Kind of a different way of looking at it, but that's how I've gotten around this in the short term.

[krobertson71]

Sorry I dont follow. I get what you are trying to accomlish via a query standpoint but not seeing how it allows me to build the panel I am after.

I am trying to achieve a panel like this

Top Hosts

ServerA.int 2000 (Linux Host comes from Logsource field for names of host)
ServerB.int 987 (Windows Host comes from Hostname field for names of host)

I am not sure how your query enables me to make this panel. Know I am using the panel type term. I have tried others and using the query method you mentioned but not getting results.

The goal is to have a panel that shows the Hostname (not the IP Address) where it's linux or Windows, and sorted by count of events as a quick "Top 10 Event producers)"

[krobertson71]

I may have solved this, but not the precise way I wanted to.

I found the "linux-syslog" pattern in /usr/local/nagioslogserver/logstash/patterns. I then edited "linux-syslog":

Selection_041.png

Now for all incoming syslog messages the name of the server is put in the field "Hostname" vs "logsource".

Selection_042.png

So now I am able to create this panel:

Selection_040.png

I wanted to show this as I wanted to make sure by editing the "linux-syslog" value of %SYSLOGHOST was not going to cause any issues. I am thinking that it will not but please correct me if I am wrong.
Also, I would rather create a filter, using grok maybe, to just add a field that equals the value of logsource. If that is not possible then this will work, if it is safe that is.

Warning to any who read this. This change is not retroactive. It will only apply to events that come in after this change.

[sreinhardt]

That is perfectly fine, and actually probably what I would have suggested doing, either that or grok transforms on the incoming data. Personally I would much rather have the remote device start by sending as much correct information as possible. It keeps the processing off NLS, makes it so that edge cases can be handled remotely as much as possible, and your inputs and filters in NLS can be as agnostic as possible, which is a HUGE concern for me. I really don't want 50 different ports open just for different input data if it's all syslog or json. I'd much prefer to parse it in and filter based on tag or something of that nature. Otherwise both of your ideas will work fine, grok additions would be perfectly acceptable, but more work on NLS when it needs to do it for every event that enters, and depending on how it's implemented, text replacement can be rather cpu intensive.

Was going to make the exact same comment about it not being retro active. Did the same thing to my IIS logs this weekend and need to input the old ones again to correct this. Seems to be the only catch I can think of. Nice job!

[krobertson71]

That is perfectly fine, and actually probably what I would have suggested doing, either that or grok transforms on the incoming data. Personally I would much rather have the remote device start by sending as much correct information as possible. It keeps the processing off NLS, makes it so that edge cases can be handled remotely as much as possible, and your inputs and filters in NLS can be as agnostic as possible, which is a HUGE concern for me. I really don't want 50 different ports open just for different input data if it's all syslog or json. I'd much prefer to parse it in and filter based on tag or something of that nature. Otherwise both of your ideas will work fine, grok additions would be perfectly acceptable, but more work on NLS when it needs to do it for every event that enters, and depending on how it's implemented, text replacement can be rather cpu intensive.

Was going to make the exact same comment about it not being retro active. Did the same thing to my IIS logs this weekend and need to input the old ones again to correct this. Seems to be the only catch I can think of. Nice job!

How would you modify the sourece to send SYSLOGHOST as HOSTNAME. Maybe a stupid question. Or it's a question you yourself are pondering?