Page 1 of 1

Filters question

Posted: Thu Jun 20, 2019 10:21 am
by Bitflogger
Hello, I am running the latest NLS, on a 64-bit VM, CentOS

Under Global Config, I see Inputs and Filters.

I would like to load old logs from Linux and Windows servers.

I looks like I need filters that correspond to some of the inputs.

Where can I get a standard filter for Windows event logs and Linux syslogs?

Earl

Re: Filters question

Posted: Thu Jun 20, 2019 2:58 pm
by cdienger
What exactly are you trying to do with filters? Filters are not necessary to import data and the default syslog input can parse syslog data while the Windows Event Log input accepts data in the json format.

Here are the inputs in case you need them:

Code: Select all

syslog {
    type => 'syslog'
    port => 5544
}

Code: Select all

 tcp {
    type => 'eventlog'
    port => 3515
    codec => json
}

Re: Filters question

Posted: Thu Jun 20, 2019 4:59 pm
by Bitflogger
Hello,

I'm trying to do something like this: cat ./old-windows-logs.txt | nc 127.0.0.1 3515

It does not seem to take it in.

Earl

Re: Filters question

Posted: Fri Jun 21, 2019 4:29 pm
by Bitflogger
Hello,

I found the log entries from using nc on the local host to send a file of logs entries from a different host. But they ended up in NLS identified with the localhost. And I had to open an older shard to see them in a query.

So if I have an old log server with 900 servers, and 1825 days of logs (files are per day), I have to move the files to the source server (which may not exist) and then send the logs to the correct port on NLS?

Earl

Re: Filters question

Posted: Mon Jun 24, 2019 10:43 am
by cdienger
That would be one option. Another would be to make sure the log entries have an entry for the host field that points to the original client's IP or hostname.

Re: Filters question

Posted: Mon Jun 24, 2019 12:48 pm
by Bitflogger
Hello,

I'm still having difficulty.

If I load logs from 2019-05-13, but my oldest registered shard is 2019-05-16, where does the information go?

I do not see a 2019-05-13 shard created.

Earl

Re: Filters question

Posted: Mon Jun 24, 2019 4:44 pm
by cdienger
If the date is parsed out properly, then it should inserted into the corresponding index. Can you share a sample of the 2019-05-13 and the logstash configuration so I can take a closer look?

Re: Filters question

Posted: Thu Jun 27, 2019 9:43 am
by Bitflogger
Hello,

I think I understand this now.

Please lock it.

Earl

Re: Filters question

Posted: Thu Jun 27, 2019 4:20 pm
by benjaminsmith
Please lock it.
Earl
We'll close this out. If you have any questions in the future, please feel free to open a new one.