Page 1 of 2

import logs from specific location

Posted: Thu Oct 15, 2015 6:48 am
by sangeetha
hi,

i want to import logs from specific path on my server. i have 4-5 logs files in this path. i want to load them and run some query. could you please help me with the steps to do so.
thanks.

Re: import logs from specific location

Posted: Thu Oct 15, 2015 9:45 am
by jolson
To import your logs from flag files, you can make use of the 'file' input:

You could use either of the following inputs, for example:

Code: Select all

file {
    path => /var/log/messages
}
-or-

Code: Select all

file {
    path => /var/log/httpd/*
}
You can define an input on the 'Administration -> Global Configuration' page of the Web GUI.

Re: import logs from specific location

Posted: Fri Oct 16, 2015 1:49 am
by sangeetha
hi,

i configured as you suggested. but still i am not able to see my logs loading on the dashboard. is there some document or video available to do this.

Re: import logs from specific location

Posted: Fri Oct 16, 2015 12:14 pm
by jolson
After you configured it, did you press the 'Apply Configuration' button?

Re: import logs from specific location

Posted: Tue Oct 20, 2015 4:07 am
by sangeetha
hi,
yes i did. but still no results.

Re: import logs from specific location

Posted: Tue Oct 20, 2015 4:37 pm
by tmcdonald
Did you use exactly what jolson posted or did you modify it to suit your environment?

What are you doing to search for those logs?

What sort of messages in the logs are you hoping to see?

Re: import logs from specific location

Posted: Wed Oct 21, 2015 1:38 am
by sangeetha
i followed these below steps.

click administration -> Global configuration -> add input -> added these below lines

file {
path => '/data/logs/'
}

here /data/logs/ is the path of my logs files available.

then click save -> verify -> Apply configuration

after this, i went into dashboard and try to query for "ERROR"

but i do not get any result

Re: import logs from specific location

Posted: Wed Oct 21, 2015 9:29 am
by jolson
Try this input:

Code: Select all

file {
path => '/data/logs/*'
}
Please "Apply Configuration" after you make the above definition. Do you see any files being generated? If not, try the following at the command line.

Switch to Nagios:

Code: Select all

su - nagios
Change to the data/logs directory:

Code: Select all

cd /data/logs
Read one of the files that exists there:

Code: Select all

tail -n100 fileexample
Add a few lines to a new file:

Code: Select all

echo "ERROR" >> newfile.log
Let me know how the above goes. When you add new lines to a file, logstash should pick up those changes almost immediately and display them to you in the Web GUI. We're also ensuring that the 'nagios' user has appropriate read and write access to the directory in question.

Jesse

Re: import logs from specific location

Posted: Wed Oct 28, 2015 3:10 am
by sangeetha
hi,

Thank you.
this worked. but it is not picking up files which are already available there. any other settings to be done.

Re: import logs from specific location

Posted: Wed Oct 28, 2015 10:36 am
by jolson
This means that Nagios Log Server has already parsed through the logs that are already there. What you can do is copy those files to force a re-read:

Code: Select all

cd /data/logs

Code: Select all

cp example-log-1 example-log-1-parse
The new log will be parsed appropriately. Any new lines in your existing logs will also be parsed appropriately, so you should be good to go!