Page 1 of 2
File Input on local log server
Posted: Wed Jul 22, 2015 4:42 am
by CFT6Server
We have some logs that are stored on an NFS mount. I have mounted the NFS volume on 1 node of our 3 node cluster. I am trying to use the "file" input to pick up the logs as if they are local logs. Nothing seems to be showing.
I am an instance specific configuration on the node:
Code: Select all
input {
file {
path => "/nfs/shared/servers/logs/*"
type => "apache"
}
}
If I got to that node and look at the inputs, it does not show anything in the #Local Inputs section, even though I see it in GUI.
This should just pick up the logs and dump it into elasticsearch? or would we need additional configurations. Right now only this node has the volume mounted. If I mount the volume on same path on all 3 nodes, would I be able to make this into a global configuration? Logically thinking I am guessing no, since the nodes can't tell each other what has been picked up?
Re: File Input on local log server
Posted: Wed Jul 22, 2015 9:34 am
by jolson
What you have posted should work with no additional filters required. Let's do a couple of tests.
Check the file permissions:
The 'nagios' user will need read access to the above files.
Check logstash configuration:
Code: Select all
cat /usr/local/nagioslogserver/logstash/etc/conf.d/*
Verify that the logstash configuration applied properly to the server in question.
This should just pick up the logs and dump it into elasticsearch?
Corrrect - there shouldn't be problems with the input you've posted.
Re: File Input on local log server
Posted: Wed Jul 22, 2015 3:59 pm
by CFT6Server
We have access to these files, here's partial output.
Code: Select all
-rw-r--r-- 1 30002 300 10121 Jun 2 23:55 errors.201506022355
-rw-r--r-- 1 30002 300 1029031 Jun 3 23:55 errors.201506032355
-rw-r--r-- 1 30002 300 7405 Jun 4 23:55 errors.201506042355
-rw-r--r-- 1 30002 300 5067 Jun 5 23:55 errors.201506052355
-rw-r--r-- 1 30002 300 5032 Jun 6 23:55 errors.201506062355
-rw-r--r-- 1 30002 300 6485 Jun 7 23:55 errors.201506072355
-rw-r--r-- 1 30002 300 6955 Jun 8 23:55 errors.201506082355
-rw-r--r-- 1 30002 300 5486 Jun 9 23:55 errors.201506092355
-rw-r--r-- 1 30002 300 4634 Jun 10 23:55 errors.201506102355
-rw-r--r-- 1 30002 300 18925 Jun 11 23:55 errors.201506112355
Reviewing the configurations on the node, it doesn't seem like it does not show the instance configuration. I created the input under "per instance" and saved/applied configurations, but no luck.
I also noticed that this node is no longer grabbing changes in the global configurations. Is this because I have instance configurations in place? I've made changes in the global config that are not showing in this node configuration.
Re: File Input on local log server
Posted: Wed Jul 22, 2015 4:17 pm
by jolson
Interesting, I recommend running a 'Verify' under your 'per instance' configuration menu. Does it come back clean?
2015-07-22 16_16_37-Instance Configuration • Nagios Log Server - Firefox Developer Edition.png
Re: File Input on local log server
Posted: Wed Jul 22, 2015 4:32 pm
by CFT6Server
It does, but the local inputs do not show up on the instance configuration.
To get the servers sync'd. I rebooted, removed instance configuration and re-applied global configurations. I then went back and put in the single instance input configuration, and it and didn't apply any local inputs.
instance.JPG
verification.JPG
local inputs is blank when viewing input file.
local inputs.JPG
Re: File Input on local log server
Posted: Wed Jul 22, 2015 4:37 pm
by jolson
It's odd to me that your verify worked properly. Give the following a shot:
Code: Select all
file {
path => "/nfs/shared/servers/logs/*"
type => "apache"
}
Nagios Log Server will input the starting and ending brackets for your configs automatically.
Re: File Input on local log server
Posted: Wed Jul 22, 2015 5:00 pm
by CFT6Server
I have done that and again verified fine, but again it does not show up when I click on view. Nothing under "#Local Inputs".
Re: File Input on local log server
Posted: Wed Jul 22, 2015 5:06 pm
by jolson
Interesting - what version of Nagios Log Server are you running? I recall a similar bug in R1.3, but it's been fixed for some time now. If you're on a later version, I'd be happy to set up a remote session with you to get this working. Let me know!
Re: File Input on local log server
Posted: Wed Jul 22, 2015 5:25 pm
by CFT6Server
We are on 2015R1.4 at the moment.
Re: File Input on local log server
Posted: Thu Jul 23, 2015 10:02 am
by jolson
Would you be willing to attempt upgrading to R2.1?