File Input on local log server

[CFT6Server]

We have some logs that are stored on an NFS mount. I have mounted the NFS volume on 1 node of our 3 node cluster. I am trying to use the "file" input to pick up the logs as if they are local logs. Nothing seems to be showing.

I am an instance specific configuration on the node:

Code: Select all

input {
  file {
    path => "/nfs/shared/servers/logs/*"
    type => "apache"
  }
}

If I got to that node and look at the inputs, it does not show anything in the #Local Inputs section, even though I see it in GUI.

This should just pick up the logs and dump it into elasticsearch? or would we need additional configurations. Right now only this node has the volume mounted. If I mount the volume on same path on all 3 nodes, would I be able to make this into a global configuration? Logically thinking I am guessing no, since the nodes can't tell each other what has been picked up?

[jolson]

What you have posted should work with no additional filters required. Let's do a couple of tests.

Check the file permissions:

Code: Select all

ls -l /nfs/shared/servers/log/*

The 'nagios' user will need read access to the above files.

Check logstash configuration:

Code: Select all

cat /usr/local/nagioslogserver/logstash/etc/conf.d/*

Verify that the logstash configuration applied properly to the server in question.

This should just pick up the logs and dump it into elasticsearch?

Corrrect - there shouldn't be problems with the input you've posted.

[CFT6Server]

We have access to these files, here's partial output.

Code: Select all

-rw-r--r-- 1 30002 300     10121 Jun  2 23:55 errors.201506022355
-rw-r--r-- 1 30002 300   1029031 Jun  3 23:55 errors.201506032355
-rw-r--r-- 1 30002 300      7405 Jun  4 23:55 errors.201506042355
-rw-r--r-- 1 30002 300      5067 Jun  5 23:55 errors.201506052355
-rw-r--r-- 1 30002 300      5032 Jun  6 23:55 errors.201506062355
-rw-r--r-- 1 30002 300      6485 Jun  7 23:55 errors.201506072355
-rw-r--r-- 1 30002 300      6955 Jun  8 23:55 errors.201506082355
-rw-r--r-- 1 30002 300      5486 Jun  9 23:55 errors.201506092355
-rw-r--r-- 1 30002 300      4634 Jun 10 23:55 errors.201506102355
-rw-r--r-- 1 30002 300     18925 Jun 11 23:55 errors.201506112355

Reviewing the configurations on the node, it doesn't seem like it does not show the instance configuration. I created the input under "per instance" and saved/applied configurations, but no luck.

I also noticed that this node is no longer grabbing changes in the global configurations. Is this because I have instance configurations in place? I've made changes in the global config that are not showing in this node configuration.

[jolson]

Interesting, I recommend running a 'Verify' under your 'per instance' configuration menu. Does it come back clean?

2015-07-22 16_16_37-Instance Configuration • Nagios Log Server - Firefox Developer Edition.png

[CFT6Server]

It does, but the local inputs do not show up on the instance configuration.

To get the servers sync'd. I rebooted, removed instance configuration and re-applied global configurations. I then went back and put in the single instance input configuration, and it and didn't apply any local inputs.

instance.JPG

verification.JPG

local inputs is blank when viewing input file.

local inputs.JPG

[jolson]

It's odd to me that your verify worked properly. Give the following a shot:

Code: Select all

      file {
        path => "/nfs/shared/servers/logs/*"
        type => "apache"
      }

Nagios Log Server will input the starting and ending brackets for your configs automatically.

[CFT6Server]

I have done that and again verified fine, but again it does not show up when I click on view. Nothing under "#Local Inputs".

[jolson]

Interesting - what version of Nagios Log Server are you running? I recall a similar bug in R1.3, but it's been fixed for some time now. If you're on a later version, I'd be happy to set up a remote session with you to get this working. Let me know!

[CFT6Server]

We are on 2015R1.4 at the moment.

[jolson]

Would you be willing to attempt upgrading to R2.1?