Filters not written to configuration

[gabe]

Hello, all:

I'm evaluating Log Server and am trying to add a Redis input filter. Via the Administration section of the GUI, I've added an input filter that looks like this:

Code: Select all

redis {
    host => "127.0.0.1"
    data_type => "list"
    key => "logstash"
    codec => json
}

After saving the filter and applying the configuration, the filter didn't seem to be working. Upon further investigation, I notice that no new filter has been added to /usr/local/nagioslogserver/logstash/etc/conf.d/000_inputs.conf. Strangely, when I use the GUI to view all of the filters, my Redis filter is shown:

Code: Select all

input {
    syslog {
        type => 'syslog'
        port => 5544
    }
    tcp {
        type => 'eventlog'
        port => 3515
        codec => json {
            charset => 'CP1252'
        }
    }
    tcp {
        type => 'import_raw'
        tags => 'import_raw'
        port => 2056
    }
    tcp {
        type => 'import_json'
        tags => 'import_json'
        port => 2057
        codec => json
    }
    redis {
        host => "127.0.0.1"
        data_type => "list"
        key => "logstash"
        codec => json
    }
}

I'm at a loss as to why the new filter is not being written to the config file. Anyone else running into issues with new custom filters being properly written to the logstash config?

Thanks,

Gabe

[eloyd]

Not to state the obvious, but have you "verified" your config before "saving" and "applying" it?

[gabe]

Yes, I did verify the config and it verified fine. I also confirmed that if I simply modify 000_inputs.conf with the additions I want and restart logstash, logstash successfully uses my redis input, my redis queue is drained as expected, and the desired entries appear in Log Server when I search for them.

Gabe

[scottwilkerson]

Did you add it as an input or a filter?

Inputs are listed in
/usr/local/nagioslogserver/logstash/etc/conf.d/000_inputs.conf

Filters are listed in
/usr/local/nagioslogserver/logstash/etc/conf.d/500_filters.conf

Also, how many instances are in your cluster?

What are the permissions on the file

Code: Select all

ls -l /usr/local/nagioslogserver/logstash/etc/conf.d/000_inputs.conf

[gabe]

Scott:

Thanks for your reply. I added it as an input as shown in my original post. My cluster is two instances; neither of them seem to get the new input. Perms and ownership shown here:

# ls -l /usr/local/nagioslogserver/logstash/etc/conf.d/
total 12
-rw-r--r--. 1 apache apache 636 Oct 30 15:51 000_inputs.conf
-rw-r--r--. 1 apache apache 957 Oct 30 15:51 500_filters.conf
-rw-r--r--. 1 apache apache 493 Oct 30 15:51 999_outputs.conf

The date shown is that when I installed Log Server, so nothing has changed with the logstash config since.

Thanks,

Gabe

[tmcdonald]

I've got the following:

Code: Select all

-rwxrwxr-x 1 nagios nagios 722 Nov  5 16:51 000_inputs.conf
-rwxrwxr-x 1 nagios nagios 978 Nov  5 16:51 500_filters.conf
-rwxrwxr-x 1 nagios nagios 493 Nov  5 16:51 999_outputs.conf

Do you have any sort of umask in place?

Code: Select all

umask

[scottwilkerson]

apache:apache definitely isn't correct...

lets start by running

chown nagios:nagios -R /usr/local/nagioslogserver

then try to apply configuration again

[nozlaf]

also have this issue, I've changed the permissions on the folders, verified they are correct and the issue persists

-rw-rw-r-- 1 nagios nagios 787 Nov 10 20:52 000_inputs.conf
-rw-rw-r-- 1 nagios nagios 220 Nov 10 20:52 500_filters.conf
-rw-rw-r-- 1 nagios nagios 493 Nov 10 20:52 999_outputs.conf

When I apply the config I can confirm that the filters file gets overwritten with the same empty contents

-rw-rw-r-- 1 nagios nagios 220 Nov 10 21:09 500_filters.conf

Code: Select all

#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Mon, 10 Nov 2014 21:09:08 -0500
#

#
# Global filters
#



#
# Local filters
#

[nozlaf]

So i've done some investigation into the issue as I didn't want to wait until tomorrow for a response from anyone

seems like "validate" doesnt really work at all and is pretty much useless / misleading

sometimes it brings up errors but generally it validates and says everything is fine, however if you try and apply the config you will get thrown to the configuration snapshots page with a message about the delete request being processed this indicates that there is a config issue.

[lmiltchev]

Let's clarify - when you try to add a new filter under the "per instance" page, the filter is not added (nothing appears on the "per instance" page) but the "global" configs get overwritten, correct?