Nagios Support Forum

Posted: **Tue Mar 10, 2015 1:58 pm**

What is the logserver supported method to send data of a certain type to another index?

We tried adding an if statement to the output section, but we found when we did this that data was being written to the default logstash-YYYY.MM.dd index and the syslog-YYYY.MM.dd index we created.

This looks to be happening because of the way logserver puts the default output entry into the configuration file.

Any ideas on how to properly do this?

Code: Select all

[root@hostname conf.d]# pwd
/usr/local/nagioslogserver/logstash/etc/conf.d
[root@hostname conf.d]# cat 999_outputs.conf 
# 
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Tue, 10 Mar 2015 13:43:44 -0500
#

#
# Required output for Nagios Log Server
#

output {
    elasticsearch {
        cluster => 'xxxxxxx-xxxxxx-xxxxxx-xxxx-xxxxxxxx'
        host => 'localhost'
        index_type => '%{type}'
        node_name =>  'xxxxxxx-xxxxxx-xxxxxx-xxxx-xxxxxxxx'
        protocol => 'transport'
        workers => 4
    }
}

#
# Global outputs
#

output {
    if [type] == 'syslog' {
       elasticsearch {
            index => 'syslog-%{+YYYY.MM.dd}'
            cluster =>  'xxxxxxx-xxxxxx-xxxxxx-xxxx-xxxxxxxx'
            host => 'localhost'
            index_type => '%{type}'
            node_name =>  'xxxxxxx-xxxxxx-xxxxxx-xxxx-xxxxxxxx'
            protocol => 'transport'
            workers => 4
        }
    }
}

#
# Local outputs
#


[root@hostname conf.d]#

Posted: **Tue Mar 10, 2015 2:53 pm**

Just to be clear - you are trying to accept logs into Logstash from some remote sender, and after accepting those logs you want to output them to a particular Elasticsearch index? Is that correct? Or are you trying to output them to a different external device?

Posted: **Tue Mar 10, 2015 2:56 pm**

We want to output them to a particular Elasticsearch index without duplicating them in the default if possible.

Posted: **Tue Mar 10, 2015 5:10 pm**

After re-creating your setup, I am seeing the same results on my cluster. I haven't found a way to delete the replicate data left behind by the first output. The only way that I can imagine doing it is by adding an 'if' statement to the original output configuration (if type = syslog action = delete) to see if that resolves it. Since the file is generated dynamically, this will not work. I have made a note to continue testing this tomorrow morning.

Posted: **Wed Mar 11, 2015 10:30 am**

Right now, it does not look like multiple outputs are possible without data duplication.

Posted: **Wed Mar 11, 2015 11:55 am**

Hello,

I have confirmed with development that it may not ever be possible for data to not hit the default index. Unfortunately you'll either need to concatenate your data into the default index, or duplicate the data.

Best,

Jesse

Nagios Support Forum

Placing data into new indexes by type

Placing data into new indexes by type

Re: Placing data into new indexes by type

Re: Placing data into new indexes by type

Re: Placing data into new indexes by type

Re: Placing data into new indexes by type

Re: Placing data into new indexes by type