A question about the output filter

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

A question about the output filter

Post by benhank »

Hey guys.
I am running an experiment with Grafana. Im trying to show that NLS can do the job of grafana and a whole lot more. So I am setting up a "pepsi challenge" of sorts with NLS, the elastic's ELK stack and grafana.
I want to have my current NLS server to export say 50gb of data in json format to the remote server thats running the "other stuff, while keeping a copy on my NLS".
I dont know how to use the output filter to make it happen, any suggestions?
Last edited by dwhitfield on Tue Feb 21, 2017 3:49 pm, edited 1 time in total.
Reason: marking with green check mark
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
rkennedy
Posts: 6579
Joined: Mon Oct 05, 2015 11:45 am

Re: A question about the output filter

Post by rkennedy »

What is on the other end, another ELK stack?

If so, you should be able to use the Logstash tcp output - https://www.elastic.co/guide/en/logstas ... s-tcp.html -

Code: Select all

tcp {
host => "<elkip>"
port => "<portacceptingjson>"
codec => "json"
}
Should do the trick, this is providing you have a TCP input accepting JSON on the ELK side of things.
Former Nagios Employee
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

Re: A question about the output filter

Post by benhank »

so on the receiving elk stack should have

Code: Select all

tcp {
    port => 9200
type= json
}
correct?
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: A question about the output filter

Post by mcapra »

The receiving ELK stack's Logstash input should be leveraging the JSON codec, unless you have a filter rule that matches if [type] == 'json' and then parses the message as JSON. Here's the stock JSON input we use for NLS, it should work just as well on an OSS ELK stack:

Code: Select all

tcp {
    type => 'import_json'
    tags => 'import_json'
    port => 2057
    codec => json
}
Former Nagios employee
https://www.mcapra.com/
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

Re: A question about the output filter

Post by benhank »

thanks man! you can lock it up
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
Locked