question about moving data

[benhank]

I have a project. YAAYY!
I have to set up the elastics elk stack [complete]
set up grafana [complete]
Have grafana aaccess the elasticsearch data stored locally and make pretty graphs[incomplete]
somehow get the data already collected in NLS to the elk stack [incomplete]

The purpose of the project is to highlight the differences between grafana and kibana, as well as showcase the features that may be included in NLS when you guys roll out the upgrades mentioned in the NLS road map
=======================================================
So far I have created a filter as detailed here:

Code: Select all

https://support.nagios.com/forum/viewtopic.php?f=38&t=42225

Which worked a little too well lol.
It pushed my NLS data to the new server, but i must have configured something wrong because I never found the data, and now I have a 24 hr gap in my NLS.
SO, what I am asking is how do I safely either configure NLS to send a COPY of its incoming logs to the ELK stack, or how do I move a copy of my NLS data to the elk server? Thanks guys!
BTW, working with elastics elk stack has reminded my why I was crying (with joy and relief) like chewbacca when I found out you guys had created NLS. NLS is SOOOO much easier to use.

[mcapra]

The lazy way to integrate grafana would be to configure the Elasticsearch settings on your Grafana machine to broadcast the HTTP API on it's LAN address. You can do this (on the Grafana machine) in /etc/elasticsearch/elasticsearch.yml by changing the http.host setting to this:

Code: Select all

http.host: localhost,GRAFANA_MACHINE_ADDRESS_HERE

And restarting the Elasticsearch service. That *should* allow you to point Nagios Log Server at the Grafana box with the following output configured on your Nagios Log Server environment:

Code: Select all

elasticsearch {
        host => 'GRAFANA_MACHINE_ADDRESS_HERE'
        protocol => 'http'
    }

2017_02_22_15_41_05_Instance_Configuration_Nagios_Log_Server.png

2017_02_22_15_41_30_Instance_Configuration_Nagios_Log_Server.png

That all worked just fine when I tried it.

[benhank]

MAN!That was FAST! whooOOOO!
I have grafana and the elk stack running on the same machine. NLS is the only thing on a separate server.
so this will allow me to send incoming data from NLS AND keep a copy of the data on my NLS server?
also

Code: Select all

2017_02_22_15_41_30_Instance_Configuration_Nagios_Log_Server.png (14.03 KiB) Not viewed yet

I think you were trying to show me another image. Just making sure I don't miss anything =D

[mcapra]

Should only be 2 images, probably just the forums being silly.

so this will allow me to send incoming data from NLS AND keep a copy of the data on my NLS server?

That's the idea It worked that way on my lab machine. NLS has a default output rule that you can't see in the GUI which handles transferring events to the local Elasticsearch database.

[benhank]

THANKS MAN Ill let you know how it went!

[mcapra]

Sure thing! Feel free to update the thread if something breaks or doesn't work right.

[benhank]

well can you show me how to get the data that NLS sent to the elk stack back?

[mcapra]

That is much trickier and will depend heavily on the Elasticsearch version the remote machine is using among other things.

You might be able to target the export application I made at your Grafana setup, but I have no idea what that machine is using for it's indices/mapping/etc. That would (hopefully) at least get the data exported as raw JSON which you could probably feed into Nagios Log Server using a file input with the JSON codec.

[benhank]

thanks man, but I accidentally deleted the data. doh!

[dwhitfield]

@benhank, I'm sorry to hear that!

I'm guessing that means we can lock this up?