Writing directly to elasticsearch db?

[polarbear1]

Greetings!

My devs are complaining feeding logs via nxlog is way too slow and they want to pipe logs directly into the elasticsearch db. I know this is probably way out of scope, but I want to get your opinion on a few questions on doing this.

1 - Any ideas on how to approach writing directly to the db?

2 - Would this break the db or any other NLS related functionality?

3 - If they are able to successfully write to the db, could I still use the NLS web UI for all the usual dashboards and alerts using that data?

Cheers.

[jolson]

While not necessarily common practice, inserting data directly into ES is supported. It's a question of whether or not you want to give the devs that kind of power. It can also be rather complex.

There are many Logstash inputs that could serve a similar purpose - you can send any raw data to a tcp/udp port, for example - it wouldn't be much slower than using the Elasticsearch API to insert data directly into the database. A full list of inputs can be found here: https://www.elastic.co/guide/en/logstas ... ugins.html

If your developers would prefer writing events directly into the Elasticsearch database, you'll need to have them read up on the proper way of doing so:
https://www.elastic.co/guide/en/elastic ... arted.html

Check out the "Data In, Data Out" section in particular - I imagine it will be very useful.

Now, you can either choose to expose port 9200 to the world - which is a very dangerous thing - or you can use our built-in API system (which I highly recommend). You can get your API key by clicking your username in Nagios Log Server.

Example API submission (using a bogus key):

Code: Select all

curl -XPUT 'http://192.168.x.x/nagioslogserver/index.php/api/backend/logstash-2015.12.10/syslog/133?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' -d '{
  "title": "My first blog entry",
  "text":  "Just trying this out...",
  "date":  "2014/01/01"
}'

[krobertson71]

Just backing up what Olson is saying here, Dev's should not have that level of authority is a production environment. I have never worked anywhere where the Dev's could interact directly with anything in a live environment.

The API option is something, as Developers, should be able to easily handle. If not, then they are most likely just wanting to do things "Their way". I have had these types of battles myself.

Tell them API is their gateway.

[jolson]

I agree with everything krobertson71 is stating - API access is really the way to go here. Let us know if you have any further questions!