Page 1 of 1

Copying NLS Logstash and Kibana configurations?

Posted: Tue Oct 01, 2019 2:03 pm
by jaimie.livingston
Is there a procedure for copying NLS Logstash (inputs, filters) and Kibana (dashboards, queries) between clusters?
SCP, Rsync?

Use Case:
I will have several NLS cluster deployments across different environments.
One cluster will be for developing new inputs/input filters and dashboards, all others are for production.

Is there a way to scp/rsync configurations from the "development" cluster to all "production" clusters.


Thanks,

Jaimie Livingston

Re: Copying NLS Logstash and Kibana configurations?

Posted: Wed Oct 02, 2019 3:55 pm
by cdienger
The logstash and kibana configs are all part of backups created by the backups job under Admin > System > Command Subsystem. The backups are stored in /store/backups/nagioslogserver/ and can be imported with:

Code: Select all

cd /usr/local/nagioslogserver/scripts/
./restore_backup.sh /store/backups/nagioslogserver/nagioslogserver.<timestamp>.tar.gz
https://assets.nagios.com/downloads/nag ... Server.pdf

Re: Copying NLS Logstash and Kibana configurations?

Posted: Thu Oct 03, 2019 4:18 pm
by jaimie.livingston
If I understand the document correctly, this will not work for me...
According to the docs:
System Backups
System backups contain the configuration settings, dashboards, users, internal logs, alerts. Also included are the Inputs, Filters and Outputs for Logstash
Since the systems I need to sync are in different environments, they will possibly have different configurations and users, and should not have any modifications to logs for access or audits.

All I want to do is copy the Kibana Global Dashboards and Logstash Inputs, Filters and Outputs from one cluster to another, without modifying any other configurations, user accounts, logs, or indexes.

Config Snapshots capture the Logstash Inputs, Filters, and Outputs, but, as best I can tell from the docs and from examining a snapshot database.json file, don't capture the Kibana Dashboards.

So, if I wanted to copy the Logstash Inputs, Filters, and Outputs AND the Kibana Global Dashboards from one cluster to another cluster, is there a method I can use? I can write my own script, if I know which files to copy...

Thanks,

Jaimie Livingston

Re: Copying NLS Logstash and Kibana configurations?

Posted: Thu Oct 03, 2019 5:02 pm
by scottwilkerson
jaimie.livingston wrote:Config Snapshots capture the Logstash Inputs, Filters, and Outputs, but, as best I can tell from the docs and from examining a snapshot database.json file, don't capture the Kibana Dashboards.
this is correct. the problem is that the kibana dashboards are associated with users...

You cna back them up with something like

Code: Select all

curl -XPOST "http://localhost:9200/kibana-int/_export?path=/store/backups/nagioslogserver/TESTBACKUP/kibana-int.tar.gz"

then transfer /store/backups/nagioslogserver/TESTBACKUP/kibana-int.tar.gz to the new server restore with something like

Code: Select all

curl -XPOST "http://localhost:9200/kibana-int/_import?path=/store/backups/nagioslogserver/TESTBACKUP/kibana-int.tar.gz"

BUT, this will have all the user dashboards too which could cause unexpected behavior

Re: Copying NLS Logstash and Kibana configurations?

Posted: Fri Oct 04, 2019 8:00 am
by jaimie.livingston
Excellent information. Thanks.

For purposes of using the Kibana API calls, what is the effective ElasticSearch/Kibana version?
A curl to localhost returned version 1.7.6, but that seems a bit out of date for year 2019...

Jaimie Livingston

Re: Copying NLS Logstash and Kibana configurations?

Posted: Fri Oct 04, 2019 8:18 am
by scottwilkerson
jaimie.livingston wrote:Excellent information. Thanks.

For purposes of using the Kibana API calls, what is the effective ElasticSearch/Kibana version?
A curl to localhost returned version 1.7.6, but that seems a bit out of date for year 2019...

Jaimie Livingston
That is the version of elasticsearch Nagios Log Server uses, we seek stability over bleeding edge for our commercial software.

The Kibana installed with Log Server is a highly modified 3.1.1

Re: Copying NLS Logstash and Kibana configurations?

Posted: Fri Oct 04, 2019 10:20 am
by jaimie.livingston
Thanks. I believe I have everything I need for this request now.
It will likely be a custom script leveraging the Elastic and Kibana APIs...

Jaimie Livingston

Re: Copying NLS Logstash and Kibana configurations?

Posted: Fri Oct 04, 2019 10:21 am
by scottwilkerson
jaimie.livingston wrote:Thanks. I believe I have everything I need for this request now.
It will likely be a custom script leveraging the Elastic and Kibana APIs...

Jaimie Livingston
Great!

Locking thread