Copying NLS Logstash and Kibana configurations?

[jaimie.livingston]

Is there a procedure for copying NLS Logstash (inputs, filters) and Kibana (dashboards, queries) between clusters?
SCP, Rsync?

Use Case:
I will have several NLS cluster deployments across different environments.
One cluster will be for developing new inputs/input filters and dashboards, all others are for production.

Is there a way to scp/rsync configurations from the "development" cluster to all "production" clusters.


Thanks,

Jaimie Livingston

[cdienger]

The logstash and kibana configs are all part of backups created by the backups job under Admin > System > Command Subsystem. The backups are stored in /store/backups/nagioslogserver/ and can be imported with:

Code: Select all

cd /usr/local/nagioslogserver/scripts/
./restore_backup.sh /store/backups/nagioslogserver/nagioslogserver.<timestamp>.tar.gz

https://assets.nagios.com/downloads/nag ... Server.pdf

[jaimie.livingston]

If I understand the document correctly, this will not work for me...
According to the docs:

System Backups
System backups contain the configuration settings, dashboards, users, internal logs, alerts. Also included are the Inputs, Filters and Outputs for Logstash

Since the systems I need to sync are in different environments, they will possibly have different configurations and users, and should not have any modifications to logs for access or audits.

All I want to do is copy the Kibana Global Dashboards and Logstash Inputs, Filters and Outputs from one cluster to another, without modifying any other configurations, user accounts, logs, or indexes.

Config Snapshots capture the Logstash Inputs, Filters, and Outputs, but, as best I can tell from the docs and from examining a snapshot database.json file, don't capture the Kibana Dashboards.

So, if I wanted to copy the Logstash Inputs, Filters, and Outputs AND the Kibana Global Dashboards from one cluster to another cluster, is there a method I can use? I can write my own script, if I know which files to copy...

Thanks,

Jaimie Livingston

[scottwilkerson]

Config Snapshots capture the Logstash Inputs, Filters, and Outputs, but, as best I can tell from the docs and from examining a snapshot database.json file, don't capture the Kibana Dashboards.

this is correct. the problem is that the kibana dashboards are associated with users...

You cna back them up with something like

Code: Select all

curl -XPOST "http://localhost:9200/kibana-int/_export?path=/store/backups/nagioslogserver/TESTBACKUP/kibana-int.tar.gz"

then transfer /store/backups/nagioslogserver/TESTBACKUP/kibana-int.tar.gz to the new server restore with something like

Code: Select all

curl -XPOST "http://localhost:9200/kibana-int/_import?path=/store/backups/nagioslogserver/TESTBACKUP/kibana-int.tar.gz"

BUT, this will have all the user dashboards too which could cause unexpected behavior

[jaimie.livingston]

Excellent information. Thanks.

For purposes of using the Kibana API calls, what is the effective ElasticSearch/Kibana version?
A curl to localhost returned version 1.7.6, but that seems a bit out of date for year 2019...

Jaimie Livingston

[scottwilkerson]

Excellent information. Thanks.

For purposes of using the Kibana API calls, what is the effective ElasticSearch/Kibana version?
A curl to localhost returned version 1.7.6, but that seems a bit out of date for year 2019...

Jaimie Livingston

That is the version of elasticsearch Nagios Log Server uses, we seek stability over bleeding edge for our commercial software.

The Kibana installed with Log Server is a highly modified 3.1.1

[jaimie.livingston]

Thanks. I believe I have everything I need for this request now.
It will likely be a custom script leveraging the Elastic and Kibana APIs...

Jaimie Livingston

[scottwilkerson]

Thanks. I believe I have everything I need for this request now.
It will likely be a custom script leveraging the Elastic and Kibana APIs...

Jaimie Livingston

Great!

Locking thread