Page 1 of 5

Nagios Log Server - Backup and Maintenance

Posted: Wed Dec 10, 2014 10:11 am
by roddergreg
I've installed the manual install of Nagios Log Server on RHEL 6.5 server. And most things seem to work however I'm having issues with the backup and maintenance.

I've set

Optimize Indexes older than 2
Disable Bloom Filter Cache older than 1
Close indexes older than 5
Delete indexes older than 10
Enable Maintenance and Backups Yes

The indexes are never optimized,closed or deleted. I can however close and delete things manually.

I've been patient as the instructions state that it might take a couple days for the backups and maintenance to start. No Luck over 10 days.

Am I missing a setting somewhere. Can someone help me troubleshoot why the backups and maintenance jobs are not running correctly

Re: Nagios Log Server - Backup and Maintenance

Posted: Wed Dec 10, 2014 3:27 pm
by lgroschen
Did you create a repository first? Then select the created repository under "Repository to store backups in" dropdown. Then select save settings.

Re: Nagios Log Server - Backup and Maintenance

Posted: Wed Dec 10, 2014 4:08 pm
by roddergreg
I've created fs backup repository in /tmp. Just like the document its called NagiosBackup. There should'nt been any permissions issues writing to /tmp
Greg

Re: Nagios Log Server - Backup and Maintenance

Posted: Wed Dec 10, 2014 5:39 pm
by lgroschen
Ok tail this log:

Code: Select all

tail -f /var/log/elasticsearch/<cluster_id>.log
There should only be one .log in the folder. Then open a new terminal and run the backup script.

Post all the output from the tail while the backup script runs here in code tags.

Re: Nagios Log Server - Backup and Maintenance

Posted: Thu Dec 11, 2014 10:20 am
by roddergreg

Code: Select all

845536790d63"}, {"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,104][INFO ][KnapsackExportAction     ] start of export: {"mode":"export","started":"2014-12-11T14:16:59.093Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}
[2014-12-11 09:16:59,104][INFO ][KnapsackExportAction     ] getting settings for indices [nagioslogserver]
[2014-12-11 09:16:59,105][INFO ][KnapsackService          ] update cluster settings: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-11T14:16:59.093Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,194][INFO ][KnapsackExportAction     ] found indices: [nagioslogserver]
[2014-12-11 09:16:59,197][INFO ][KnapsackExportAction     ] getting mappings for index nagioslogserver and types []
[2014-12-11 09:16:59,217][INFO ][KnapsackExportAction     ] found mappings: [cf_option, node, reactor_server, query, snapshot, _default_, alert, commands, snmp_reactor, nrdp_server, user]
[2014-12-11 09:16:59,226][INFO ][KnapsackExportAction     ] adding mapping: cf_option
[2014-12-11 09:16:59,226][INFO ][KnapsackExportAction     ] adding mapping: node
[2014-12-11 09:16:59,226][INFO ][KnapsackExportAction     ] adding mapping: reactor_server
[2014-12-11 09:16:59,226][INFO ][KnapsackExportAction     ] adding mapping: query
[2014-12-11 09:16:59,227][INFO ][KnapsackExportAction     ] adding mapping: snapshot
[2014-12-11 09:16:59,227][INFO ][KnapsackExportAction     ] adding mapping: _default_
[2014-12-11 09:16:59,227][INFO ][KnapsackExportAction     ] adding mapping: alert
[2014-12-11 09:16:59,256][INFO ][KnapsackService          ] add: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-11T14:16:59.093Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,259][INFO ][KnapsackService          ] update cluster settings: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-11T14:16:59.093Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-11T14:16:59.254Z","path":"file:///store/backups/nagioslogserver/1418307418/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,269][INFO ][KnapsackExportAction     ] adding mapping: commands
[2014-12-11 09:16:59,284][INFO ][KnapsackExportAction     ] adding mapping: snmp_reactor
[2014-12-11 09:16:59,284][INFO ][KnapsackExportAction     ] adding mapping: nrdp_server
[2014-12-11 09:16:59,285][INFO ][KnapsackExportAction     ] adding mapping: user
[2014-12-11 09:16:59,285][INFO ][KnapsackExportAction     ] getting aliases for index nagioslogserver
[2014-12-11 09:16:59,285][INFO ][KnapsackExportAction     ] found 0 aliases
[2014-12-11 09:16:59,297][INFO ][KnapsackService          ] add: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-11T14:16:59.093Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-11T14:16:59.254Z","path":"file:///store/backups/nagioslogserver/1418307418/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,317][INFO ][KnapsackService          ] update cluster settings: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-11T14:16:59.093Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-11T14:16:59.254Z","path":"file:///store/backups/nagioslogserver/1418307418/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-11T14:16:59.294Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver_log.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,340][INFO ][KnapsackExportAction     ] end of export: {"mode":"export","started":"2014-12-11T14:16:59.093Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, packets = 201, total bytes transferred = 45752, rate = 9150.400000
[2014-12-11 09:16:59,344][INFO ][KnapsackService          ] remove: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-11T14:16:59.093Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-11T14:16:59.254Z","path":"file:///store/backups/nagioslogserver/1418307418/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-11T14:16:59.294Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver_log.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,346][INFO ][KnapsackService          ] update cluster settings: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-11T14:16:59.254Z","path":"file:///store/backups/nagioslogserver/1418307418/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-11T14:16:59.294Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver_log.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,371][INFO ][KnapsackExportAction     ] start of export: {"mode":"export","started":"2014-12-11T14:16:59.254Z","path":"file:///store/backups/nagioslogserver/1418307418/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}
[2014-12-11 09:16:59,371][INFO ][KnapsackExportAction     ] getting settings for indices [kibana-int]
[2014-12-11 09:16:59,372][INFO ][KnapsackExportAction     ] found indices: [kibana-int]
[2014-12-11 09:16:59,373][INFO ][KnapsackExportAction     ] getting mappings for index kibana-int and types []
[2014-12-11 09:16:59,374][INFO ][KnapsackExportAction     ] found mappings: [temp, dashboard]
[2014-12-11 09:16:59,374][INFO ][KnapsackExportAction     ] adding mapping: temp
[2014-12-11 09:16:59,374][INFO ][KnapsackExportAction     ] adding mapping: dashboard
[2014-12-11 09:16:59,374][INFO ][KnapsackExportAction     ] getting aliases for index kibana-int
[2014-12-11 09:16:59,375][INFO ][KnapsackExportAction     ] found 0 aliases
[2014-12-11 09:16:59,466][INFO ][KnapsackExportAction     ] end of export: {"mode":"export","started":"2014-12-11T14:16:59.254Z","path":"file:///store/backups/nagioslogserver/1418307418/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, packets = 17, total bytes transferred = 4605, rate = 921.000000
[2014-12-11 09:16:59,469][INFO ][KnapsackService          ] remove: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-11T14:16:59.254Z","path":"file:///store/backups/nagioslogserver/1418307418/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-11T14:16:59.294Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver_log.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,470][INFO ][KnapsackService          ] update cluster settings: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-11T14:16:59.294Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver_log.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:16:59,493][INFO ][KnapsackExportAction     ] start of export: {"mode":"export","started":"2014-12-11T14:16:59.294Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver_log.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}
[2014-12-11 09:16:59,493][INFO ][KnapsackExportAction     ] getting settings for indices [nagioslogserver_log]
[2014-12-11 09:16:59,494][INFO ][KnapsackExportAction     ] found indices: [nagioslogserver_log]
[2014-12-11 09:16:59,495][INFO ][KnapsackExportAction     ] getting mappings for index nagioslogserver_log and types []
[2014-12-11 09:16:59,496][INFO ][KnapsackExportAction     ] found mappings: [JOBS, CONFIG, MAINTENANCE, POLLER, SECURITY, INFO, BACKUP, ALERT]
[2014-12-11 09:16:59,496][INFO ][KnapsackExportAction     ] adding mapping: JOBS
[2014-12-11 09:16:59,497][INFO ][KnapsackExportAction     ] adding mapping: CONFIG
[2014-12-11 09:16:59,497][INFO ][KnapsackExportAction     ] adding mapping: MAINTENANCE
[2014-12-11 09:16:59,497][INFO ][KnapsackExportAction     ] adding mapping: POLLER
[2014-12-11 09:16:59,497][INFO ][KnapsackExportAction     ] adding mapping: SECURITY
[2014-12-11 09:16:59,497][INFO ][KnapsackExportAction     ] adding mapping: INFO
[2014-12-11 09:16:59,497][INFO ][KnapsackExportAction     ] adding mapping: BACKUP
[2014-12-11 09:16:59,498][INFO ][KnapsackExportAction     ] adding mapping: ALERT
[2014-12-11 09:16:59,498][INFO ][KnapsackExportAction     ] getting aliases for index nagioslogserver_log
[2014-12-11 09:16:59,498][INFO ][KnapsackExportAction     ] found 0 aliases
[2014-12-11 09:17:03,404][INFO ][KnapsackExportAction     ] end of export: {"mode":"export","started":"2014-12-11T14:16:59.294Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver_log.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, packets = 172500, total bytes transferred = 26583724, rate = 5316744.800000
[2014-12-11 09:17:03,405][INFO ][KnapsackService          ] remove: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"}, {"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}, {"mode":"export","started":"2014-12-11T14:16:59.294Z","path":"file:///store/backups/nagioslogserver/1418307418/nagioslogserver_log.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]
[2014-12-11 09:17:03,406][INFO ][KnapsackService          ] update cluster settings: plugin.knapsack.export.state -> [{"mode":"export","started":"2014-12-02T15:55:27.222Z","path":"file:///store/backups/nagioslogserver/1417535727/nagioslogserver.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-03T15:55:27.399Z","path":"file:///store/backups/nagioslogserver/1417622127/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-04T15:55:31.799Z","path":"file:///store/backups/nagioslogserver/1417708531/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:31.987Z","path":"file:///store/backups/nagioslogserver/1417881331/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-06T15:55:32.329Z","path":"file:///store/backups/nagioslogserver/1417881332/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-07T15:55:36.815Z","path":"file:///store/backups/nagioslogserver/1417967736/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:31.707Z","path":"file:///store/backups/nagioslogserver/1418054131/nagioslogserver.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"},{"mode":"export","started":"2014-12-08T15:55:36.728Z","path":"file:///store/backups/nagioslogserver/1418054136/kibana-int.tar.gz","node_name":"13809291-08db-40fa-b771-845536790d63"},{"mode":"export","started":"2014-12-09T15:55:31.694Z","path":"file:///store/backups/nagioslogserver/1418140531/kibana-int.tar.gz","node_name":"bae70b56-3057-466a-acfd-c7631c35033c"}]

Re: Nagios Log Server - Backup and Maintenance

Posted: Thu Dec 11, 2014 12:48 pm
by lgroschen
I think this is a problem with the job being stuck. Let me look through the code so I can create a query you can run and we can find out if this is the issue.

Re: Nagios Log Server - Backup and Maintenance

Posted: Thu Dec 11, 2014 1:02 pm
by jomann
Can you please run the following in the command line on the nagios log server:

Code: Select all

curl -XGET 'http://localhost:9200/nagioslogserver/commands/_search?pretty&q=command:do_maintenance'
Please post the output of that - we can then see what's going on with the do_maintenance command which is supposed to be running every day.

Re: Nagios Log Server - Backup and Maintenance

Posted: Thu Dec 11, 2014 1:49 pm
by roddergreg

Code: Select all

 # curl -XGET 'http://localhost:9200/nagioslogserver/commands/_search?pretty&q=command:do_maintenance'
{
  "took" : 19,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 5.610157,
    "hits" : [ {
      "_index" : "nagioslogserver",
      "_type" : "commands",
      "_id" : "7CefFIprSgmAv0HEM0dxPg",
      "_score" : 5.610157,
      "_source":{"created":"2014-10-28 11:46:59","active":1,"status":"waiting","type":"system","node":"global","command":"do_maintenance","run_time":1418399772,"frequency":"86400","last_run_output":"Maintenance and Backup jobs are being executed","last_run_time":"2014-12-11 10:56:12","last_run_status":"SUCCESS"}
    } ]
  }
}

Re: Nagios Log Server - Backup and Maintenance

Posted: Fri Dec 12, 2014 4:40 pm
by Nagios Support
The devs will be back in the office on Monday and will follow up with you.

Re: Nagios Log Server - Backup and Maintenance

Posted: Mon Dec 15, 2014 1:46 pm
by lgroschen
roddergreg,

Can you post that last curl command again? We'll see if anything changed. Also, can you post a screenshot of your cluster status page?