Snapshot Storage Setup

[hcltech]

I have my snapshot storage setup but its been over a week and its done nothing. Can someone look at what i have setup and let me know what I am doing wrong here?

[cdienger]

What version of NLS and what OS is it installed on?

Do you see the remote storage mounted and can you write to it? Try running:

Code: Select all

touch /mnt/storage_repository/test
ll /mnt/storage_repository/test
mount

and manually start a snapshot from the command line:

Code: Select all

/usr/local/nagioslogserver/scripts/curator.sh snapshot --repository 'QNAP' --ignore_unavailable indices --older-than 1 --time-unit days --timestring %Y.%m.%d

[hcltech]

I am able to write files to the storage server no issues. When I tried to run your command I got the following error.

root@cl-p-nagioslog01:~# /usr/local/nagioslogserver/scripts/curator.sh snapshot --repository 'QNAP' --ignore_unavailable indices --older-than 1 --time-unit days
--time string %Y.%m.%d'
> ^C
root@cl-p-nagioslog01:~# /usr/local/nagioslogserver/scripts/curator.sh snapshot --repository 'QNAP' --ignore_unavailable indices --older-than 1 --time-unit days
--time string '%Y.%m.%d'
/usr/lib/python2.7/dist-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.25.3) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
Usage: curator snapshot indices [OPTIONS]
Try "curator snapshot indices --help" for help.

Error: no such option: --time (Possible options: --time-unit, --timestring)
root@cl-p-nagioslog01:~# /usr/local/nagioslogserver/scripts/curator.sh snapshot --repository 'QNAP' --ignore_unavailable indices --older-than 1 --time-unit days
--time string %Y.%m.%d
/usr/lib/python2.7/dist-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.25.3) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
Usage: curator snapshot indices [OPTIONS]
Try "curator snapshot indices --help" for help.

Error: no such option: --time (Possible options: --time-unit, --timestring)
root@cl-p-nagioslog01:~# /usr/local/nagioslogserver/scripts/curator.sh snapshot --repository 'QNAP' --ignore_unavailable indices --older-than 1 --time-unit days
--time 'string %Y.%m.%d'
/usr/lib/python2.7/dist-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.25.3) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
Usage: curator snapshot indices [OPTIONS]
Try "curator snapshot indices --help" for help.

Error: no such option: --time (Possible options: --time-unit, --timestring)

[cdienger]

Typo in the command provided. Corrected now. Try this:

Code: Select all

/usr/local/nagioslogserver/scripts/curator.sh snapshot --repository 'QNAP' --ignore_unavailable indices --older-than 1 --time-unit days --timestring %Y.%m.%d

Also provide the output of:

Code: Select all

ll /mnt/storage_repository/

Sometimes a corrupt backup will prevent items from listing correctly and we can identify them by their size of 0 bytes.

[hcltech]

root@cl-p-nagioslog01:~# /usr/local/nagioslogserver/scripts/curator.sh snapshot --repository 'QNAP' --ignore_unavailable indices --older-than 1 --time-unit days --timestring %Y.%m.%d
/usr/lib/python2.7/dist-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.25.3) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
2019-09-06 15:18:01,002 INFO Job starting: snapshot indices
2019-09-06 15:18:01,003 WARNING Overriding default connection timeout. New timeout: 21600
2019-09-06 15:18:01,038 INFO Action snapshot will be performed on the following indices: [u'logstash-2019.08.25', u'logstash-2019.08.26', u'logstash-2019.08.27', u'logstash-2019.08.28', u'logstash-2019.08.29', u'logstash-2019.08.30', u'logstash-2019.08.31', u'logstash-2019.09.01', u'logstash-2019.09.02', u'logstash-2019.09.03', u'logstash-2019.09.04', u'logstash-2019.09.05']
2019-09-06 15:18:04,036 INFO Snapshot name: curator-20190906191804
2019-09-06 16:51:34,966 INFO Snapshot curator-20190906191804 successfully completed.
2019-09-06 16:51:34,977 INFO Job completed successfully.
root@cl-p-nagioslog01:~# ll /mnt/storage_repository/
total 16
drwxr-xr-x 2 root root 0 Sep 6 16:51 ./
drwxr-xr-x 4 root root 4096 Aug 27 15:06 ../
-rwxr-xr-x 1 root root 45 Sep 6 16:51 index*
drwxr-xr-x 2 root root 0 Sep 6 15:18 indices/
-rwxr-xr-x 1 root root 521 Sep 6 15:18 metadata-curator-20190906191804*
dr-xr-xr-x 2 root root 0 Aug 27 14:52 '@Recently-Snapshot'/
drwxr-xr-x 2 root root 0 Sep 6 15:18 '@Recycle'/
-rwxr-xr-x 1 root root 272 Sep 6 16:51 snapshot-curator-20190906191804*

[cdienger]

Make sure the proper urllib3 library is installed with:

Code: Select all

pip uninstall urllib3
yum install python-urllib3

and then try running the curator.sh command again and check in the web UI. If the issue still persists, please provide a profile. It can be gathered under Admin > System > System Status > Download System Profile or from the command line with:

/usr/local/nagioslogserver/scripts/profile.sh

This will create /tmp/system-profile.tar.gz.

Note that this file can be very large and may not be able to be uploaded through the system. This is usually due to the logs in the Logstash and/or Elasticseach directories found in it. If it is too large, please open the profile, extract these directories/files and send them separately.

[hcltech]

Is it going to matter that this is installed on Ubuntu and not CentOS ?

The pip uninstall command worked but I know yum wont and when i do the apt-get equivalent i get this.
root@cl-p-nagioslog01:~# apt install python-urllib3
Reading package lists... Done
Building dependency tree
Reading state information... Done
python-urllib3 is already the newest version (1.22-1ubuntu0.18.04.1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

[cdienger]

Sorry, I assumed CentOS. Is the curator command still failing with the same message on the command line now that the pip uninstall was run?

[hcltech]

no that command worked after I ran it without the typos sorry. But its still not running auto by setting up the stuff in the settings like I posted in the first post.

[cdienger]

Edit /usr/local/nagioslogserver/scripts/curator.sh to look like:

Code: Select all

#!/bin/sh

/usr/local/bin/curator "$@"

This will be fixed in the next release.