Log Server not creating backups

[dlukinski]

Hello Server Support

Our LOG server is not creating backups. Configuration as attached, /mnt/local storage accessible, but no backups created.

Please help.

[mcapra]

Assuming this is NLS 1.4.x, can you share the output of the following commands:

Code: Select all

curator --debug snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d
ls -al /mnt/backup

[dlukinski]

Assuming this is NLS 1.4.x, can you share the output of the following commands:

Code: Select all

curator --debug snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d
ls -al /mnt/backup

curator produces VERY long output. Some portion of it is in the first file.
LS output includes snapshots copied there manully by me

[mcapra]

Failed to verify all nodes have repository access.

That shows up in the curator output. Do all nodes on your NLS cluster have write access to /mnt/backup?

Also please share the contents of your elasticsearch logs (usually located at /var/log/elasticsearch/*.log).

[dlukinski]

Failed to verify all nodes have repository access.

That shows up in the curator output. Do all nodes on your NLS cluster have write access to /mnt/backup?

Also please share the contents of your elasticsearch logs (usually located at /var/log/elasticsearch/*.log).

backup directory would be 777
attached logs

where would create_backup.sh dump the backup?
- is there an argument to use 'mnt/backup' ?

[mcapra]

Looking at the previous ls -al output:

Code: Select all

[root@fikc-naglsprod01 /]# ls -al /mnt/backup
total 88
drwxr-xr-x  3 root   root    4096 Oct 13 18:49 .

This looks like 755 permissions on the path, not 777. Can you try a chmod 777 /mnt/backup and see if that fixes things?

[avandemore]

Backups will be in /store/backups/nagioslogserver/$TIMESTAMP

[dlukinski]

Looking at the previous ls -al output:

Code: Select all

[root@fikc-naglsprod01 /]# ls -al /mnt/backup
total 88
drwxr-xr-x  3 root   root    4096 Oct 13 18:49 .

This looks like 755 permissions on the path, not 777. Can you try a chmod 777 /mnt/backup and see if that fixes things?

Did that.

How I could test-backup to /mnt/backup ?

[mcapra]

You can "Reset All Jobs" in the Admin->Command Subsystem menu, or run the following command from the CLI:

Code: Select all

curator snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d

If it doesn't work, i'd like to see the full debug output again:

Code: Select all

curator --debug snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d

[dlukinski]

You can "Reset All Jobs" in the Admin->Command Subsystem menu, or run the following command from the CLI:

Code: Select all

curator snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d

If it doesn't work, i'd like to see the full debug output again:

Code: Select all

curator --debug snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d

-------------------------------
Seem to be successful:

2016-10-14 16:14:47,142 INFO Snapshot name: curator-20161014161447
2016-10-14 16:26:07,820 INFO Snapshot curator-20161014161447 successfully completed.
2016-10-14 16:26:07,820 INFO Job completed successfully.