Page 1 of 2

Log Server not creating backups

Posted: Thu Oct 13, 2016 1:55 pm
by dlukinski
Hello Server Support

Our LOG server is not creating backups. Configuration as attached, /mnt/local storage accessible, but no backups created.

Please help.

Re: Log Server not creating backups

Posted: Thu Oct 13, 2016 1:57 pm
by mcapra
Assuming this is NLS 1.4.x, can you share the output of the following commands:

Code: Select all

curator --debug snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d
ls -al /mnt/backup

Re: Log Server not creating backups

Posted: Thu Oct 13, 2016 2:53 pm
by dlukinski
mcapra wrote:Assuming this is NLS 1.4.x, can you share the output of the following commands:

Code: Select all

curator --debug snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d
ls -al /mnt/backup

curator produces VERY long output. Some portion of it is in the first file.
LS output includes snapshots copied there manully by me

Re: Log Server not creating backups

Posted: Thu Oct 13, 2016 3:06 pm
by mcapra
Failed to verify all nodes have repository access.
That shows up in the curator output. Do all nodes on your NLS cluster have write access to /mnt/backup?

Also please share the contents of your elasticsearch logs (usually located at /var/log/elasticsearch/*.log).

Re: Log Server not creating backups

Posted: Thu Oct 13, 2016 3:50 pm
by dlukinski
mcapra wrote:
Failed to verify all nodes have repository access.
That shows up in the curator output. Do all nodes on your NLS cluster have write access to /mnt/backup?

Also please share the contents of your elasticsearch logs (usually located at /var/log/elasticsearch/*.log).
backup directory would be 777
attached logs

where would create_backup.sh dump the backup?
- is there an argument to use 'mnt/backup' ?

Re: Log Server not creating backups

Posted: Thu Oct 13, 2016 5:03 pm
by mcapra
Looking at the previous ls -al output:

Code: Select all

[root@fikc-naglsprod01 /]# ls -al /mnt/backup
total 88
drwxr-xr-x  3 root   root    4096 Oct 13 18:49 .
This looks like 755 permissions on the path, not 777. Can you try a chmod 777 /mnt/backup and see if that fixes things?

Re: Log Server not creating backups

Posted: Thu Oct 13, 2016 5:07 pm
by avandemore
Backups will be in /store/backups/nagioslogserver/$TIMESTAMP

Re: Log Server not creating backups

Posted: Fri Oct 14, 2016 10:14 am
by dlukinski
mcapra wrote:Looking at the previous ls -al output:

Code: Select all

[root@fikc-naglsprod01 /]# ls -al /mnt/backup
total 88
drwxr-xr-x  3 root   root    4096 Oct 13 18:49 .
This looks like 755 permissions on the path, not 777. Can you try a chmod 777 /mnt/backup and see if that fixes things?
Did that.

How I could test-backup to /mnt/backup ?

Re: Log Server not creating backups

Posted: Fri Oct 14, 2016 10:16 am
by mcapra
You can "Reset All Jobs" in the Admin->Command Subsystem menu, or run the following command from the CLI:

Code: Select all

curator snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d
If it doesn't work, i'd like to see the full debug output again:

Code: Select all

curator --debug snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d

Re: Log Server not creating backups

Posted: Fri Oct 14, 2016 11:28 am
by dlukinski
mcapra wrote:You can "Reset All Jobs" in the Admin->Command Subsystem menu, or run the following command from the CLI:

Code: Select all

curator snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d
If it doesn't work, i'd like to see the full debug output again:

Code: Select all

curator --debug snapshot --repository Local indices --older-than 1 --time-unit days --timestring %Y.%m.%d
-------------------------------
Seem to be successful:

2016-10-14 16:14:47,142 INFO Snapshot name: curator-20161014161447
2016-10-14 16:26:07,820 INFO Snapshot curator-20161014161447 successfully completed.
2016-10-14 16:26:07,820 INFO Job completed successfully.