Daily Snapshot Missing/Backups Job Failing

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
lazzarinof
Posts: 50
Joined: Thu Sep 23, 2021 12:26 pm

Re: Daily Snapshot Missing/Backups Job Failing

Post by lazzarinof »

Good afternoon Perry,

I'm not sure if it's stuck on any particular export: I removed all of the .json files (it looks like they're not deleting automatically, and our collection has grown quite large!).

I updated the create_backup.sh script, to wait for 300 iterations, and it's running now (but will take a while... can't rush perfection!).

I'll be back with an update once that's complete.

Thank you,
-Frank
lazzarinof
Posts: 50
Joined: Thu Sep 23, 2021 12:26 pm

Re: Daily Snapshot Missing/Backups Job Failing

Post by lazzarinof »

Darn, it looks like it timed out waiting for an available spot (just took longer).

I exited before screen-capping, so I ran it again and got the attached screenshot, showing the loop again. I'll let it run again, because no harm there.

Thank you,
-Frank
You do not have the required permissions to view the files attached to this post.
User avatar
pbroste
Posts: 1288
Joined: Tue Jun 01, 2021 1:27 pm

Re: Daily Snapshot Missing/Backups Job Failing

Post by pbroste »

Hello @lazzarinof

What did the second run-through on the backup script look like?

Thanks,
Perry
lazzarinof
Posts: 50
Joined: Thu Sep 23, 2021 12:26 pm

Re: Daily Snapshot Missing/Backups Job Failing

Post by lazzarinof »

Good afternoon Perry,

I got the same results the second time: it timed out after running through all 300 iterations.

Thank you,
-Frank
User avatar
pbroste
Posts: 1288
Joined: Tue Jun 01, 2021 1:27 pm

Re: Daily Snapshot Missing/Backups Job Failing

Post by pbroste »

Hello @lazzarinof

Let's run through the following and let me know the results:

Code: Select all

curl -XPUT localhost:9200/_cluster/settings -d '{"transient" : {"plugin.knapsack.export.state" : "[]"}}'

Code: Select all

bash -x /usr/local/nagioslogserver/scripts/create_backup.sh | tee /tmp/backup.log
Thanks,
Perry
lazzarinof
Posts: 50
Joined: Thu Sep 23, 2021 12:26 pm

Re: Daily Snapshot Missing/Backups Job Failing

Post by lazzarinof »

Good morning Perry,

It looks like we have a winner!
After the curl command, the backup you listed worked. And the standard backup is working too.

It looks like the .json files are still present, but I'm hoping they'll clear out as the backups are cleaned out (otherwise I'll remove them later).

Thank you!
-Frank
You do not have the required permissions to view the files attached to this post.
User avatar
pbroste
Posts: 1288
Joined: Tue Jun 01, 2021 1:27 pm

Re: Daily Snapshot Missing/Backups Job Failing

Post by pbroste »

Hello @lazzarinof

Excellent, let us know how things look and follow up and let us know if we can lock this.

Thanks,
Perry
lazzarinof
Posts: 50
Joined: Thu Sep 23, 2021 12:26 pm

Re: Daily Snapshot Missing/Backups Job Failing

Post by lazzarinof »

Good afternoon Perry,

It's all looking good! Lock 'er up, and thank you again!

-Frank
Locked