NL Snapshots not working properly.

[danniiffxi]

Hi Guys

Recently (5th March 2021) migrated NL to CentOS7, ever since the migration I am seeing these results on the snapshots, they are constantly stuck on IN_PROGRESS and there is no end date or time. How do I fix this please? Is it even backing up correctly?

[benjaminsmith]

Hi @danniiffxi,

Have you tried to reset the jobs or manually running the snapshots_maintenance job. To do so, go to Admin > System Status > Command Subsystem.

Do all the nodes have access to the repository with the correct permissions.

If that's not the issue, do you see any errors the cron log?

Code: Select all

tail -n 100 /var/log/cron

Regards,
Benjamin

Reference
Managing Snapshots and Maintenance

[danniiffxi]

Hi Benjamin,

I reset all job when I removed the last old node form the cluster and I was on the 2 new servers.

I'm pretty sure the nodes had the correct permissions to the snapshot storage folder, but I ran the commands in the attached doc you sent with your last post anyway. I am not seeing any errors in the cron log, just the following which drops into the log file every min.

Code: Select all

Mar 25 15:41:01 naglp02 CROND[14791]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1)
Mar 25 15:41:01 naglp02 CROND[14793]: (apache) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs/apache > /usr/local/nagioslogserver/var/jobs_apache.log 2>&1)
Mar 25 15:41:01 naglp02 CROND[14792]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1)
Mar 25 15:42:01 naglp02 CROND[15012]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1)
Mar 25 15:42:01 naglp02 CROND[15013]: (apache) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs/apache > /usr/local/nagioslogserver/var/jobs_apache.log 2>&1)
Mar 25 15:42:01 naglp02 CROND[15014]: (nagios) CMD (/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1)

Also when I run

Code: Select all

tail -f /usr/local/nagioslogserver/var/jobs.log

I get the following

Code: Select all

Running command do_maintenance with args ' ' for job id: snapshots_maintenance
tail: /usr/local/nagioslogserver/var/jobs.log: file truncated
Running command do_maintenance with args ' ' for job id: snapshots_maintenance
tail: /usr/local/nagioslogserver/var/jobs.log: file truncated
Running command do_maintenance with args ' ' for job id: snapshots_maintenance

So I am not seeing any errors.

However the job has been running now for almost 5 hours. so I am going to leave it overnight and report back tomorrow.

[danniiffxi]

Yup all working now. Looks like it may have been the permissions after all. You can lock this one.

Many thanks!

[benjaminsmith]

Hi @danniiffxi

Excellent. Glad that's working now.

We'll close this out.