Page 2 of 3
Re: Local Scheduled Backup failure
Posted: Fri Jan 13, 2017 8:12 am
by NetNeutron
# ls -lR /usr/local/nagiosxi/html/includes/components/scheduledbackups/
Code: Select all
/usr/local/nagiosxi/html/includes/components/scheduledbackups/:
total 432
-rw-r----- 1 nagios nagios 5803 Dec 13 09:25 ajaxcalls.php
-rw-r----- 1 nagios nagios 2025 Dec 13 09:25 CHANGELOG.txt
drwxr-x--- 2 nagios nagios 4096 Aug 16 10:13 images
drwxr-x--- 2 nagios nagios 4096 Aug 16 10:13 includes
-rwxr-x--- 1 nagios nagios 323 Dec 13 09:25 install.sh
-rw-r----- 1 nagios nagios 93972 Dec 13 09:25 localbackups.php
-rw-r----- 1 nagios nagios 171636 Dec 13 09:25 managebackups.php
-rw-r----- 1 nagios nagios 149076 Dec 13 09:25 scheduledbackups.inc.php
/usr/local/nagiosxi/html/includes/components/scheduledbackups/images:
total 16
-rw-r----- 1 nagios nagios 781 Dec 13 09:25 accept.png
-rw-r----- 1 nagios nagios 4178 Dec 13 09:25 ajax-loader.gif
-rw-r----- 1 nagios nagios 701 Dec 13 09:25 exclamation.png
/usr/local/nagiosxi/html/includes/components/scheduledbackups/includes:
total 8
-rw-r----- 1 nagios nagios 93 Dec 13 09:25 ftp_test_upload.txt
-rw-r----- 1 nagios nagios 110 Dec 13 09:25 ssh_test_secure_copy.txt
Re: Local Scheduled Backup failure
Posted: Fri Jan 13, 2017 9:49 am
by tgriep
Thanks for the Log files. Bad thing is that there is nothing bad in it.
You can either move this to a support ticket by emailing
[email protected] and linking this post or you can PM me your system profile.
To send us your system profile. login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and email it back to us.
Re: Local Scheduled Backup failure
Posted: Fri Jan 13, 2017 3:46 pm
by tgriep
Thanks for the Profile. I am seeing some issues with MYSQL database and that could cause the issue you are having.
Login as root to the XI server and run the following to repair the database.
Code: Select all
mysqlcheck -f -r -u root -pnagiosxi --use_frm --databases nagios nagiosql nagiosxi
Then to be safe, run the following to clear out the message queue of any old data trying to to be written to the database.
Run the following as root.
Code: Select all
service nagios stop
service ndo2db stop
for i in `ipcs -q | grep nagios |awk '{print $2}'`; do ipcrm -q $i; done
service ndo2db start
service nagios start
Then run this and post the output.
Code: Select all
ipcs -q
mysql -u root -pnagiosxi -e "show global status like '%used_connections%'; show variables like 'max_connections';"
Then try and schedule a backup to see if it works.
Re: Local Scheduled Backup failure
Posted: Sat Jan 14, 2017 2:13 pm
by NetNeutron
# ipcs -q
Code: Select all
------ Message Queues --------
key msqid owner perms used-bytes messages
0xaf000002 327680 nagios 600 30541824 29826
# mysql -u root -pnagiosxi -e "show global status like '%used_connections%'; show variables like 'max_connections';"
Code: Select all
+----------------------+-------+
| Variable_name | Value |
+----------------------+-------+
| Max_used_connections | 152 |
+----------------------+-------+
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 151 |
+-----------------+-------+
Re: Local Scheduled Backup failure
Posted: Mon Jan 16, 2017 10:43 am
by tgriep
The max_connections to the MYSQL server are not high enough and that is causing issues when the Nagios processes are writing to the mysql database.
To increase those settings, edit the /etc/my.cnf file
Under the [mysqld] section add the following setting:
Save the file and restart mysql by running the following.
Then re-run the repair of the mysql database by running
Code: Select all
mysqlcheck -f -r -u root -pnagiosxi --use_frm --databases nagios nagiosql nagiosxi
Delete the current scheduled local backup, recreate it and see if that fixes the issue.
Re: Local Scheduled Backup failure
Posted: Mon Jan 16, 2017 11:13 am
by SteveBeauchemin
Another possibility maybe?
Did sudoers get an update for 5.4 that added this
Code: Select all
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/backup_xi.sh *
If that is missing would backup fail?
Just a thought.
Steve B
Re: Local Scheduled Backup failure
Posted: Mon Jan 16, 2017 1:43 pm
by tgriep
Thanks SteveBeauchemin for the tip.
Re: Local Scheduled Backup failure
Posted: Wed Jan 18, 2017 8:08 am
by NetNeutron
Still no joy.
# mysql -u root -pnagiosxi -e "show global status like '%used_connections%'; show variables like 'max_connections';"
Code: Select all
+----------------------+-------+
| Variable_name | Value |
+----------------------+-------+
| Max_used_connections | 110 |
+----------------------+-------+
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 300 |
+-----------------+-------+
I did the restart and the repair.
This line is in sudoers
Code: Select all
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/backup_xi.sh *
I did a local backup 10 minutes into the future
# cat /usr/local/nagiosxi/var/components/scheduledbackups.log
Code: Select all
01-18-2017 07:10:02 DEBUG: Running scheduled local backup ...
01-18-2017 07:10:02 INFO: Creating a local backup: nagiosxi.1484741402
01-18-2017 07:10:02 DEBUG: Sending create local backup command to CmdSubsystem
No such file or directory exists in /store/backups/nagiosxi
# ll /store/backups/nagiosxi/ | grep 1484741402
Temporarily I am running backup_xi.sh as a cronjob and it is working...
Code: Select all
15 0 * * * /usr/local/nagiosxi/scripts/backup_xi.sh >/dev/null 2>&1
Please advise...
Re: Local Scheduled Backup failure
Posted: Wed Jan 18, 2017 11:40 am
by tgriep
Let's check these log files for any errors.
Run this as root on the XI server.
Code: Select all
tail -f /usr/local/nagiosxi/var/e*.log
Then submit another scheduled backup job and post the output of the tail command.
Re: Local Scheduled Backup failure
Posted: Thu Jan 19, 2017 7:58 am
by NetNeutron
Good Morning.
I sent the tail output in a PM.