Schedule local backups are no longer operational

[dlukinski]

Hello XI support

Our XI local backups are no longer being created after 5.6.3 upgrade done (from 5.5.11)

- enabled, configured, but not created

Thank you

[lmiltchev]

Go to Admin > Scheduled Backups > Local, schedule a backup to run in a few minutes, and click on "Update Settings". Start a running tail on cmdsubsys.log:

Code: Select all

tail -f /usr/local/nagiosxi/var/cmdsubsys.log

to start capturing the output. Wait until the backup is complete, stop the tail (ctrl+c), copy/paste the output on the forum.

Also, enable debugging for schedule backups by modifying the last line in the /usr/local/nagiosxi/html/includes/constants.inc.php file as such:

// Scheduled backups constants
// 0 - Debug, 1 - Info, 2 - Warning, 3 - Error
define('SB_LOGLEVEL', 0);

Save, exit, and restart apache:

Code: Select all

service httpd restart

Run a local scheduled backup, then post the /usr/local/nagiosxi/var/components/scheduledbackups.log on the forum.

[dlukinski]

Go to Admin > Scheduled Backups > Local, schedule a backup to run in a few minutes, and click on "Update Settings". Start a running tail on cmdsubsys.log:

Code: Select all

tail -f /usr/local/nagiosxi/var/cmdsubsys.log

to start capturing the output. Wait until the backup is complete, stop the tail (ctrl+c), copy/paste the output on the forum.

Also, enable debugging for schedule backups by modifying the last line in the /usr/local/nagiosxi/html/includes/constants.inc.php file as such:

// Scheduled backups constants
// 0 - Debug, 1 - Info, 2 - Warning, 3 - Error
define('SB_LOGLEVEL', 0);

Save, exit, and restart apache:

Code: Select all

service httpd restart

Run a local scheduled backup, then post the /usr/local/nagiosxi/var/components/scheduledbackups.log on the forum.

Manually triggered backups work, but not the scheduled:

Code: Select all

.PROCESSING COMMAND ID 32522...
PROCESS COMMAND: CMD=1117, DATA=a:2:{i:0;s:19:"nagiosxi.1563375901";i:1;s:16:"/mnt/nfs/backup/";}
CMDLINE=sudo /usr/local/nagiosxi/scripts/backup_xi.sh -n nagiosxi.1563375901 -d /mnt/nfs/backup
No entry for terminal type "unknown";
using dumb terminal settings.
Stopping nagios: .done.
Starting nagios: done.
Backing up NagiosQL...
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
Backing up Nagios Core...
tar: Removing leading `/' from member names

- any way to re-create CRONed backup jobs ?

[tgriep]

Can you post the post the /usr/local/nagiosxi/var/components/scheduledbackups.log file from the Nagios server so we can check it for errors?

Scheduled Backups in XI are ran by XI, they are not ran by a cron job.
An entry is setup in the MYSQL database which is read by the cmdsubsys.php script and submitted to run on the system.

To view the settings, run this command.

Code: Select all

echo 'select * from xi_options;' | mysql  -t -pnagiosxi nagiosxi |grep backup

[dlukinski]

Can you post the post the /usr/local/nagiosxi/var/components/scheduledbackups.log file from the Nagios server so we can check it for errors?

Scheduled Backups in XI are ran by XI, they are not ran by a cron job.
An entry is setup in the MYSQL database which is read by the cmdsubsys.php script and submitted to run on the system.

To view the settings, run this command.

Code: Select all

echo 'select * from xi_options;' | mysql  -t -pnagiosxi nagiosxi |grep backup

Log attached, command results are down below:

Code: Select all

[root@fikc-nagxiprod01 components]# echo 'select * from xi_options;' | mysql  -t -pnagiosxi nagiosxi |grep backup
ERROR 1049 (42000): Unknown database 'nagiosxi'
[root@fikc-nagxiprod01 components]#

[tgriep]

I bet your system is still using the Postgresql database to host the nagiosxi database so run this command instead.

Code: Select all

echo 'select * from xi_options;' | psql nagiosxi nagiosxi |grep backup

Run these commands as well and post the output.

Code: Select all

ls -l /store/backups
ls -l /store/backups/nagiosxi

The scheduledbackup log file has MYSQL errors so I would like to check the following file to see if the settings are correct for the MYSQL server, get the file and post it here.

Code: Select all

/usr/local/nagiosxi/html/config.inc.php

Thanks.

[dlukinski]

I bet your system is still using the Postgresql database to host the nagiosxi database so run this command instead.

Code: Select all

echo 'select * from xi_options;' | psql nagiosxi nagiosxi |grep backup

Run these commands as well and post the output.

Code: Select all

ls -l /store/backups
ls -l /store/backups/nagiosxi

The scheduledbackup log file has MYSQL errors so I would like to check the following file to see if the settings are correct for the MYSQL server, get the file and post it here.

Code: Select all

/usr/local/nagiosxi/html/config.inc.php

Thanks.

Please see the output and the fle attached

Code: Select all

[root@fikc-nagxiprod01 components]# echo 'select * from xi_options;' | psql nagiosxi nagiosxi |grep backup
       154 | bpi_backupfile                                    | /usr/local/nagiosxi/etc/components/bpi.conf.backup
       127 | scheduled_local_backups                           | a:9:{s:6:"active";i:1;s:4:"freq";s:5:"daily";s:3:"day";s:1:"0";s:5:"hours";s:2:"15";s:4:"mins";s:2:"05";s:3:"dir";s:16:"/mnt/nfs/backup/";s:5:"limit";s:2:"15";s:7:"lastrun";i:1563548701;s:7:"nextrun";i:1563635100;}
       128 | scheduledbackups_last_backup_size                 | 4323524362
       125 | scheduled_ftp_backups                             | a:15:{s:6:"active";i:0;s:4:"freq";s:5:"daily";s:3:"day";s:1:"0";s:5:"hours";s:2:"01";s:4:"mins";s:2:"11";s:6:"server";s:11:"10.96.30.30";s:4:"port";s:2:"21";s:8:"username";s:9:"nagiosbkp";s:8:"password";s:40:"1f220470659d344563mEGgi0hCT17hTvhu5qTA==";s:3:"dir";s:15:"/Production/XI/";s:7:"passive";i:1;s:6:"secure";i:0;s:5:"limit";s:2:"33";s:7:"lastrun";s:10:"1501117860";s:7:"nextrun";i:1563412260;}
       126 | scheduled_ssh_backups                             | a:17:{s:6:"active";i:0;s:4:"freq";s:5:"daily";s:3:"day";s:1:"0";s:5:"hours";s:2:"15";s:4:"mins";s:2:"00";s:6:"server";s:13:"10.102.36.163";s:4:"port";s:2:"15";s:8:"username";s:14:"nagiosxibackup";s:8:"password";s:40:"da49ca99b7119383K/79Atvh9y9WZqcLzBgK9Q==";s:9:"auth_type";s:8:"password";s:11:"pubkey_file";s:0:"";s:12:"privkey_file";s:0:"";s:16:"privkey_password";s:0:"";s:3:"dir";s:21:"/home/nagiosxibackup/";s:5:"limit";s:1:"7";s:7:"lastrun";s:10:"1563319381";s:7:"nextrun";i:1563462000;}
You have mail in /var/spool/mail/root
[root@fikc-nagxiprod01 components]# ls -l /store/backups
total 16
drwxr-xr-x 5 root   root   4096 Jul 19 07:04 mysql
drwxr-xr-x 2 nagios nagios 4096 Jul 17 14:33 nagiosxi
drwxr-xr-x 2 root   root   4096 Jun 14 15:10 nagvis
drwxr-xr-x 5 root   root   4096 Jul 19 07:00 postgresql
[root@fikc-nagxiprod01 components]# ls -l /store/backups/nagiosxi
total 40740804
-rw-r--r-- 1 nagios nagios 4170461235 Jun 12 14:51 1560350114.tar.gz
-rw-r--r-- 1 nagios nagios 4185590845 Jun 14 15:08 1560523825.tar.gz
-rw-r--r-- 1 nagios nagios 4276130617 Jul  4 14:52 1562250990.tar.gz
-rw-r--r-- 1 nagios nagios 4300876147 Jul 15 14:11 1563198890.tar.gz
-rw-r--r-- 1 nagios nagios 4316322270 Jul 17 14:33 1563372945.tar.gz
-rw-r--r-- 1 nagios nagios 4081553562 Jun  2 00:52 nagiosxi.1559431381.tar.gz
-rw-r--r-- 1 nagios nagios 4079481754 Jun  3 00:23 nagiosxi.1559517781.tar.gz
-rw-r--r-- 1 nagios nagios 4095356581 Jun  4 00:34 nagiosxi.1559604181.tar.gz
-rw-r--r-- 1 nagios nagios 4102117787 Jun  5 00:59 nagiosxi.1559690581.tar.gz
-rw-r--r-- 1 nagios nagios 4110617470 Jun  6 00:24 nagiosxi.1559776981.tar.gz
-rw-r--r-- 1 nagios nagios      11230 Jul 27  2017 tmp_xi_vars.cfg
[root@fikc-nagxiprod01 components]#

[tgriep]

The backup is setup to save the backup to this folder.

Code: Select all

/mnt/nfs/backup/

Make sure it is mounted on the server and it is still writable be the nagios user.

Lets increase the backup timeout to an hour, edit this file

Code: Select all

/usr/local/nagiosxi/html/config.inc.php

Add this to the bottom of that file

Code: Select all

$cfg['backup_timeout'] = 3600;

Save the change and restart Apache and Nagios

Code: Select all

service httpd restart
service nagios restart

Wait until the next tine it runs and see if the backup completes.

[dlukinski]

The backup is setup to save the backup to this folder.

Code: Select all

/mnt/nfs/backup/

Make sure it is mounted on the server and it is still writable be the nagios user.

Lets increase the backup timeout to an hour, edit this file

Code: Select all

/usr/local/nagiosxi/html/config.inc.php

Add this to the bottom of that file

Code: Select all

$cfg['backup_timeout'] = 3600;

Save the change and restart Apache and Nagios

Code: Select all

service httpd restart
service nagios restart

Wait until the next tine it runs and see if the backup completes.

Something is wrong:

- directory is writeable (manually initiated backups work)
- temp folder is created for scheduled backups with large files
- some backups from yesterday and a day before exist
- NO backups are listed under the "Local Backup Archive" aside from those initiated manually

Same errors for the scheduled backups as before

[tgriep]

For this

- temp folder is created for scheduled backups with large files

What is the name of the temp folder that is created and what files are in them?

For this

- some backups from yesterday and a day before exist

What backups are you talking about? What are the names and folders are the files in?

For this

- NO backups are listed under the "Local Backup Archive" aside from those initiated manually

Those files are for the /store/backups/nagiosxi/. The scheduled backup file are in this folder /mnt/nfs/backup/ and will not be displayed.