Local Scheduled Backup failure

[NetNeutron]

Good Morning,
We seem to be having an issue with our Local Scheduled Backups.
I had them running Daily at 00:05 at Location /store/backups/nagiosxi/
I did a Test permissions which returned -> The directory exists and is writeable.

I purged all of the unzipped folders from the /store/backups/nagiosxi/ directory.
The /store/backups/nagiosxi has 388GB available to it.

I set a backup for 5 minutes into the future. The /usr/local/nagiosxi/var/components/scheduledbackups.log shows

Code: Select all

01-12-2017 07:55:01 DEBUG: Running scheduled local backup ...
01-12-2017 07:55:01 INFO: Creating a local backup: nagiosxi.1484225701
01-12-2017 07:55:01 DEBUG: Sending create local backup command to CmdSubsystem

The tail -f /usr/local/nagiosxi/var/cmdsubsys.log periodically showed

Code: Select all

PROCESSED 0 COMMANDS
SSL connect error
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated

I do not see the folder /store/backups/nagiosxi/1484225701 or a gz file 1484225701.tar.gz

Code: Select all

# sestatus
SELinux status:                 disabled
# ls -la /usr/local/nagiosxi/scripts/backup_xi.sh
-rwxrwx--- 1 nagios nagios 5672 Dec 13 09:25 /usr/local/nagiosxi/scripts/backup_xi.sh
# echo $0
-bash
# awk -F':' '{print $1 " " $7}' /etc/passwd | grep root
root /bin/bash

System:

• Nagios XI Version : 5.3.3
  nagios1 2.6.32-642.6.2.el6.x86_64 x86_64
  Red Hat Enterprise Linux Server release 6.8 (Santiago)
  Gnome is not installed

Scheduled Backups

• Provides the ability to schedule full Nagios XI backups.
  1.1.3 09/15/2016 Nagios Enterprises, LLC

Please advise...

[tgriep]

If your server has a lot of data, it could take many minutes to run the backup so the messages could be in the log file now but were not there when you checked.
Can you attach the log file so we can review it?

Code: Select all

/usr/local/nagiosxi/var/components/scheduledbackups.log

Run the following command as root on the XI server and post the output.

Code: Select all

ps -ef --cols=300

Thanks.

[NetNeutron]

# cat /usr/local/nagiosxi/var/components/scheduledbackups.log

Code: Select all

01-01-2017 00:00:01 DEBUG: Running scheduled local backup ...
01-01-2017 00:00:01 INFO: Creating a local backup: nagiosxi.1483246801
01-01-2017 00:00:01 DEBUG: Sending create local backup command to CmdSubsystem
01-01-2017 01:00:02 DEBUG: Running scheduled SSH backup ...
01-01-2017 01:00:02 DEBUG: Sending create local backup command to CmdSubsystem
01-01-2017 01:06:50 ERROR: Scheduled SSH Backup Failed: Could not authenticate on SSH server
01-08-2017 00:00:01 DEBUG: Running scheduled local backup ...
01-08-2017 00:00:01 INFO: Creating a local backup: nagiosxi.1483851601
01-08-2017 00:00:01 DEBUG: Sending create local backup command to CmdSubsystem
01-08-2017 01:00:01 DEBUG: Running scheduled SSH backup ...
01-08-2017 01:00:01 DEBUG: Sending create local backup command to CmdSubsystem
01-08-2017 01:22:17 ERROR: Scheduled SSH Backup Failed: Could not authenticate on SSH server
01-09-2017 11:00:02 DEBUG: Running scheduled SSH backup ...
01-09-2017 11:00:02 DEBUG: Sending create local backup command to CmdSubsystem
01-09-2017 11:23:10 ERROR: Scheduled SSH Backup Failed: Local file to transfer does not exist
01-12-2017 00:05:01 DEBUG: Running scheduled local backup ...
01-12-2017 00:05:01 INFO: Creating a local backup: nagiosxi.1484197501
01-12-2017 00:05:01 DEBUG: Sending create local backup command to CmdSubsystem
01-12-2017 07:55:01 DEBUG: Running scheduled local backup ...
01-12-2017 07:55:01 INFO: Creating a local backup: nagiosxi.1484225701
01-12-2017 07:55:01 DEBUG: Sending create local backup command to CmdSubsystem

[NetNeutron]

# ps -ef --cols=300

Code: Select all

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 Jan09 ?        00:00:01 /sbin/init
root         2     0  0 Jan09 ?        00:00:00 [kthreadd]
root         3     2  0 Jan09 ?        00:00:04 [migration/0]
root         4     2  0 Jan09 ?        00:00:01 [ksoftirqd/0]
root         5     2  0 Jan09 ?        00:00:00 [stopper/0]
root         6     2  0 Jan09 ?        00:00:00 [watchdog/0]
root         7     2  0 Jan09 ?        00:00:15 [migration/1]
root         8     2  0 Jan09 ?        00:00:00 [stopper/1]
root         9     2  0 Jan09 ?        00:00:01 [ksoftirqd/1]
root        10     2  0 Jan09 ?        00:00:00 [watchdog/1]
root        11     2  0 Jan09 ?        00:00:06 [migration/2]
root        12     2  0 Jan09 ?        00:00:00 [stopper/2]
root        13     2  0 Jan09 ?        00:00:00 [ksoftirqd/2]
root        14     2  0 Jan09 ?        00:00:00 [watchdog/2]
root        15     2  0 Jan09 ?        00:00:04 [migration/3]
root        16     2  0 Jan09 ?        00:00:00 [stopper/3]
root        17     2  0 Jan09 ?        00:00:01 [ksoftirqd/3]
root        18     2  0 Jan09 ?        00:00:00 [watchdog/3]
root        19     2  0 Jan09 ?        00:00:21 [migration/4]
root        20     2  0 Jan09 ?        00:00:00 [stopper/4]
root        21     2  0 Jan09 ?        00:00:01 [ksoftirqd/4]
root        22     2  0 Jan09 ?        00:00:00 [watchdog/4]
root        23     2  0 Jan09 ?        00:00:06 [migration/5]
root        24     2  0 Jan09 ?        00:00:00 [stopper/5]
root        25     2  0 Jan09 ?        00:00:00 [ksoftirqd/5]
root        26     2  0 Jan09 ?        00:00:00 [watchdog/5]
root        27     2  0 Jan09 ?        00:00:08 [events/0]
root        28     2  0 Jan09 ?        00:00:06 [events/1]
root        29     2  0 Jan09 ?        00:00:07 [events/2]
root        30     2  0 Jan09 ?        00:00:06 [events/3]
root        31     2  0 Jan09 ?        00:00:07 [events/4]
root        32     2  0 Jan09 ?        00:00:19 [events/5]
root        33     2  0 Jan09 ?        00:00:00 [events/0]
root        34     2  0 Jan09 ?        00:00:00 [events/1]
root        35     2  0 Jan09 ?        00:00:00 [events/2]
root        36     2  0 Jan09 ?        00:00:00 [events/3]
root        37     2  0 Jan09 ?        00:00:00 [events/4]
root        38     2  0 Jan09 ?        00:00:00 [events/5]
root        39     2  0 Jan09 ?        00:00:00 [events_long/0]
root        40     2  0 Jan09 ?        00:00:00 [events_long/1]
root        41     2  0 Jan09 ?        00:00:00 [events_long/2]
root        42     2  0 Jan09 ?        00:00:00 [events_long/3]
root        43     2  0 Jan09 ?        00:00:00 [events_long/4]
root        44     2  0 Jan09 ?        00:00:00 [events_long/5]
root        45     2  0 Jan09 ?        00:00:00 [events_power_ef]
root        46     2  0 Jan09 ?        00:00:00 [events_power_ef]
root        47     2  0 Jan09 ?        00:00:00 [events_power_ef]
root        48     2  0 Jan09 ?        00:00:00 [events_power_ef]
root        49     2  0 Jan09 ?        00:00:00 [events_power_ef]
root        50     2  0 Jan09 ?        00:00:00 [events_power_ef]
root        51     2  0 Jan09 ?        00:00:00 [cgroup]
root        52     2  0 Jan09 ?        00:00:00 [khelper]
root        53     2  0 Jan09 ?        00:00:00 [netns]
root        54     2  0 Jan09 ?        00:00:00 [async/mgr]
root        55     2  0 Jan09 ?        00:00:00 [pm]
root        56     2  0 Jan09 ?        00:00:00 [sync_supers]
root        57     2  0 Jan09 ?        00:00:00 [bdi-default]
root        58     2  0 Jan09 ?        00:00:00 [kintegrityd/0]
root        59     2  0 Jan09 ?        00:00:00 [kintegrityd/1]
root        60     2  0 Jan09 ?        00:00:00 [kintegrityd/2]
root        61     2  0 Jan09 ?        00:00:00 [kintegrityd/3]
root        62     2  0 Jan09 ?        00:00:00 [kintegrityd/4]
root        63     2  0 Jan09 ?        00:00:00 [kintegrityd/5]
root        64     2  0 Jan09 ?        00:00:02 [kblockd/0]
root        65     2  0 Jan09 ?        00:00:02 [kblockd/1]
root        66     2  0 Jan09 ?        00:00:03 [kblockd/2]
root        67     2  0 Jan09 ?        00:00:05 [kblockd/3]
root        68     2  0 Jan09 ?        00:00:01 [kblockd/4]
root        69     2  0 Jan09 ?        00:00:00 [kblockd/5]
root        70     2  0 Jan09 ?        00:00:00 [kacpid]
root        71     2  0 Jan09 ?        00:00:00 [kacpi_notify]
root        72     2  0 Jan09 ?        00:00:00 [kacpi_hotplug]
root        73     2  0 Jan09 ?        00:00:00 [ata_aux]
root        74     2  0 Jan09 ?        00:00:00 [ata_sff/0]
root        75     2  0 Jan09 ?        00:00:00 [ata_sff/1]
root        76     2  0 Jan09 ?        00:00:00 [ata_sff/2]
root        77     2  0 Jan09 ?        00:00:00 [ata_sff/3]
root        78     2  0 Jan09 ?        00:00:00 [ata_sff/4]
root        79     2  0 Jan09 ?        00:00:00 [ata_sff/5]
root        80     2  0 Jan09 ?        00:00:00 [ksuspend_usbd]
root        81     2  0 Jan09 ?        00:00:00 [khubd]
root        82     2  0 Jan09 ?        00:00:00 [kseriod]
root        83     2  0 Jan09 ?        00:00:00 [md/0]
root        84     2  0 Jan09 ?        00:00:00 [md/1]
root        85     2  0 Jan09 ?        00:00:00 [md/2]
root        86     2  0 Jan09 ?        00:00:00 [md/3]
root        87     2  0 Jan09 ?        00:00:00 [md/4]
root        88     2  0 Jan09 ?        00:00:00 [md/5]
root        89     2  0 Jan09 ?        00:00:00 [md_misc/0]
root        90     2  0 Jan09 ?        00:00:00 [md_misc/1]
root        91     2  0 Jan09 ?        00:00:00 [md_misc/2]
root        92     2  0 Jan09 ?        00:00:00 [md_misc/3]
root        93     2  0 Jan09 ?        00:00:00 [md_misc/4]
root        94     2  0 Jan09 ?        00:00:00 [md_misc/5]
root        95     2  0 Jan09 ?        00:00:00 [linkwatch]
root        98     2  0 Jan09 ?        00:00:00 [khungtaskd]
root        99     2  0 Jan09 ?        00:00:03 [kswapd0]
root       100     2  0 Jan09 ?        00:00:00 [ksmd]
root       101     2  0 Jan09 ?        00:00:30 [khugepaged]
root       102     2  0 Jan09 ?        00:00:00 [aio/0]
root       103     2  0 Jan09 ?        00:00:00 [aio/1]
root       104     2  0 Jan09 ?        00:00:00 [aio/2]
root       105     2  0 Jan09 ?        00:00:00 [aio/3]
root       106     2  0 Jan09 ?        00:00:00 [aio/4]
root       107     2  0 Jan09 ?        00:00:00 [aio/5]
root       108     2  0 Jan09 ?        00:00:00 [crypto/0]
root       109     2  0 Jan09 ?        00:00:00 [crypto/1]
root       110     2  0 Jan09 ?        00:00:00 [crypto/2]
root       111     2  0 Jan09 ?        00:00:00 [crypto/3]
root       112     2  0 Jan09 ?        00:00:00 [crypto/4]
root       113     2  0 Jan09 ?        00:00:00 [crypto/5]
root       120     2  0 Jan09 ?        00:00:00 [kthrotld/0]
root       121     2  0 Jan09 ?        00:00:00 [kthrotld/1]
root       122     2  0 Jan09 ?        00:00:00 [kthrotld/2]
root       123     2  0 Jan09 ?        00:00:00 [kthrotld/3]
root       124     2  0 Jan09 ?        00:00:00 [kthrotld/4]
root       125     2  0 Jan09 ?        00:00:00 [kthrotld/5]
root       126     2  0 Jan09 ?        00:00:00 [pciehpd]
root       128     2  0 Jan09 ?        00:00:00 [kpsmoused]
root       129     2  0 Jan09 ?        00:00:00 [usbhid_resumer]
root       130     2  0 Jan09 ?        00:00:00 [deferwq]
root       163     2  0 Jan09 ?        00:00:00 [kdmremove]
root       164     2  0 Jan09 ?        00:00:00 [kstriped]
root       197     2  0 Jan09 ?        00:00:00 [ttm_swap]
root       375     2  0 Jan09 ?        00:00:00 [scsi_eh_0]
root       377     2  0 Jan09 ?        00:00:00 [scsi_eh_1]
root       389     2  0 Jan09 ?        00:00:00 [scsi_eh_2]
root       390     2  0 Jan09 ?        00:00:00 [vmw_pvscsi_wq_2]
root       546     2  0 Jan09 ?        00:00:00 [kdmflush]
root       548     2  0 Jan09 ?        00:00:00 [kdmflush]
root       566     2  0 Jan09 ?        00:00:10 [jbd2/dm-0-8]
root       567     2  0 Jan09 ?        00:00:00 [ext4-dio-unwrit]
root       665     1  0 Jan09 ?        00:00:00 /sbin/udevd -d
root       833     2  0 Jan09 ?        00:00:02 [vmmemctl]
root      1089     2  0 Jan09 ?        00:00:00 [kdmflush]
root      1091     2  0 Jan09 ?        00:00:00 [kdmflush]
root      1094     2  0 Jan09 ?        00:00:00 [kdmflush]
root      1095   665  0 Jan09 ?        00:00:00 /sbin/udevd -d
root      1097     2  0 Jan09 ?        00:00:00 [kdmflush]
root      1098   665  0 Jan09 ?        00:00:00 /sbin/udevd -d
root      1138     2  0 Jan09 ?        00:00:37 [flush-253:4]
root      1139     2  0 Jan09 ?        00:00:00 [flush-253:5]
root      1151     2  0 Jan09 ?        00:00:05 [flush-253:2]
root      1152     2  0 Jan09 ?        00:00:06 [flush-253:3]
root      1181     2  0 Jan09 ?        00:00:00 [kjournald]
root      1182     2  0 Jan09 ?        00:00:01 [jbd2/dm-5-8]
root      1183     2  0 Jan09 ?        00:00:00 [ext4-dio-unwrit]
root      1184     2  0 Jan09 ?        00:00:12 [jbd2/dm-4-8]
root      1185     2  0 Jan09 ?        00:00:00 [ext4-dio-unwrit]
root      1186     2  0 Jan09 ?        00:00:05 [jbd2/dm-3-8]
root      1187     2  0 Jan09 ?        00:00:00 [ext4-dio-unwrit]
root      1188     2  0 Jan09 ?        00:00:09 [jbd2/dm-2-8]
root      1189     2  0 Jan09 ?        00:00:00 [ext4-dio-unwrit]
root      1288     2  0 Jan09 ?        00:00:03 [kauditd]
root      1484     2  0 Jan09 ?        00:00:07 [flush-253:0]
root      1643     1  0 Jan09 ?        00:00:13 auditd
root      1665     1  0 Jan09 ?        00:00:14 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5
root      1699     1  0 Jan09 ?        00:00:07 irqbalance --pid=/var/run/irqbalance.pid
rpc       1717     1  0 Jan09 ?        00:00:00 rpcbind
root      1733     1  0 Jan09 ?        00:00:07 /usr/sbin/sssd -f -D
root      1735  1733  0 Jan09 ?        00:00:16 /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --debug-to-files
root      1736  1733  0 Jan09 ?        00:00:20 /usr/libexec/sssd/sssd_pam --uid 0 --gid 0 --debug-to-files
rpcuser   1757     1  0 Jan09 ?        00:00:00 rpc.statd
dbus      1792     1  0 Jan09 ?        00:00:00 dbus-daemon --system
root      1830     1  0 Jan09 ?        00:00:00 /usr/sbin/acpid
root      1953     1  0 Jan09 ?        00:01:10 /usr/sbin/snmpd -LS0-6d -Lf /dev/null -p /var/run/snmpd.pid
root      1970     1  0 Jan09 ?        00:00:00 /usr/sbin/sshd
root      1981     1  0 Jan09 ?        00:00:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
ntp       1992     1  0 Jan09 ?        00:00:01 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
root      2029     1  0 Jan09 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld.pid --basedir=/usr --user=mysql
mysql     2131  2029  5 Jan09 ?        04:05:20 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
root      2168     1  0 Jan09 ?        00:00:04 sendmail: accepting connections
smmsp     2177     1  0 Jan09 ?        00:00:00 sendmail: Queue runner@00:15:00 for /var/spool/clientmqueue
root      2205     1  0 Jan09 ?        00:00:00 /usr/sbin/abrtd
root      2215     1  0 Jan09 ?        00:00:03 abrt-dump-oops -d /var/spool/abrt -rwx /var/log/messages
root      2226     1  0 Jan09 ?        00:00:06 /usr/sbin/httpd
root      2238     1  0 Jan09 ?        00:00:06 crond
nagios    2249     1  0 Jan09 ?        00:00:11 /usr/local/nagios/bin/npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg
root      2267     1  0 Jan09 ?        00:00:00 /usr/sbin/atd
root      2273     1  0 Jan09 ?        00:00:31 /usr/bin/dsmc sched
root      2278     1  0 Jan09 ?        00:00:06 /usr/bin/dsmc sched -servername=aixtsm9n_arc
nagios    2294     1  0 Jan09 ?        00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
root      2308     1  0 Jan09 ?        00:00:00 rhnsd
root      2319     1  0 Jan09 ?        00:00:00 /usr/bin/rhsmcertd
ajaxterm  2331     1  0 Jan09 ?        00:00:49 python /usr/share/ajaxterm/ajaxterm.py --daemon --port=8022 --uid=ajaxterm
apache    2333  2226  1 10:51 ?        00:00:21 /usr/sbin/httpd
apache    2334  2226  1 10:51 ?        00:00:21 /usr/sbin/httpd
apache    2339  2226  1 10:51 ?        00:00:32 /usr/sbin/httpd
apache    2358  2226  1 10:51 ?        00:00:29 /usr/sbin/httpd
apache    2365  2226  1 10:51 ?        00:00:29 /usr/sbin/httpd
apache    2366  2226  1 10:51 ?        00:00:28 /usr/sbin/httpd
apache    2372  2226  1 10:51 ?        00:00:22 /usr/sbin/httpd
apache    2386  2226  1 10:51 ?        00:00:27 /usr/sbin/httpd
apache    2390  2226  1 10:51 ?        00:00:23 /usr/sbin/httpd
apache    2391  2226  1 10:51 ?        00:00:20 /usr/sbin/httpd
root      2406     1  0 Jan09 tty1     00:00:00 /sbin/mingetty /dev/tty1
root      2408     1  0 Jan09 tty2     00:00:00 /sbin/mingetty /dev/tty2
root      2410     1  0 Jan09 tty3     00:00:00 /sbin/mingetty /dev/tty3
root      2412     1  0 Jan09 tty4     00:00:00 /sbin/mingetty /dev/tty4
root      2414     1  0 Jan09 tty5     00:00:00 /sbin/mingetty /dev/tty5
root      2416     1  0 Jan09 tty6     00:00:00 /sbin/mingetty /dev/tty6
root      2647  1733  0 Jan11 ?        00:01:14 /usr/libexec/sssd/sssd_be --domain LDAP --uid 0 --gid 0 --debug-to-files
apache    3727  2226  1 10:51 ?        00:00:29 /usr/sbin/httpd
apache    3996  2226  1 10:51 ?        00:00:26 /usr/sbin/httpd
apache    4420  2226  1 10:51 ?        00:00:23 /usr/sbin/httpd
apache    4582  2226  1 10:51 ?        00:00:20 /usr/sbin/httpd
root      5109  1970  0 07:25 ?        00:00:00 sshd: gja6 [priv]
gja6      5111  5109  0 07:25 ?        00:00:00 sshd: gja6@pts/0
gja6      5112  5111  0 07:25 pts/0    00:00:00 -PHSshell
root      5131  5112  0 07:25 pts/0    00:00:00 sudo su -
root      5134  5131  0 07:25 pts/0    00:00:00 su -
root      5135  5134  0 07:25 pts/0    00:00:00 -bash
apache   11065  2226  1 10:10 ?        00:01:00 /usr/sbin/httpd
apache   11066  2226  1 10:10 ?        00:00:56 /usr/sbin/httpd
apache   11083  2226  1 10:10 ?        00:01:00 /usr/sbin/httpd
root     11663  2238  0 11:24 ?        00:00:00 CROND
root     11664  2238  0 11:24 ?        00:00:00 CROND
root     11665  2238  0 11:24 ?        00:00:00 CROND
root     11666  2238  0 11:24 ?        00:00:00 CROND
root     11667  2238  0 11:24 ?        00:00:00 CROND
root     11668  2238  0 11:24 ?        00:00:00 CROND
nagios   11671 11664  0 11:24 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
nagios   11672 11668  0 11:24 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
nagios   11674 11665  0 11:24 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php > /usr/local/nagiosxi/var/event_handler.log 2>&1
nagios   11675 11667  0 11:24 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
nagios   11676 11666  0 11:24 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
nagios   11677 11663  0 11:24 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
nagios   11680 11674  0 11:24 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php
nagios   11681 11671  0 11:24 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php
nagios   11682 11675  0 11:24 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php
nagios   11683 11676  0 11:24 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
nagios   11684 11672  0 11:24 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php
nagios   11685 11677  0 11:24 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
root     13281  5135  0 11:24 pts/0    00:00:00 ps -ef --cols=300
nagios   15166     1  1 Jan11 ?        00:18:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   15168 15166  0 Jan11 ?        00:01:18 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   15169 15166  0 Jan11 ?        00:01:36 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   15170 15166  0 Jan11 ?        00:01:40 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   15171 15166  0 Jan11 ?        00:01:15 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   15172 15166  0 Jan11 ?        00:02:44 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   15173 15166  0 Jan11 ?        00:01:58 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   15174 15166  0 Jan11 ?        00:02:13 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   15176 15166  0 Jan11 ?        00:01:16 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   15177 15166  0 Jan11 ?        00:01:22 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   15180  2294  0 Jan11 ?        00:01:43 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   15181 15180  5 Jan11 ?        01:28:06 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   15189 15166  0 Jan11 ?        00:00:02 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
apache   16024  2226  1 10:57 ?        00:00:18 /usr/sbin/httpd
apache   16025  2226  1 10:57 ?        00:00:22 /usr/sbin/httpd
apache   16026  2226  1 10:57 ?        00:00:22 /usr/sbin/httpd

[tgriep]

Can you login to the XI server as root, run the following and post the output?

Code: Select all

echo 'select * from xi_options;' | mysql  -t -pnagiosxi nagiosxi |grep backup
ls -l /store/backups/nagiosxi

[NetNeutron]

# echo 'select * from xi_options;' | mysql -t -pnagiosxi nagiosxi |grep backup

Code: Select all

|        47 | scheduled_ftp_backups                             | a:15:{s:6:"active";i:0;s:4:"freq";s:6:"weekly";s:3:"day";s:1:"0";s:5:"hours";s:2:"00";s:4:"mins";s:2:"00";s:6:"server";s:0:"";s:4:"port";s:1:"0";s:8:"username";s:0:"";s:8:"password";s:0:"";s:3:"dir";s:1:"/";s:7:"passive";i:0;s:6:"secure";i:0;s:5:"limit";s:1:"7";s:7:"lastrun";s:1:"0";s:7:"nextrun";i:1484456400;}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
|        48 | scheduled_ssh_backups                             | a:13:{s:6:"active";i:1;s:4:"freq";s:6:"weekly";s:3:"day";s:1:"0";s:5:"hours";s:2:"01";s:4:"mins";s:2:"00";s:6:"server";s:31:<REDACTED>";s:4:"port";s:2:"22";s:8:"username";s:4:"<REDACTED>";s:8:"password";s:8:"<REDACTED>";s:3:"dir";s:13:"/opt/nagios1/";s:5:"limit";s:1:"7";s:7:"lastrun";s:10:"1483977600";s:7:"nextrun";i:1484460000;}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
|        49 | scheduled_local_backups                           | a:9:{s:6:"active";i:1;s:4:"freq";s:5:"daily";s:3:"day";s:1:"0";s:5:"hours";s:2:"07";s:4:"mins";s:2:"55";s:3:"dir";s:24:"/store/backups/nagiosxi/";s:5:"limit";s:2:"30";s:7:"lastrun";i:1484225700;s:7:"nextrun";i:1484312100;}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|        50 | scheduledbackups_last_backup_size                 | 1659032530                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
|       120 | bpi_backupfile                                    | /usr/local/nagiosxi/etc/components/bpi.conf.backup                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|        50 | scheduledbackups_last_backup_size                 | 1659032530                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
|       120 | bpi_backupfile                                    | /usr/local/nagiosxi/etc/components/bpi.conf.backup                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

|

[tgriep]

You missed this command from my earlier post, can you run it and post the output?

Code: Select all

ls -l /store/backups/nagiosxi

The server thinks it ran and the size of of the backup is 1.6 Gig so it may be timing out.
Try increasing the timeout for the backup be editing this file

Code: Select all

/usr/local/nagiosxi/html/config.inc.php

Add this to the bottom

Code: Select all

$cfg['backup_timeout'] = 3600;

Save the file and restart nagios and the apache process by running

Code: Select all

service nagios restart
service apache restart

Let us know if this fixes the issue.

[NetNeutron]

# ls -l /store/backups/nagiosxi

Code: Select all

total 20558420
-rw-r----- 1 nagios nagios  136210504 Aug 23 13:10 1471972193.tar.gz
-rw-r--r-- 1 nagios nagios 1133638551 Sep 23 12:06 1474646446.tar.gz
-rw-r--r-- 1 nagios nagios 1768204181 Oct  7 07:11 1475838065.tar.gz
-rw-r--r-- 1 nagios nagios 1630203329 Dec 12 17:09 1481580203.tar.gz
-rw-r----- 1 nagios nagios 1586660129 Dec 14 14:07 1481742022.tar.gz
-rw-r--r-- 1 nagios nagios 1586688931 Dec 14 14:08 1481742101.tar.gz
-rw-r----- 1 nagios nagios 1667278712 Dec 22 10:50 1482421456.tar.gz
-rw-r--r-- 1 nagios nagios 1657131942 Dec 30 14:28 1483125709.tar.gz
-rw-r----- 1 nagios nagios 1657467964 Jan  9 09:14 1483970323.tar.gz
-rw-r----- 1 nagios nagios 1659220807 Jan 11 08:40 1484140657.tar.gz
-rw-r----- 1 nagios nagios 1634826311 Dec 12 22:54 autoupgrade_backup.1481600859.tar.gz
-rw-r----- 1 nagios nagios 1633904347 Dec 13 09:24 autoupgrade_backup.1481638703.tar.gz
-rw-r----- 1 nagios nagios 1641328295 Dec 18 00:06 nagiosxi.1482037201.tar.gz
-rw-r----- 1 nagios nagios 1658970697 Jan  1 00:06 nagiosxi.1483246801.tar.gz

[NetNeutron]

I added

Code: Select all

$cfg['backup_timeout'] = 3600;

I then did
# service nagios restart

Code: Select all

Running configuration check...
Stopping nagios:. done.
Starting nagios: done.

# service httpd restart

Code: Select all

Stopping httpd:                                            [  OK  ]
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 172.18.17.31 for ServerName
                                                           [  OK  ]

Okay I have scheduled a local backup for a few minutes in the future.
I am doing a tail -f /usr/local/nagiosxi/var/cmdsubsys.log

This showed up in scheduledbackups.log

Code: Select all

01-12-2017 15:20:02 DEBUG: Running scheduled local backup ...
01-12-2017 15:20:02 INFO: Creating a local backup: nagiosxi.1484252402
01-12-2017 15:20:02 DEBUG: Sending create local backup command to CmdSubsystem

So far nothing in cmdsubsys.log
#tail -f /usr/local/nagiosxi/var/cmdsubsys.log

Code: Select all

.....................................................tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
.
PROCESSED 0 COMMANDS
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
......................
PROCESSED 0 COMMANDS
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
.
PROCESSED 0 COMMANDS
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
............................................................tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
.
PROCESSED 0 COMMANDS

Nothing for Jan 12 has shown up in /store/backups/nagiosxi/

[tgriep]

Lets check the permissions of the PHP scripts that run the backup.
Run this command as root and post the output.

Code: Select all

ls -lR /usr/local/nagiosxi/html/includes/components/scheduledbackups/

Then run this tail command on the XI server

Code: Select all

tail -f /var/log/httpd/*_log

Schedule a backup to happen a few minutes in the future and post the output from that tail command.