Offloading MySQL & xi-sys.cfg

[andyb4u]

Hi,

I've offloaded MySQL as per the steps in https://assets.nagios.com/downloads/nag ... Server.pdf.

Does the /usr/local/nagiosxi/var/xi-sys.cfg file have to be manually updated to reflect the changes in database connection settings in order for the backup_xi.sh to work? If so is there a list of variables that need to be changed?

Regards,
Andy

[dwhitfield]

Generally, the answer would be no. Did you make manual changes to your usr/local/nagiosxi/var/xi-sys.cfg? Can you PM that file or just attach it to your post?

[andyb4u]

No we didn't make any change to xi-sys.cfg file. I guess the reason I'm asking this is that we are having problems with scheduled local and scheduled SSH backups. I'm monitoring the cmdsubsys.log file and it isn't firing the backup commands as per the set schedule. If I run backup_xi.sh from command line it is completes OK.

[dwhitfield]

This could be several things. Can you schedule a backup and then send a tail of /var/log/cron?

Also, can you PM me your Profile? You can download it by going to Admin > System Config > System Profile and click the ***Download Profile*** button towards the top. If for whatever reason you *cannot* download the profile, please put the output of View System Info (5.3.4+, Show Profile if older) in the thread (that will at least get us some info). This will give us access to many of the logs we would otherwise ask for individually. If security is a concern, you can unzip the profile take out what you like, and then zip it up again. We may end up needing something you remove, but we can ask for that specifically.

You can also generate a profile manually using the script at /usr/local/nagiosxi/html/includes/components/profile/getprofile.sh

That should generate a profile in /usr/local/nagiosxi/var/components/ which you can get off the server with an application such as FileZilla.

After you PM the profile, please update this thread. Updating this thread is the only way for it to show back up on our dashboard.

If you get an error that PROFILE BUILD FAILED, please see https://support.nagios.com/kb/article.p ... ategory=44

UPDATE: Profile shared with techs

[andyb4u]

Apologies for the late reply. Our scheduled SSH backup failed to run this morning (set to run daily at 05:30). To add to my last comment about not seeing the command being fired in the cmdsubsys.log file, when I've been monitoring in the past when it failed it never created any local files in the /store/backups/nagiosxi directory. When I was monitoring a successful scheduled ssh backup I could see it building up the folder structure of the backup in this directory, creating the tar file and then removing it after the ssh transfer.

I've PM'd you the system profile and below is an extract of the /var/log/cron file around this time:

Dec 13 05:28:01 PR-ITA-NAGV01 CROND[9430]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Dec 13 05:28:01 PR-ITA-NAGV01 CROND[9432]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Dec 13 05:28:01 PR-ITA-NAGV01 CROND[9435]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Dec 13 05:28:01 PR-ITA-NAGV01 CROND[9434]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Dec 13 05:28:01 PR-ITA-NAGV01 CROND[9436]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Dec 13 05:28:01 PR-ITA-NAGV01 CROND[9444]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Dec 13 05:28:01 PR-ITA-NAGV01 CROND[9445]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Dec 13 05:28:01 PR-ITA-NAGV01 CROND[9450]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Dec 13 05:28:01 PR-ITA-NAGV01 CROND[9452]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Dec 13 05:29:01 PR-ITA-NAGV01 CROND[12083]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Dec 13 05:29:01 PR-ITA-NAGV01 CROND[12085]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Dec 13 05:29:01 PR-ITA-NAGV01 CROND[12086]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Dec 13 05:29:01 PR-ITA-NAGV01 CROND[12088]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Dec 13 05:29:01 PR-ITA-NAGV01 CROND[12084]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Dec 13 05:29:01 PR-ITA-NAGV01 CROND[12094]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Dec 13 05:29:01 PR-ITA-NAGV01 CROND[12098]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Dec 13 05:29:01 PR-ITA-NAGV01 CROND[12106]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Dec 13 05:29:01 PR-ITA-NAGV01 CROND[12104]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Dec 13 05:33:01 PR-ITA-NAGV01 CROND[22481]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Dec 13 05:33:01 PR-ITA-NAGV01 CROND[22483]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Dec 13 05:33:01 PR-ITA-NAGV01 CROND[22485]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Dec 13 05:33:01 PR-ITA-NAGV01 CROND[22488]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Dec 13 05:33:01 PR-ITA-NAGV01 CROND[22487]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Dec 13 05:33:01 PR-ITA-NAGV01 CROND[22492]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Dec 13 05:33:01 PR-ITA-NAGV01 CROND[22497]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Dec 13 05:33:01 PR-ITA-NAGV01 CROND[22498]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Dec 13 05:33:01 PR-ITA-NAGV01 CROND[22499]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)

[tgriep]

Can you look on the scheduled backup log file to see if there are any errors and post them here?

Code: Select all

/usr/local/nagiosxi/var/components/scheduledbackups.log

Also, run the following and post the output to see if the scheduled backup information is being added to the MYSQL server's database.

Code: Select all

echo 'select * from xi_options;' | mysql  -t -u nagiosxi -pnagiosxi --database nagiosxi -h 10.27.80.218 |grep backup

[andyb4u]

Hi,

Please find attached the scheduled backup log file (the log level is set to debug for scheduled backups) and the results of the SQL query (amended to use the correct password for our nagiosxi user).

Regards,
Andy

[tgriep]

Thanks for the files, the settings look good to me.
It could be that the permissions of the folders the backup creates the local backup is not correct. The nagios user account needs to write to the folder or that the account is expired on the system.
Can you run the following commands and post the output?

Code: Select all

ls -l  /
ls -l  /store/backups/
ls -l  /store/backups/nagiosxi/
df -h
chage -l nagios

[andyb4u]

Here you go. Local backups work but any scheduled backup will fail intermittently. Both the test ssh connection and test ssh transfer complete successfully.

Commands 1.JPG

Commands 2.JPG

[tgriep]

If it is intermittent then it could be the timeout may not be large enough.
It looks like it was changed on your system but let's increase it again.

Edit the /usr/local/nagiosxi/html/config.inc.php and on the bottom of the file, change the following from

Code: Select all

$cfg['backup_timeout'] = 5400;

to

Code: Select all

$cfg['backup_timeout'] = 7200;

Save the file and restart apache by running
service httpd restart

Then wait for the next run to happen and see if it works.

Also, these 2 commands did not run when you typed them in.
Can you run them again and post the output?

Code: Select all

ls -l /store/backups/
ls -l /store/backups/nagiosxi/