Nagios Support Forum

Posted: **Thu Feb 27, 2020 6:28 pm**

lmiltchev wrote:
I have tried on gentoo and ubunto box it works from command line using SCP and SFTP but it doesn't work from Web. gives generic error of permission and directory.
Can you show us a successful log in from the CLI?
[img]
Commandline-Auth-scp-trans.PNG
[/img]

Also, go to Admin > System Backups > Scheduled Backups > SSH tab, and show us a screenshot of the page.
[img]
Gui-Test-Conn.PNG
[/img]
[img]
Gui-SCP-Trans.PNG
[/img]

Let's do some more troubleshooting.
1. Make sure you don't have any folders (unfinished backups) under the /store/backups/nagiosxi directory on the Nagios XI server. If you do, remove them:
Code: Select all
cd /store/backups/nagiosxi
rm -rf <directory>
checked,
2. Make sure that the permissions on the scheduledbackups.log are correct (user and group can write to the log):
Code: Select all
ls -la /usr/local/nagiosxi/var/components/scheduledbackups.log
Done

3. Enable debugging for scheduled backups by following the steps, outlined in the KB article below:
https://support.nagios.com/kb/article/n ... l-578.html

Example:
Code: Select all
define('SB_LOGLEVEL', 0);
[/b]

Done
4. Increase the backup timeout in the "/usr/local/nagiosxi/html/config.inc.php" file, but adding this line (if you don't already have it):
Code: Select all
$cfg['backup_timeout'] = <seconds>;
Note: User a value that is relevant to your environment.
Done
5. Restart apache:
Code: Select all
service httpd restart
Done

6. Schedule a SSH backup from the GUI for a few minutes in the future.

7. Start a running tail on cmdsubsys.log:
Code: Select all
tail -f /usr/local/nagiosxi/var/cmdsubsys.log
and watch the log as the backup starts.

8. Post the output from the "/usr/local/nagiosxi/var/cmdsubsys.log" on the forum.
[*][root@nagiosxi nagiosxi]# tail -f /usr/local/nagiosxi/var/cmdsubsys.log
PROCESSED 0 COMMANDS
.............................................................
PROCESSED 0 COMMANDS
............................................................
PROCESSED 0 COMMANDS
...........................................................
PROCESSED 0 COMMANDS
.............................................................
PROCESSED 0 COMMANDS
............................................................
PROCESSED 0 COMMANDS
..........................................................
PROCESSED 0 COMMANDS
...........................................................
PROCESSED 0 COMMANDS
.PROCESSING COMMAND ID 16315...
PROCESS COMMAND: CMD=1117, DATA=a:2:{i:0;s:19:"nagiosxi.1582843261";i:1;s:0:"";}
CMDLINE=sudo /usr/local/nagiosxi/scripts/backup_xi.sh -n nagiosxi.1582843261
No entry for terminal type "unknown";
using dumb terminal settings.
Stopping nagios: .done.
Starting nagios: done.
Backing up NagiosQL...
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
Backing up Nagios Core...
tar: Removing leading `/' from member names
tar: /usr/local/nagios/var/ndo.sock: socket ignored
tar: /usr/local/nagios/var/rw/nagios.qh: socket ignored
.......................................tar: /usr/local/nagios/var: file changed as we read it
...............tar: /usr/local/nagios/share/perfdata/VHOST_-_CNS_-_MGMT_-_CDR/_HOST_.rrd: file changed as we read it
tar: /usr/local/nagios/share/perfdata/VHOST_-_CNS_-_MGMT_-_CDR: file changed as we read it
....
PROCESSED 0 COMMANDS
..............................tar: /usr/local/nagios/share/perfdata/SWITCH_-_cmetro-sw2.ak: file changed as we read it
...............................
PROCESSED 0 COMMANDS
......................................tar: /usr/local/nagios/share/perfdata/SWITCH_-_dv-csc-sw-2.office: file changed as we read it
.....................
PROCESSED 0 COMMANDS
................tar: /usr/local/nagios/share/perfdata/VHOST_-_SGSCAPP001.ivasp.net: file changed as we read it
...........................................
PROCESSED 0 COMMANDS
.........................tar: /usr/local/nagios/share/perfdata/SWITCH_-_cnexus-220-sw1.ak/cisco_nexus_hardware.xml: file changed as we read it
...tar: /usr/local/nagios/share/perfdata/SWITCH_-_cnexus-220-sw1.ak: file changed as we read it
..............................
PROCESSED 0 COMMANDS
....tar: /usr/local/nagios/share/perfdata/SWITCH_-_ccore-sky-sw2.ak: file changed as we read it
.......................................................
PROCESSED 0 COMMANDS
..............................................................
PROCESSED 0 COMMANDS
.........................................................
PROCESSED 0 COMMANDS
.......................tar: /usr/local/nagios/share/perfdata/SWITCH_-_ZeroOne_Core_Stack: file changed as we read it
............tar: /usr/local/nagios/share/perfdata/VHOST_-_XC5-FSR-AK2/_HOST_.xml.7326: File removed before we read it
.tar: /usr/local/nagios/share/perfdata/VHOST_-_XC5-FSR-AK2: file changed as we read it
.................tar: /usr/local/nagios/share/perfdata/Management_-_VCentre_-_Network/_HOST_.rrd: file changed as we read it
tar: /usr/local/nagios/share/perfdata/Management_-_VCentre_-_Network: file changed as we read it
.......
PROCESSED 0 COMMANDS
..................................Backing up Nagios XI...
tar: Removing leading `/' from member names
........................
PROCESSED 0 COMMANDS
.............................................tar: /usr/local/nagiosxi/var/cmdsubsys.log: file changed as we read it
..Backing up MRTG...
tar: Removing leading `/' from member names
............
PROCESSED 0 COMMANDS
...........................................................
PROCESSED 0 COMMANDS
...........................................................
PROCESSED 0 COMMANDS
................tar: /var/lib/mrtg: file changed as we read it
Backing up the SNMP directories
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
.Backing up NRDP...
tar: Removing leading `/' from member names
......Backing up Nagvis...
tar: Removing leading `/' from member names
.Backing up nagios user home dir...
tar: Removing leading `/' from member names
Backing up MySQL databases...
...............Backing up PostgresQL databases...
.Backing up cronjobs for Apache...
Backing up logrotate config files...
Backing up Apache config files...
Compressing backup...
..................
PROCESSED 0 COMMANDS
..........................................................
PROCESSED 0 COMMANDS
.........................................................
PROCESSED 0 COMMANDS
............................................................
PROCESSED 0 COMMANDS
...............................................
===============
BACKUP COMPLETE
===============
Backup stored in /store/backups/nagiosxi/nagiosxi.1582843261.tar.gz
OUTPUT=Backup stored in /store/backups/nagiosxi/nagiosxi.1582843261.tar.gz
RETURNCODE=0

PROCESSED 1 COMMANDS
.PROCESSING COMMAND ID 16316...
PROCESS COMMAND: CMD=1119, DATA=a:2:{i:0;s:19:"nagiosxi.1582843261";i:1;s:24:"/store/backups/nagiosxi/";}
CMDLINE=rm -rf /store/backups/nagiosxi/nagiosxi.1582843261.tar.gz
OUTPUT=
RETURNCODE=0
.........
PROCESSED 1 COMMANDS
............................................................
PROCESSED 0 COMMANDS
..........................................................
PROCESSED 0 COMMANDS
...........................................................
PROCESSED 0 COMMANDS
............................................................
PROCESSED 0 COMMANDS
...........................................................
PROCESSED 0 COMMANDS
.............................................................
PROCESSED 0 COMMANDS
....................................^C
[root@nagiosxi nagiosxi]# [*]

9. Post the "/usr/local/nagiosxi/var/components/scheduledbackups.log" on the forum.

No logs populated
[*][root@nagiosxi ~]# tail -f /usr/local/nagiosxi/var/components/scheduledbackups.log
02-19-2020 23:56:05 ERROR: Scheduled SSH Backup Failed: File was not transferred successfully
02-26-2020 10:20:01 ERROR: Scheduled SSH Backup Failed: File was not transferred successfully[*]

Backupjob successfully created and compressed backup but didn't trigger the transfer i guess

[root@nagiosxi nagiosxi]# ls -l
total 12070900
-rw-r--r-- 1 nagios nagios 2955065551 Mar 14 2019 autoupgrade_backup.1552519934.tar.gz
-rw-r--r-- 1 nagios nagios 0 Nov 6 11:13 backuptest.txt
-rw-r--r-- 1 nagios nagios 3202141242 Dec 10 12:29 nagiosxi.1575933241.tar.gz
-rw-r--r-- 1 nagios nagios 3227699226 Feb 26 12:41 nagiosxi.1582673117.tar.gz
drwxr-xr-x 8 root root 4096 Feb 28 11:55 nagiosxi.1582843261
-rw-r--r-- 1 root root 2975563776 Feb 28 11:59 nagiosxi.1582843261.tar.gz
[root@nagiosxi nagiosxi]# ls -l
total 9165020
-rw-r--r-- 1 nagios nagios 2955065551 Mar 14 2019 autoupgrade_backup.1552519934.tar.gz
-rw-r--r-- 1 nagios nagios 0 Nov 6 11:13 backuptest.txt
-rw-r--r-- 1 nagios nagios 3202141242 Dec 10 12:29 nagiosxi.1575933241.tar.gz
-rw-r--r-- 1 nagios nagios 3227699226 Feb 26 12:41 nagiosxi.1582673117.tar.gz
[root@nagiosxi nagiosxi]#

Posted: **Fri Feb 28, 2020 10:36 am**

The php function that is used in the component is trying to change the permissions of the file to 644, and it is possible that the nagiosxi user is not able to do that, so the transfer is failing. Can you run the following command on the remote machine, and show the output?

Code: Select all

ls -lad /var /var/data /var/data/chroot /var/data/chroot/sftp /var/data/chroot/sftp/nagiosxi

FYI, when I try to transfer a file from the CLI via scp (the same way you do), I am asked:

Enter passphrase for key '/usr/local/nagiosxi/var/keys/ssh.xi.1522249594'

but I don't see:

You are NOT welcome to connect!

I am not sure why you are receiving this message. Also, did you have to break out of the connection, using "ctrl+c"?

example01.PNG

Posted: **Sun Mar 01, 2020 5:11 pm**

lmiltchev wrote:The php function that is used in the component is trying to change the permissions of the file to 644, and it is possible that the nagiosxi user is not able to do that, so the transfer is failing. Can you run the following command on the remote machine, and show the output?
Code: Select all
ls -lad /var /var/data /var/data/chroot /var/data/chroot/sftp /var/data/chroot/sftp/nagiosxi

@mgmt-tori ~ $ ls -lad /var /var/data /var/data/chroot /var/data/chroot/sftp /var/data/chroot/sftp/nagiosxi
drwxr-xr-x 18 root root 4096 Oct 29 15:22 /var
drwxr-xr-x 10 root root 4096 Dec 13 11:54 /var/data
drwxr-xr-x 3 root root 4096 Jan 17 2017 /var/data/chroot
drwxr-xr-x 5 root root 4096 Aug 21 2019 /var/data/chroot/sftp
drwx------ 2 nagiosxi nagiosxi 4096 Feb 26 12:43 /var/data/chroot/sftp/nagiosxi

FYI, when I try to transfer a file from the CLI via scp (the same way you do), I am asked:
Enter passphrase for key '/usr/local/nagiosxi/var/keys/ssh.xi.1522249594'

but I don't see:
You are NOT welcome to connect!

That is default msg within network on our servers and devices

I am not sure why you are receiving this message. Also, did you have to break out of the connection, using "ctrl+c"?
example01.PNG

Yes, used "ctrl+c"

Posted: **Sun Mar 01, 2020 5:12 pm**

I am a little curious if someone else is having same issue or some issue related to backup via web interface as well?

Posted: **Mon Mar 02, 2020 12:39 pm**

It's too bad there is not enough info in the scheduledbackups.log even after enabling debugging. Let's try something else. Schedule a SSH backup from the GUI to be run in a few minutes in the future. Wait until it's done. You can watch the cmdsubsys.log to see when it finishes.

Code: Select all

tail -f /usr/local/nagiosxi/var/cmdsubsys.log

Next, run the following command and show the output:

Code: Select all

grep ssh2_scp_send /var/log/httpd/error_log

Here's something else you can try - change the backup directory. Use a directory that is in the "nagiosxi" user's home. Test the SCP Transfer from the GUI again. Did it work now?

Also, it seems like you are using some custom SSH settings, e.g. custom port, etc., so it's hard to say what's going on. Hopefully, we will find some clues in the apache error log. If not, you can PM me the sshd_conf file from the remote box, and we will try to recreate the issue in-house.

Posted: **Mon Mar 02, 2020 5:56 pm**

Next, run the following command and show the output:
Code: Select all
grep ssh2_scp_send /var/log/httpd/error_log

nothing in logs
agiosxi nagiosxi]# grep ssh2_scp_send /var/log/httpd/error_log
[root@nagiosxi nagiosxi]#
[root@nagiosxi nagiosxi]#

Here's something else you can try - change the backup directory. Use a directory that is in the "nagiosxi" user's home. Test the SCP Transfer from the GUI again. Did it work now?

Didn't Work

Also, it seems like you are using some custom SSH settings, e.g. custom port, etc., so it's hard to say what's going on. Hopefully, we will find some clues in the apache error log. If not, you can PM me the sshd_conf file from the remote box, and we will try to recreate the issue in-house.

will PM you config, please make sure you are using nagiosxi on centos 6 for test

In addition to that, nagiosxi on centos is using pecl ssh2 0.11.0-dev
can this be an issue,compatibility? https://pecl.php.net/package/ssh2/0.11.0
if so how to fix it ?

Posted: **Tue Mar 03, 2020 12:07 pm**

The pecl ssh2 0.11.0-dev package may or may not be an issue. What I believe is that the issue is caused by your sshd_config file. It is heavily customized...

We test our software against the "default" ssh settings. We are not going to modify our software to work with numerous custom settings that users may decide to implement, and that we cannot even test in our environment... It will be on you to make sure your custom settings work with our software.

I am afraid we are not going to be able to help you with this issue. It is out of scope of Nagios support.

Having said that, I tried using a custom port just to rule this out as a point of failure. I was able to transfer the "test" file from the GUI, and backup my test XI server just fine.

Here's the sshd_config that I used:

Code: Select all

#	$OpenBSD: sshd_config,v 1.80 2008/07/02 02:24:18 djm Exp $

# This is the sshd server system-wide configuration file.  See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/local/bin:/bin:/usr/bin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented.  Uncommented options change a
# default value.

Port 2222
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

# Disable legacy (protocol version 1) support in the server for new
# installations. In future the default will change to require explicit
# activation of protocol 1
Protocol 2

# HostKey for protocol version 1
#HostKey /etc/ssh/ssh_host_key
# HostKeys for protocol version 2
#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_dsa_key

# Lifetime and size of ephemeral version 1 server key
#KeyRegenerationInterval 1h
#ServerKeyBits 1024

# Logging
# obsoletes QuietMode and FascistLogging
#SyslogFacility AUTH
SyslogFacility AUTHPRIV
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
#PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

#RSAAuthentication yes
#PubkeyAuthentication yes
#AuthorizedKeysFile	.ssh/authorized_keys
#AuthorizedKeysCommand none
#AuthorizedKeysCommandRunAs nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#RhostsRSAAuthentication no
# similar for protocol version 2
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# RhostsRSAAuthentication and HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
PasswordAuthentication yes

# Change to no to disable s/key passwords
#ChallengeResponseAuthentication yes
ChallengeResponseAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no
#KerberosUseKuserok yes

# GSSAPI options
#GSSAPIAuthentication no
GSSAPIAuthentication yes
#GSSAPICleanupCredentials yes
GSSAPICleanupCredentials yes
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no

# Set this to 'yes' to enable PAM authentication, account processing, 
# and session processing. If this is enabled, PAM authentication will 
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
#UsePAM no
UsePAM yes

# Accept locale-related environment variables
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
#X11Forwarding no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PrintMotd yes
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
#UsePrivilegeSeparation yes
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#ShowPatchLevel no
#UseDNS yes
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none

# no default banner path
#Banner none

# override default of no subsystems
Subsystem	sftp	/usr/libexec/openssh/sftp-server

# Example of overriding settings on a per-user basis
#Match User anoncvs
#	X11Forwarding no
#	AllowTcpForwarding no
#	ForceCommand cvs server

Nagios XI:

example01.PNG

Remote machine:

Code: Select all

[root@centos6 nagiosxi]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:f8:f4:1c brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.237/16 brd 192.168.255.255 scope global eth0
    inet6 fe80::20c:29ff:fef8:f41c/64 scope link
       valid_lft forever preferred_lft forever
[root@centos6 nagiosxi]# pwd
/var/data/chroot/sftp/nagiosxi
[root@centos6 nagiosxi]# ll
total 761944
-rw-r--r--. 1 nagiosxi nagiosxi 780221670 Mar  3 04:37 nagiosxi.1583253122.tar.gz
-rw-r--r--. 1 nagiosxi nagiosxi       110 Mar  3 04:58 ssh_test_secure_copy.txt
[root@centos6 nagiosxi]# cat /etc/issue
CentOS release 6.7 (Final)
Kernel \r on an \m

Posted: **Tue Mar 03, 2020 3:22 pm**

do I need to use default sshd conf to get it successful? as in that will be the solution or there will be further to that?

Posted: **Tue Mar 03, 2020 5:42 pm**

Scheduled SSH backup should work with the "default" config. However, I am not saying that you must use the "default". You are welcome to use whatever you want but you will have to find a way to make it work on your end.

I would recommend trying your custom config in a test environment first, and tweaking it until you make it work. Once it is working, you can use it in production.

Each Nagios XI license is approved for up to three installations: one primary monitoring/production, one backup/failover, and one test environment.

https://support.nagios.com/kb/article/n ... s-145.html

Posted: **Thu Mar 05, 2020 7:13 pm**

Hi,
I used your pasted sshd_conf on gentoo with OpenSSH_8.0p1, OpenSSL 1.1.1d 10 Sep 2019
it is not working from web. so i think there isn't any thing wrong with the sshd_conf file i was previously using .
tested on Arch linux OpenSSH_8.1p1, OpenSSL 1.1.1d 10 Sep 2019 didn't work
can you please confirm if you were testing from centos 6 to a box that hat openssh_8 on wards? Now its more looks like a compatibility issue to me :S

Nagios Support Forum

Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh

Re: Nagios Backup from GUI via ssh