Nagios Support Forum

Nagios is unable to execute the check command, however, if I manually run the command I receive the response just fine:

OS NFS I/O BW - /droraclebk

UNKNOWN 07-03-2014 11:10:38 0d 1h 0m 10s 5/5 Remote command execution failed: Permission denied, please try again.


# /usr/lib/nagios/plugins/check_by_ssh -H hostname.com -l root -C '/usr/local/ng_plugins/check_io_bw -m emcvn'
IOBW hostname.com OK - /oraclebk (emcvnxnas:/oraclebk) Reads: 444105.1974 KB/Sec, 433.6965 MB/Sec, 0.4235 GB/Sec, Writes: 67120.6927 KB/Sec, 65.5476 MB/Sec, 0.0640 GB/Sec Test_Retries=1 Test_file_size=1024.00 Kb (1.00 Mb) (Read 433.6965 > 50 > 30, Write 65.5476 > 10 > 5)|bw_read=454763722.1567bps bw_write=68731589.3315bps bw_test_retries=1

Looks like it's a permissions issue, most likely on the remote server.

On your Nagios server, run:
and post the results here.

Then on the remote server, run:
and again post the results here.

# ls -l /usr/lib/nagios/plugins/check_by_ssh
-rwxr-xr-x 1 nagios apache 40872 Jun 25 2007 /usr/lib/nagios/plugins/check_by_ssh


# ls -l /usr/local/ng_plugins/check_io_bw
-rwxr-xr-x 1 root nagios 10526 Jul 5 2012 /usr/local/ng_plugins/check_io_bw


This check is failing for all hosts. I pretty certain its some permissions issue somewhere local to the nagios server install.

Did you follow a particular guide for this? If so, can you share the link?

If not can you tell us what you did to set this up?

Nothing really...

I inherited this environment, but Im not new to Nagios.

These checks were working until about 2 weeks ago, when we did a kernel update and server cycle.

From what I can tell, is the guy before me wrote a few perl scripts to pull specific I/O data, and other analytics, etc and compiled them into a rpm (ng_plugins)...

They do work, the PERL is solid... i.e. A manual run on the command line from our nagios server using check_by_ssh to "hostname.com" using a plugin to check the I/O on a EMC NAS NFS Share, returns the following:

(root)# /usr/lib/nagios/plugins/check_by_ssh -H hostname.com -l root -C '/usr/local/ng_plugins/check_io_bw -m emcvn'

IOBW hostname.com OK - /oraclebk (emcvnxnas:/oraclebk) Reads: 444105.1974 KB/Sec, 433.6965 MB/Sec, 0.4235 GB/Sec, Writes: 67120.6927 KB/Sec, 65.5476 MB/Sec, 0.0640 GB/Sec Test_Retries=1 Test_file_size=1024.00 Kb (1.00 Mb) (Read 433.6965 > 50 > 30, Write 65.5476 > 10 > 5)|bw_read=454763722.1567bps bw_write=68731589.3315bps bw_test_retries=1

However,

When the polling cycle hits, and Nagios does its thing, I get this returned to the Server Status Details page:

OS NFS I/O BW - /droraclebk UNKNOWN 07-03-2014 13:40:40 0d 3h 27m 17s 5/5 Remote command execution failed: Permission denied, please try again.

I should add, Im not the only body with access to this...

But no one is copping to messing with anything.

Thus my plight.

I would definitely suggest reviewing this document as it should outline all the requirements for check_by_ssh, since you do not know how it was set up:

http://assets.nagios.com/downloads/nagi ... ng_SSH.pdf

Yes, I get it, SSH keys.

All the correct public keys are there.

know_hosts file = 0600

That works fine, as I said, I run the command manually from a bash shell, and it works great.

It appears you have been running the command manually as root, not the nagios user. Try su-ing to the nagios user and executing your command again. If only the root user has the ssh keys setup, that would definitely explain why you are having difficulty.

(root)# /usr/lib/nagios/plugins/check_by_ssh -H hostname.com -l root -C '/usr/local/ng_plugins/check_io_bw -m emcvn'