Unable to run Docker Commands

[jjbursik]

Facing an issue where Nagios is unable to run docker --version command through nrpe however I'm able to run the command as user nrpe on the remote host.

running as nrpe on remote host:

Code: Select all

[root@remote ~]# su nrpe
bash-4.2$ sh /etc/nagios/check_container.sh test
nrpe
Docker version 18.06.0-ce, build 0ffa825

running through Nagios (you can see docker version isn't printing):

Code: Select all

Status Information:	
nrpe

[mcapra]

Try redirecting the Docker command's STDERR into STDOUT in your script and see if that changes the results.

Would it also be possible for you to share the contents of check_container.sh as well as your NRPE configuration file?

[jjbursik]

Here is the shell script:

Code: Select all

#!/bin/bash
whoami
RESPONSE=$(docker --version)
echo $RESPONSE
if [ ! "$RESPONSE" ];
        then
                echo "response is null"
                exit 2
        else
                echo "$RESPONSE"
                exit 0
fi

Here is the nrpe.cfg:

Code: Select all

#############################################################################
# NRPE Config File
#############################################################################

# LOG FACILITY
log_facility=daemon

# PID FILE
pid_file=/var/run/nrpe/nrpe.pid

# PORT NUMBER
server_port=5666

# NRPE USER
nrpe_user=nrpe

# NRPE GROUP
nrpe_group=nrpe

# ALLOWED HOST ADDRESSES
allowed_hosts=127.0.0.1,1.1.1.1

# COMMAND ARGUMENT PROCESSING
dont_blame_nrpe=1

# BASH COMMAND SUBTITUTION
allow_bash_command_substitution=0

# DEBUGGING OPTION
debug=0

# COMMAND TIMEOUT
command_timeout=60

# CONNECTION TIMEOUT
connection_timeout=300

# COMMAND DEFINITIONS
command[check_users]=/usr/lib64/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib64/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
command[check_disk]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 10% -p /var
command[check_procs]=/usr/lib64/nagios/plugins/check_procs -w 150 -c 200
command[check_ping]=/usr/lib64/nagios/plugins/check_ping!100.0,20%!500.0,60%
command[check_container]=/usr/bin/bash /etc/nagios/check_container.sh 
include_dir=/etc/nrpe.d/

Worth noting that check_users, check_disk, and all other commands return back successful.

[jjbursik]

Worth noting that all other commands are successful as well.. here are the configs you asked for:

Shell Script:

Code: Select all

#!/bin/bash
whoami
RESPONSE=$(docker --version)
echo $RESPONSE
if [ ! "$RESPONSE" ];
        then
                echo "response is null"
                exit 2
        else
                echo "$RESPONSE"
                exit 0
fi

nrpe.cfg:

#############################################################################
# NRPE Config File
#############################################################################

# LOG FACILITY
log_facility=daemon

# PID FILE
pid_file=/var/run/nrpe/nrpe.pid

# PORT NUMBER
server_port=5666

# NRPE USER
nrpe_user=nrpe

# NRPE GROUP
nrpe_group=nrpe

# ALLOWED HOST ADDRESSES
allowed_hosts=127.0.0.1,1.1.11.111

# COMMAND ARGUMENT PROCESSING
dont_blame_nrpe=1

# BASH COMMAND SUBTITUTION
allow_bash_command_substitution=0

# DEBUGGING OPTION
debug=0

# COMMAND TIMEOUT
command_timeout=60

# CONNECTION TIMEOUT
connection_timeout=300

# COMMAND DEFINITIONS
command[check_users]=/usr/lib64/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib64/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
command[check_disk]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 10% -p /var
command[check_procs]=/usr/lib64/nagios/plugins/check_procs -w 150 -c 200
command[check_ping]=/usr/lib64/nagios/plugins/check_ping!100.0,20%!500.0,60%
command[check_container]=/usr/bin/bash /etc/nagios/check_container.sh
include_dir=/etc/nrpe.d/

[scottwilkerson]

It looks like it is printing on the second line which would be only shown in longoutput

Can you remove the echo of nrpe from the script?

[jjbursik]

it doesn't seem like the echo was the issue as I get from Nagios:

Code: Select all

Status Information:   
nrpe
response is null

[jjbursik]

I turned debug on and get this error:

Code: Select all

WARNING: my_system() seteuid(0): Operation not permitted

[scottwilkerson]

I turned debug on and get this error:

Code: Select all

WARNING: my_system() seteuid(0): Operation not permitted

You may need to put the full path to docker in the script

[mcapra]

A brief explanation of this through the lens of executing java commands via NRPE:

It's worth mentioning that, if you're relying on Bash to resolve the java command on behalf of the nagios user, this won't necessarily work when NRPE executes a given command since NRPE doesn't do so through Bash. You could modify check_jmx to include the full path to the java binary to rule out potential environmental issues. Something like this:

Code: Select all

#!/bin/sh
#
# Nagios plugin to monitor Java JMX (http://java.sun.com/jmx)attributes.
#
RDIR=`dirname $0`
/path/to/jdk/bin/java  -jar $RDIR/jmxquery.jar "$@"

[scottwilkerson]

yep