Nagios Docker Stack Plugin works locally but not remote

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
TheBadMan
Posts: 1
Joined: Tue Sep 22, 2015 11:04 am

Nagios Docker Stack Plugin works locally but not remote

Post by TheBadMan »

I have a plugin to check the services running on docker.
I've gone through all of the suggestions from the dozens of posts I have seen here, but I'm still not able to get this to work.
The commands all run individually and the script runs as well, locally.
I added the nrpe user to the wheel and docker groups and set the wheel group in visudo to run anything without a password.

I modified /etc/passwd temporarily to allow the nrpe user to login with a shell. The script runs with or without sudo.

This part passes successfully:

Code: Select all

    /bin/docker > /dev/null 2>&1
    if [ $? -eq 1 ]; then
       echo "UNKNOWN - Missing docker binary"
       exit 3
    fi
This part fails. The command exists with a '1' and I can't get any output from the docker command. I've tried several things, including writing it to a file, but every time the file is blank.

Code: Select all

    /bin/docker stack ls | grep $STACK
    if [ $? -eq 1 ]; then
      echo "UNKNOWN - $STACK does not exist.  $(/usr/bin/sudo /bin/docker stack ls) \n  $(whoami)"
    exit 3
    fi
This is the output I'm getting:
UNKNOWN - ValidService does not exist. \n nrpe.
Any idea what permission I'm missing or is it something else?
kyang

Re: Nagios Docker Stack Plugin works locally but not remote

Post by kyang »

Are you using a custom plugin you created or something from the Nagios Exchange or elsewhere?

This is the output you're getting from using NRPE?

This is just a reference to a check_docker plugin from the Nagios Exchange. (Unsure if you are using this or a custom plugin?)
https://github.com/timdaman/check_docker
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: Nagios Docker Stack Plugin works locally but not remote

Post by mcapra »

The Docker daemon is particularly picky about what methods are used to communicate with it. I would highly recommend using the Docker SDK if you want to do Docker things with NRPE.

Having implemented similar orchestration, I've found popen (which NRPE uses) to be an incredibly unreliable way to interact with the Docker daemon.

That aside, I haven't encountered any problems invoking the docker command via NRPE:

Code: Select all

[root@capra_nag tmp]# /usr/local/nagios/libexec/check_nrpe -H 10.35.6.65 -c docker_test
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Here's my test script configured on the remote docker machine (associated with NRPE's docker_test command):

Code: Select all

echo $(docker ps 2>&1)
exit 0
I haven't done anything fancier than add the NRPE user (nagios) to the Docker group (docker). That and redirect STDERR to STDOUT in my calls. No sudo funny-business.
Former Nagios employee
https://www.mcapra.com/
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nagios Docker Stack Plugin works locally but not remote

Post by scottwilkerson »

Thanks for adding that @mcapra
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Locked