Error when using check_docker.py

[emarmonti]

Hello,

Thank you for providing an updated/patched check_docker.py file. Unfortunately I'm still having the problem. I also tried running the check manually (based off the arguments the Docker Wizard put into the service check.

Code: Select all

[nagios@ma1p-nagiosxi libexec]$ ./check_ncpa.py -H ma1p-dock02 -t 'XXXX-key-from-nagiosXI-XXXX' -P 5693 -M plugins/check_docker.py -q "args=-H http:/v1.30/ ,args=-s /var/run/docker.sock,args=--check-type 'containers_cpu',args=--all,args=-t 30,args=--networks-use-avg,args=-w '30',args=-c '50'"
Traceback (most recent call last):
  File "/usr/local/ncpa/plugins/check_docker.py", line 988, in <module>
    _ = main()
  File "/usr/local/ncpa/plugins/check_docker.py", line 959, in main
    checks = choose_checks(options)
  File "/usr/local/ncpa/plugins/check_docker.py", line 226, in choose_checks
    check_data = get_threshold_maps(options.warning, options.critical, selection)
  File "/usr/local/ncpa/plugins/check_docker.py", line 329, in get_threshold_maps
    for triplet in zip(attrs.keys(), warning_list, critical_list):
AttributeError: 'list' object has no attribute 'keys'
[nagios@ma1p-nagiosxi libexec]$
[nagios@ma1p-nagiosxi libexec]$ md5sum /usr/local/ncpa/plugins/check_docker.py
aad080c4eb8d753b05fc4b9c8ead1485  /usr/local/ncpa/plugins/check_docker.py
[nagios@ma1p-nagiosxi libexec]$
[nagios@ma1p-nagiosxi libexec]$ ls -l /usr/local/ncpa/plugins/check_docker.py
-rwxr-xr-x 1 nagios nagios 39293 Jun 10 12:13 /usr/local/ncpa/plugins/check_docker.py
[nagios@ma1p-nagiosxi libexec]$

[npolovenko]

@emarmonti, Have you tried acessing the Docker API interface directly using the new plugin I uploaded?

Code: Select all

 /usr/local/nagios/libexec/check_docker.py -H http://dockerIP:dockerPort/ --check-type 'containers_cpu'

Here's an example:

Code: Select all

[root@centos7x64 libexec]# /usr/local/nagios/libexec/check_docker.py -H http://192.168.23.5:1234/ --check-type 'containers_cpu'
the total usage check is {'value': 0.03941240601503759, 'warning': (0, 50.0, False), 'critical': (0, 75.0, False), 'name': 'total_usage', 'containers': []}
OK: total_usage returned OK (value 0.039412406015%),  | total_usage=0.039412406015%;50.0;75.0

[emarmonti]

Hello,

I tried running it manually and it fails to connect:

Code: Select all

[nagios@ma1p-nagiosxi libexec]$ /usr/local/nagios/libexec/check_docker.py -H http://ma1p-docker01:5693/ --check-type 'containers_cpu'
UNKNOWN: cURL call failed
[nagios@ma1p-nagiosxi libexec]$ telnet ma1p-docker01 5693
Trying 10.1.12.170...
Connected to ma1p-docker01.
Escape character is '^]'.
^]
telnet> Connection closed.
[nagios@ma1p-nagiosxi libexec]$

You can see that I'm able to connect to port 5693 on the server. When I was on the docker server, I was able to see my telnet connection via netstat. I also tried restarting the ncpa_listener service to no avail.

Code: Select all

[root@ma1p-dock01 ncpa]# /etc/init.d/ncpa_listener restart
Restarting ncpa_listener (via systemctl):                  [  OK  ]
[root@ma1p-dock01 ncpa]#

For reference, I'll attach the ncpa.cfg file from /usr/local/ncpa/etc/ncpa.cfg

[npolovenko]

@emarmonti, 5693 is the NCPA port. If you're running the check_docker command directly you need to expose the docker API.
https://www.ivankrizsan.se/2016/05/18/e ... ntu-16-04/

Also, check out this article:
https://assets.nagios.com/downloads/nag ... ios-XI.pdf
At "Using Docker Remote API".


If you can't bind the Docker Daemon to the TCP port, please make sure that you can access the Docker API from the NCPA server.

Code: Select all

curl -k "http://dockerIP:port

And show me the output.

[dclsystems]

So file you provided is partially working, I cannot get values for containers_CPU and containers_memory eg:

root@docker2:/usr/local/ncpa/plugins# python /usr/local/ncpa/plugins/check_docker.py -H http://127.0.0.1:2375 /var/run/docker.sock --check-type 'containers_memory' -C '276c17b669b4' -t 0 --networks-use-avg --total-usage --total-average --memory-unit GiB -w '80,80' -c '90,90' --debug
containers
['276c17b669b4']
End selection + type
hit containers_list_to_dict
hit is_docker_container_ID
hit talk_to_docker
curl 'http://127.0.0.1:2375/containers/json?& ... %5D}&all=1' -g -f
hit threshold_string_to_tuple
hit threshold_string_to_tuple
hit threshold_string_to_tuple
hit threshold_string_to_tuple
hit get_container_IDs
hit do_check
hit check_containers_memory
ID_list is [u'276c17b669b45340aa99f973fdb9afe491206d7ab98bc938208bfcc0d3b9c82a']
hit talk_to_docker
curl 'http://127.0.0.1:2375/containers/276c17 ... ream=false' -g -f
hit process_value
hit process_usage
container_id_to_usage is
{u'276c17b669b45340aa99f973fdb9afe491206d7ab98bc938208bfcc0d3b9c82a': 19333120}
Traceback (most recent call last):
File "/usr/local/ncpa/plugins/check_docker.py", line 983, in <module>
_ = main()
File "/usr/local/ncpa/plugins/check_docker.py", line 957, in main
(perfdata, exit) = process_value(checks, values) # Gives a complete perfdata string and an exit code
File "/usr/local/ncpa/plugins/check_docker.py", line 949, in process_value
(perfdata, exit) = valid_checks[options.check_type][1](checks, values)
File "/usr/local/ncpa/plugins/check_docker.py", line 871, in process_usage
checks['total_usage'].setValue(total_usage)
KeyError: 'total_usage'


root@docker2:/usr/local/ncpa/plugins# docker version
Client:
Version: 18.09.6
API version: 1.39
Go version: go1.10.8
Git commit: 481bc77
Built: Sat May 4 02:35:57 2019
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 18.09.6
API version: 1.39 (minimum version 1.12)
Go version: go1.10.8
Git commit: 481bc77
Built: Sat May 4 01:59:36 2019
OS/Arch: linux/amd64
Experimental: false
root@docker2:/usr/local/ncpa/plugins# python -v
2.7

For containers_running I needed to modify script to use lower case "running" for status matching.

Thanks
RD

[swolf]

For the other techs on this thread, I sent RD a PM with instructions.

[SMART-IT]

After updating the python script with the one on this thread & providing it the necessary execution privileges (chmod 777 ...).
There are 14 containers running on that host ... but I don't seem to be able to search / query for much details

me@srvudevwebportal:~$ sudo /usr/local/ncpa/plugins/check_docker.py -H http://v1.35/ -s /var/run/docker.sock --check-type 'containers_memory' -C '276c17b669b4' -t 0 --networks-use-avg --total-usage --total-average --memory-unit GiB -w '80,80' -c '90,90' --debug
containers
['276c17b669b4']
End selection + type
hit containers_list_to_dict
hit is_docker_container_ID
hit talk_to_docker
curl --unix-socket /var/run/docker.sock 'http://v1.35/containers/json?&filters={ ... %5D}&all=1' -g -f
hit is_docker_container_name
hit talk_to_docker
curl --unix-socket /var/run/docker.sock 'http://v1.35/containers/json?&filters={ ... }&all=true' -g -f
hit threshold_string_to_tuple
hit threshold_string_to_tuple
hit threshold_string_to_tuple
hit threshold_string_to_tuple
hit get_container_IDs
hit do_check
hit check_containers_memory
ID_list is []
hit process_value
hit process_usage
container_id_to_usage is
{}
Traceback (most recent call last):
File "/usr/local/ncpa/plugins/check_docker.py", line 983, in <module>
_ = main()
File "/usr/local/ncpa/plugins/check_docker.py", line 957, in main
(perfdata, exit) = process_value(checks, values) # Gives a complete perfdata string and an exit code
File "/usr/local/ncpa/plugins/check_docker.py", line 949, in process_value
(perfdata, exit) = valid_checks[options.check_type][1](checks, values)
File "/usr/local/ncpa/plugins/check_docker.py", line 871, in process_usage
checks['total_usage'].setValue(total_usage)
KeyError: 'total_usage'

Could I also benefit from a PM ?

\T,

--
Thomas Lionel SMETS, CISSP / ISO-27034-LI / ISO-27001-LI / Data Officer

[ssax]

PM stands for private message and is only when we request something or are sending you something in the forums.

What is the full output of these commands?

Code: Select all

curl --unix-socket /var/run/docker.sock 'http://v1.35/containers/json?&filters={%22id%22:%5B%22276c17b669b4%22%5D}&all=1' -g -f
curl --unix-socket /var/run/docker.sock 'http://v1.35/containers/json?&filters={%22name%22:%5B%22276c17b669b4%22%5D}&all=true' -g -f

[SMART-IT]

For me the

check_docker.py

script is not located in the

Code: Select all

/usr/local/nagios

but in the

Code: Select all

/usr/local/ncpa/plugins

[ssax]

Follow page 2 of this guide onwards for NCPA:

Code: Select all

https://assets.nagios.com/downloads/nagiosxi/docs/How-to-Monitor-Docker-Containers-with-Nagios-XI.pdf

Let us know if you have any questions