Page 1 of 1

Monitoring Tomcat with NCPA failure on XI server

Posted: Fri Feb 25, 2022 10:29 am
by 3dxops
I am trying to monitor Apache Tomcat with NCPA on XI version 5.8.7.
On my tomcat server my ncpa.cfg has the jar file callout in plugins section ".jar = /opt/ds/java/default/bin/java -jar $plugin_name $plugin_args".
On my tomcat server I have NCPA installed and I have the check_jvm.jar in the plugins directory. I also have setup JMX ports on my tomcat instances to connect to like the following:

Code: Select all

CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
Also on my tomcat server I can run the following check with no issue, it returns fine.

Code: Select all

java -jar check_jvm.jar -s service:jmx:rmi:///jndi/rmi://localhost:9010/jmxrmi -u tomcat -p tomcat -t tomcat -C ClassCount:"http-nio-8080"  -w 10000 -c 50000
Class Count returned WARNING with 24719 classes loaded | tomcat_classes=24719;10000;50000;;
When I run the check from my NagiosXI server i get the following error. All my other NCPA checks work fine.

Code: Select all

/usr/local/nagios/libexec/check_ncpa.py -H 10.35.132.245 -t Passw0rd1 -P 5693 -M plugins/check_jvm.jar -q "args=-s service:jmx:rmi:///jndi/rmi://localhost:9010/jmxrmi,args=-u '3dx',args=-p 'x3dTtT7',args=-t tomcat,args=-C 'MemorySimpleHeap,MemoryEden,MemorySurvivor,MemoryOld,MemoryMetaspace,ProcessCPUUsage,SystemCPUUsage,Uptime,ThreadCount',args=-w '17179869184,,,,34359738368,50,70,1500000:,100',args=-c '32212254720,32212254720,32212254720,32212254720,68719476736,70,90,300000:,150'"
UNKNOWN: An error occured connecting to API. (HTTP error: '500 INTERNAL SERVER ERROR')
Any Ideas why this would be failing from NagiosXI server?

Here is the failed log from my tomcat server's NCPA/var/log file when this check runs.

Code: Select all

2022-02-25 15:39:28,714 12640 INFO ::ffff:10.35.134.248 - - [2022-02-25 15:39:28] "GET /api/interface/ens192/bytes_recv/?token=Passw0rd1&warning=10&critical=100&delta=True&units=M&check=1 HTTP/1.1" 200 316 0.010956
2022-02-25 15:39:42,849 12640 ERROR Exception on /api/plugins/check_jvm.jar/ [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
  File "/root/ncpa/agent/listener/server.py", line 292, in token_auth_decoration
  File "/root/ncpa/agent/listener/server.py", line 1099, in api
  File "/root/ncpa/agent/listener/nodes.py", line 308, in run_check
  File "/root/ncpa/agent/listener/pluginnodes.py", line 115, in execute_plugin
  File "/usr/local/lib/python2.7/site-packages/gevent/subprocess.py", line 585, in __init__
  File "/usr/local/lib/python2.7/site-packages/gevent/subprocess.py", line 554, in __init__
  File "/usr/local/lib/python2.7/site-packages/gevent/subprocess.py", line 1312, in _execute_child
OSError: [Errno 13] Permission denied
2022-02-25 15:39:42,851 12640 INFO ::ffff:10.35.134.248 - - [2022-02-25 15:39:42] "GET /api/plugins/check_jvm.jar/?token=Passw0rd1&check=1&args=-s+service%3Ajmx%3Armi%3A%2F%2F%2Fjndi%2Frmi%3A%2F%2Flocalhost%3A9010%2Fjmxrmi&args=-u+%273dx%27&args=-p+%27x3dTtT7%27&args=-t+tomcat&args=-C+%27MemorySimpleHeap%2CMemoryEden%2CMemorySurvivor%2CMemoryOld%2CMemoryMetaspace%2CProcessCPUUsage%2CSystemCPUUsage%2CUptime%2CThreadCount%27&args=-w+%2717179869184%2C%2C%2C%2C34359738368%2C50%2C70%2C1500000%3A%2C100%27&args=-c+%2732212254720%2C32212254720%2C32212254720%2C32212254720%2C68719476736%2C70%2C90%2C300000%3A%2C150%27 HTTP/1.1" 500 2362 0.015912
2022-02-25 15:39:47,131 12640 INFO ::ffff:10.35.134.248 - - [2022-02-25 15:39:47] "GET /api/disk/logical/%7Cboot/?token=Passw0rd1&warning=70&critical=90&check=1 HTTP/1.1" 200 399 0.068758

Re: Monitoring Tomcat with NCPA failure on XI server

Posted: Fri Feb 25, 2022 5:07 pm
by gsmith
HI

From looking at

Code: Select all

2022-02-25 15:39:42,849 12640 ERROR Exception on /api/plugins/check_jvm.jar/ [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
  File "/root/ncpa/agent/listener/server.py", line 292, in token_auth_decoration
  File "/root/ncpa/agent/listener/server.py", line 1099, in api
  File "/root/ncpa/agent/listener/nodes.py", line 308, in run_check
  File "/root/ncpa/agent/listener/pluginnodes.py", line 115, in execute_plugin
  File "/usr/local/lib/python2.7/site-packages/gevent/subprocess.py", line 585, in __init__
  File "/usr/local/lib/python2.7/site-packages/gevent/subprocess.py", line 554, in __init__
  File "/usr/local/lib/python2.7/site-packages/gevent/subprocess.py", line 1312, in _execute_child
OSError: [Errno 13] Permission denied
take a look on the permissions for check_jvm.jar please


Thanks

Re: Monitoring Tomcat with NCPA failure on XI server

Posted: Tue Mar 29, 2022 1:37 pm
by 3dxops
Thanks, for the help. issue solved