Monitoring Tomcat with NCPA failure on XI server
Posted: Fri Feb 25, 2022 10:29 am
I am trying to monitor Apache Tomcat with NCPA on XI version 5.8.7.
On my tomcat server my ncpa.cfg has the jar file callout in plugins section ".jar = /opt/ds/java/default/bin/java -jar $plugin_name $plugin_args".
On my tomcat server I have NCPA installed and I have the check_jvm.jar in the plugins directory. I also have setup JMX ports on my tomcat instances to connect to like the following:
Also on my tomcat server I can run the following check with no issue, it returns fine.
When I run the check from my NagiosXI server i get the following error. All my other NCPA checks work fine.
Any Ideas why this would be failing from NagiosXI server?
Here is the failed log from my tomcat server's NCPA/var/log file when this check runs.
On my tomcat server my ncpa.cfg has the jar file callout in plugins section ".jar = /opt/ds/java/default/bin/java -jar $plugin_name $plugin_args".
On my tomcat server I have NCPA installed and I have the check_jvm.jar in the plugins directory. I also have setup JMX ports on my tomcat instances to connect to like the following:
Code: Select all
CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
Code: Select all
java -jar check_jvm.jar -s service:jmx:rmi:///jndi/rmi://localhost:9010/jmxrmi -u tomcat -p tomcat -t tomcat -C ClassCount:"http-nio-8080" -w 10000 -c 50000
Class Count returned WARNING with 24719 classes loaded | tomcat_classes=24719;10000;50000;;
Code: Select all
/usr/local/nagios/libexec/check_ncpa.py -H 10.35.132.245 -t Passw0rd1 -P 5693 -M plugins/check_jvm.jar -q "args=-s service:jmx:rmi:///jndi/rmi://localhost:9010/jmxrmi,args=-u '3dx',args=-p 'x3dTtT7',args=-t tomcat,args=-C 'MemorySimpleHeap,MemoryEden,MemorySurvivor,MemoryOld,MemoryMetaspace,ProcessCPUUsage,SystemCPUUsage,Uptime,ThreadCount',args=-w '17179869184,,,,34359738368,50,70,1500000:,100',args=-c '32212254720,32212254720,32212254720,32212254720,68719476736,70,90,300000:,150'"
UNKNOWN: An error occured connecting to API. (HTTP error: '500 INTERNAL SERVER ERROR')
Here is the failed log from my tomcat server's NCPA/var/log file when this check runs.
Code: Select all
2022-02-25 15:39:28,714 12640 INFO ::ffff:10.35.134.248 - - [2022-02-25 15:39:28] "GET /api/interface/ens192/bytes_recv/?token=Passw0rd1&warning=10&critical=100&delta=True&units=M&check=1 HTTP/1.1" 200 316 0.010956
2022-02-25 15:39:42,849 12640 ERROR Exception on /api/plugins/check_jvm.jar/ [GET]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
File "/root/ncpa/agent/listener/server.py", line 292, in token_auth_decoration
File "/root/ncpa/agent/listener/server.py", line 1099, in api
File "/root/ncpa/agent/listener/nodes.py", line 308, in run_check
File "/root/ncpa/agent/listener/pluginnodes.py", line 115, in execute_plugin
File "/usr/local/lib/python2.7/site-packages/gevent/subprocess.py", line 585, in __init__
File "/usr/local/lib/python2.7/site-packages/gevent/subprocess.py", line 554, in __init__
File "/usr/local/lib/python2.7/site-packages/gevent/subprocess.py", line 1312, in _execute_child
OSError: [Errno 13] Permission denied
2022-02-25 15:39:42,851 12640 INFO ::ffff:10.35.134.248 - - [2022-02-25 15:39:42] "GET /api/plugins/check_jvm.jar/?token=Passw0rd1&check=1&args=-s+service%3Ajmx%3Armi%3A%2F%2F%2Fjndi%2Frmi%3A%2F%2Flocalhost%3A9010%2Fjmxrmi&args=-u+%273dx%27&args=-p+%27x3dTtT7%27&args=-t+tomcat&args=-C+%27MemorySimpleHeap%2CMemoryEden%2CMemorySurvivor%2CMemoryOld%2CMemoryMetaspace%2CProcessCPUUsage%2CSystemCPUUsage%2CUptime%2CThreadCount%27&args=-w+%2717179869184%2C%2C%2C%2C34359738368%2C50%2C70%2C1500000%3A%2C100%27&args=-c+%2732212254720%2C32212254720%2C32212254720%2C32212254720%2C68719476736%2C70%2C90%2C300000%3A%2C150%27 HTTP/1.1" 500 2362 0.015912
2022-02-25 15:39:47,131 12640 INFO ::ffff:10.35.134.248 - - [2022-02-25 15:39:47] "GET /api/disk/logical/%7Cboot/?token=Passw0rd1&warning=70&critical=90&check=1 HTTP/1.1" 200 399 0.068758