Service check timed out after 60.01 seconds

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
Prabha2028
Posts: 2
Joined: Wed Jun 30, 2021 10:48 am

Service check timed out after 60.01 seconds

Post by Prabha2028 »

Hi ,

When i am running the check_jmx from nagios server to other remote servers(windows servers), i am getting the below error.

Can someone help me on this?

[root@brxpdlibmonl01 Prod]# /usr/local/nagios/plugins/check_jmx_wrapper -U service:jmx:rmi:///jndi/rmi://172.18.1.91:19080/jmxrmi -O Catalina:type=ThreadPool,name=http-9080 -A currentThreadCount -vvvv -w 175 -c 250
JMX CRITICAL Connection refused to host: 172.18.1.91; nested exception is:
java.net.ConnectException: Connection timed out connecting to Catalina:type=ThreadPool,name=http-9080 by URL service:jmx:rmi:///jndi/rmi://172.18.1.91:19080/jmxrmijava.rmi.ConnectException: Connection refused to host: 172.18.1.91; nested exception is:
java.net.ConnectException: Connection timed out
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:129)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
at org.nagios.JMXQuery.connect(JMXQuery.java:53)
at org.nagios.JMXQuery.main(JMXQuery.java:75)
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

Ports are connected successfully:
[root@brxpdlibmonl01 Prod]# telnet 172.18.1.91 9080
Trying 172.18.1.91...
Connected to 172.18.1.91.
Escape character is '^]'.
^C^CConnection closed by foreign host.
[root@brxpdlibmonl01 Prod]# telnet 172.18.1.91 19080
Trying 172.18.1.91...
Connected to 172.18.1.91.
Escape character is '^]'.



Thanks,
Prabhas
User avatar
pbroste
Posts: 1288
Joined: Tue Jun 01, 2021 1:27 pm

Re: Service check timed out after 60.01 seconds

Post by pbroste »

Hello Prabhas,

Let's check to see if the ports are listening:

Code: Select all

sudo ss -tulwn | grep '9080|19080'
Also here is a line to test the check_jmx plugin:

Code: Select all

./check_jmx -U service:jmx:rmi:///jndi/rmi://localhost:7199/jmxrmi -O java.lang:type=Memory -A HeapMemoryUsage -K used -I HeapMemoryUsage -J used -vvvv -w 4248302272 -c 5498760192
Regards,
Perry
Prabha2028
Posts: 2
Joined: Wed Jun 30, 2021 10:48 am

Re: Service check timed out after 60.01 seconds

Post by Prabha2028 »

Hi Perry,

Thanks for your support.

I have excuted given commands and pasted the results below.

[root@brxpdlibmonl01 nagios]# sudo ss -tulwn | grep '9080|19080'
sudo: PAM account management error: Permission denied
[root@brxpdlibmonl01 nagios]# ./check_jmx -U service:jmx:rmi:///jndi/rmi://localhost:8080/jmxrmi -O java.lang:type=Memory -A HeapMemoryUsage -K used -I HeapMemoryUsage -J used -vvvv -w 4248302272 -c 5498760192
-bash: ./check_jmx: No such file or directory
[root@brxpdlibmonl01 nagios]# ./check_jmx -U service:jmx:rmi:///jndi/rmi://localhost:7199/jmxrmi -O java.lang:type=Memory -A HeapMemoryUsage -K used -I HeapMemoryUsage -J used -vvvv -w 4248302272 -c 5498760192
-bash: ./check_jmx: No such file or directory
[root@brxpdlibmonl01 nagios]#



Thanks,
Prabhas
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: Service check timed out after 60.01 seconds

Post by mcapra »

Depending on the firewall in-use on the 172.18.1.91 machine, telnet may not be enough to test that the port is truly open and that the check_jmx traffic is not being filtered out by the firewall. Might try nmap or netcat instead. Both are great tools to have on your Nagios server for troubleshooting.

Code: Select all

yum install nmap nc
netcat -z -n -v 172.18.1.91 19080
nmap -sS -O -p19080 172.18.1.91
It would also be useful to see specifically how you're configuring the JMX endpoint within your Tomcat instance. Specifically, the values of com.sun.management.jmxremote.local.only and java.rmi.server.hostname. But there are other properties that could cause connection failures.
Former Nagios employee
https://www.mcapra.com/
Locked