Service check timed out after 60.01 seconds

Engage with the community of users including those using the open source solutions.
Includes Nagios Core, Plugins, and NCPA

Service check timed out after 60.01 seconds

Postby Prabha2028 » Mon Jul 12, 2021 8:00 am

Hi ,

When i am running the check_jmx from nagios server to other remote servers(windows servers), i am getting the below error.

Can someone help me on this?

[root@brxpdlibmonl01 Prod]# /usr/local/nagios/plugins/check_jmx_wrapper -U service:jmx:rmi:///jndi/rmi://172.18.1.91:19080/jmxrmi -O Catalina:type=ThreadPool,name=http-9080 -A currentThreadCount -vvvv -w 175 -c 250
JMX CRITICAL Connection refused to host: 172.18.1.91; nested exception is:
java.net.ConnectException: Connection timed out connecting to Catalina:type=ThreadPool,name=http-9080 by URL service:jmx:rmi:///jndi/rmi://172.18.1.91:19080/jmxrmijava.rmi.ConnectException: Connection refused to host: 172.18.1.91; nested exception is:
java.net.ConnectException: Connection timed out
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:129)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
at org.nagios.JMXQuery.connect(JMXQuery.java:53)
at org.nagios.JMXQuery.main(JMXQuery.java:75)
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

Ports are connected successfully:
[root@brxpdlibmonl01 Prod]# telnet 172.18.1.91 9080
Trying 172.18.1.91...
Connected to 172.18.1.91.
Escape character is '^]'.
^C^CConnection closed by foreign host.
[root@brxpdlibmonl01 Prod]# telnet 172.18.1.91 19080
Trying 172.18.1.91...
Connected to 172.18.1.91.
Escape character is '^]'.



Thanks,
Prabhas
Prabha2028
 
Posts: 2
Joined: Wed Jun 30, 2021 10:48 am

Re: Service check timed out after 60.01 seconds

Postby pbroste » Tue Jul 13, 2021 4:31 pm

Hello Prabhas,

Let's check to see if the ports are listening:

Code: Select all
sudo ss -tulwn | grep '9080|19080'


Also here is a line to test the check_jmx plugin:


Code: Select all
./check_jmx -U service:jmx:rmi:///jndi/rmi://localhost:7199/jmxrmi -O java.lang:type=Memory -A HeapMemoryUsage -K used -I HeapMemoryUsage -J used -vvvv -w 4248302272 -c 5498760192


Regards,
Perry
User avatar
pbroste
 
Posts: 524
Joined: Tue Jun 01, 2021 1:27 pm

Re: Service check timed out after 60.01 seconds

Postby Prabha2028 » Wed Jul 21, 2021 7:42 am

Hi Perry,

Thanks for your support.

I have excuted given commands and pasted the results below.

[root@brxpdlibmonl01 nagios]# sudo ss -tulwn | grep '9080|19080'
sudo: PAM account management error: Permission denied
[root@brxpdlibmonl01 nagios]# ./check_jmx -U service:jmx:rmi:///jndi/rmi://localhost:8080/jmxrmi -O java.lang:type=Memory -A HeapMemoryUsage -K used -I HeapMemoryUsage -J used -vvvv -w 4248302272 -c 5498760192
-bash: ./check_jmx: No such file or directory
[root@brxpdlibmonl01 nagios]# ./check_jmx -U service:jmx:rmi:///jndi/rmi://localhost:7199/jmxrmi -O java.lang:type=Memory -A HeapMemoryUsage -K used -I HeapMemoryUsage -J used -vvvv -w 4248302272 -c 5498760192
-bash: ./check_jmx: No such file or directory
[root@brxpdlibmonl01 nagios]#



Thanks,
Prabhas
Prabha2028
 
Posts: 2
Joined: Wed Jun 30, 2021 10:48 am

Re: Service check timed out after 60.01 seconds

Postby mcapra » Wed Jul 21, 2021 1:09 pm

Depending on the firewall in-use on the 172.18.1.91 machine, telnet may not be enough to test that the port is truly open and that the check_jmx traffic is not being filtered out by the firewall. Might try nmap or netcat instead. Both are great tools to have on your Nagios server for troubleshooting.

Code: Select all
yum install nmap nc
netcat -z -n -v 172.18.1.91 19080
nmap -sS -O -p19080 172.18.1.91


It would also be useful to see specifically how you're configuring the JMX endpoint within your Tomcat instance. Specifically, the values of com.sun.management.jmxremote.local.only and java.rmi.server.hostname. But there are other properties that could cause connection failures.
Former Nagios employee
https://www.mcapra.com/
User avatar
mcapra
 
Posts: 3725
Joined: Thu May 05, 2016 3:54 pm


Return to Community Support

Who is online

Users browsing this forum: No registered users and 20 guests