ERROR: Process name table : No response from remote host

[xlin125]

We are using Nagios XI 2014R2.7 to monitor Linux (CentOS 6.x) using SNMP. After replacing the old IP address with the new IP address in the "Address" field of the host under Config->Hosts for the host running SNMP agent, I noticed the SNMP monitors (services) failed randomly because the old IP address was polled with SNMP query from the Nagios XI server. The following is the error shown in the Status Information field. Note that '135.144.3.85' is an old IP address that has been replaced, and is offline.

ERROR: Process name table : No response from remote host '135.144.3.85'.

What is wrong and why this old IP address is still polled even it is replaced? Where is it kept? Is this a database issue?

Thanks in advance.

[rkennedy]

Can you check your service configurations to see if the IPs are hard coded? Are these active or passive checks?

[xlin125]

No, the service configuration is not hard-coded with IP address. Instead, it takes the IP address from $HOSTADDRESS.

For example, in the service configuration:
Command view:
$USER1$/check_snmp_storage_wizard.pl -H $HOSTADDRESS$ $ARG1$

The host is added to "Manage Hosts", and this host is added to host configuration with the new IP address hard-coded in the IP address field.

This is an active check: from Nagios XI server to the host that runs SNMP agent (agentless monitoring) using SNMP.

[hsmith]

How often is this happening? Was it working for a period of time and then changed, and is failing consistently? You didn't change servers and restore from a previous backup did you?

Can you verify the IP is right in the service configuration?

This is pretty strange.

[rkennedy]

Also, from the command line of the Nagios server - can you snmpwalk against the machine? What is the result?

[xlin125]

It looks like 50% chance going to the correct IP address and 50% chance going to the old IP address. Which IP address is snmp queried looks random.

We do not hard code the IP address in the service configuration. Instead, the IP address is obtained the host configuration.

The command line snmpwalk always works because we use the correct IP address.

We did not change servers and restore from a previous backup.

Yes, it is very strange! It only happens on this Nagios XI server. Other Nagios XI servers do not have this problem (I tested it with the same service configurations)

[lmiltchev]

This is indeed very strange! Check to see if you have multiple instances of nagios running:

Code: Select all

ps -ef | grep "bin/nagios"

Try stopping/starting services:

Code: Select all

service nagios stop
killall nagios
service ndo2db stop
service ndo2db start
service nagios start

[xlin125]

Yes, there were 141 instances of nagios running on this Nagios XI server. I followed your instructions to stop/restart services. Then the problem was gone.

I really appreciate your help and support! THANKS!!!

[lmiltchev]

It is hard to believe you had 141 instances of nagios running on the same XI server... For example, this:

nagios 1582 1 0 Jan22 ? 00:03:08 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 1584 1582 0 Jan22 ? 00:00:20 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios 1585 1582 0 Jan22 ? 00:00:20 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios 1586 1582 0 Jan22 ? 00:00:21 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios 1587 1582 0 Jan22 ? 00:00:20 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios 1601 1582 0 Jan22 ? 00:00:12 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg

would be 1 instance of nagios running (not 6) - same PID. I suspect you didn't counted the instances correctly.

Anyway, I am glad your issue, has been resolved! If you have any more issues/questions, please start a new thread.