SNMP error with snmp version 1 (No response from remote host

[mkhan12282]

Nagios Core: 4.0.1

We have nagios polling some of the cisco servers service checks for call manager, unity and one attempt shows this error, and the next attempt at the next minute is successfull. This mainly happening for the add on: check_hr_process

[06-05-2015 12:28:29] SERVICE ALERT: MY.HOST.X;sys-proc-hostagt;OK;SOFT;2;OK: 1 process(es) running
[06-05-2015 12:27:32] SERVICE ALERT: MY.HOST.X;sys-proc-hostagt;CRITICAL;SOFT;1;CRITICAL: SNMP error with snmp version 1 (No response from remote host "172.16.241.14")
[06-05-2015 12:13:57] SERVICE ALERT: MY.HOST.X;sys-proc-CuFileSync;OK;SOFT;2;OK: 1 process(es) running
[06-05-2015 12:13:04] SERVICE ALERT: MY.HOST.X;sys-proc-CuFileSync;CRITICAL;SOFT;1;CRITICAL: SNMP error with snmp version 1 (No response from remote host "172.16.241.14")
[06-05-2015 12:08:41] SERVICE ALERT: MY.HOST.X;sys-proc-AuditLog;OK;SOFT;2;OK: 1 process(es) running
[06-05-2015 12:07:47] SERVICE ALERT: MY.HOST.X;sys-proc-AuditLog;CRITICAL;SOFT;1;CRITICAL: SNMP error with snmp version 1 (No response from remote host "172.16.241.14")

When we check the same service from nagios command line, each attempt is successful.
./check_hr_process -v 1 -H 172.16.241.14 -C tiger -w 1 -c 1 -d hostagt -t 60

[root@NAGIOS01 libexec]# ./check_hr_process -v 1 -H 172.16.241.14 -C tiger -w 1 -c 1 -d hostagt -t 60
OK: 1 process(es) running
[root@NAGIOS01 libexec]# ./check_hr_process -v 1 -H 172.16.241.14 -C tiger -w 1 -c 1 -d hostagt -t 60
OK: 1 process(es) running
[root@NAGIOS01 libexec]# ./check_hr_process -v 1 -H 172.16.241.14 -C tiger -w 1 -c 1 -d hostagt -t 60
OK: 1 process(es) running

Ping test from nagios to the servers shows no packet loss.
[root@NAGIOS01 libexec]# ping 172.16.241.14 -s 1200 -c 5
PING 172.16.241.14 (172.16.241.14) 1200(1228) bytes of data.
1208 bytes from 172.16.241.14: icmp_seq=1 ttl=60 time=10.9 ms
1208 bytes from 172.16.241.14: icmp_seq=2 ttl=60 time=19.4 ms
1208 bytes from 172.16.241.14: icmp_seq=3 ttl=60 time=17.5 ms
1208 bytes from 172.16.241.14: icmp_seq=4 ttl=60 time=13.0 ms
1208 bytes from 172.16.241.14: icmp_seq=5 ttl=60 time=17.4 ms

--- 172.16.241.14 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4023ms
rtt min/avg/max/mdev = 10.988/15.678/19.472/3.157 ms

We have also set the timeout value to -t in checkcommands.cfg as below.[/b]

define command {
command_name check_hr_process
command_line $USER1$/check_hr_process -v 1 -H $HOSTADDRESS$ -C $ARG1$ -w $ARG2$ -c $ARG3$ -d $ARG4$ -t 60
}

Any advice where should (if any) timeout settings to be modified in nagios?
Thank you.

[lmiltchev]

Have you tried pinging the device and/or running the snmpwalk right after the check failed? Have you tried using v2 instead of 1, provided your device and the plugin supports it?

You can try increasing the timeout a bit to see if this is going to help. In addition to modifying the timeout value in the command, you will have to modify the service_check_timeout value in the nagios.cfg:

Code: Select all

service_check_timeout=

[mkhan12282]

Thanks for the input.

OK, so...
Changed to snmp coummunity to v2.
set timeout (as you suggested) to 120 (previously it was 60) in both locations.
I started ping to 3 hosts and collecting the result in the background i.e. nohup

I would let you know how this goes.
Thank you.

[mkhan12282]

For the past 1 hour, no issues. Don't know if that is also due to the continuous ping to the 3 hosts.
Let me stop the pings and see if nagios reports any issues.
I will keep you posted.
Thank you.

[lmiltchev]

No problem - we will keep this post open for a while. Let us know if the issue reappears.

[mkhan12282]

Hi lmiltchev

Deffinately, i can see lot of improvement so thank you on that.
But noticed the following/attached (given below is for one host as example) that after 9am i can see the following. (nothing between 12am - 09am) I could also see the similar for other host/services. I can think of if it could be due to intermittent packet loss on the network. I will run another ping test to see how this goes during the day.

nagios host alert history

[jdalrymple]

Or potentially the host is super busy and unable to process SNMP requests in time. Sounds ridiculous, but could very well be.

Are you monitoring this system's load?

[mkhan12282]

No packet loss found via continuous ping from nagios cli to the 3 hosts.
Load is not monitored, but will look into to put this in place.

Thank you both of you for tremendous support. I can say that 99% of the issue has been fixed.
Ta..

[ssax]

Great, can we mark this as resolved and lock the thread or do you want us to leave it open for a few days?

[mkhan12282]

We are good to close it.
Thank you.
MK