check_snmp_storage.pl and ERROR: Description/Type table : No

[medleyb]

Error " ERROR: Description/Type table : No response from remote host " occurs about 50% of the time various VMs. And it is not tied to time of day (happens randomly across a 24 hour period) and only on 4 or 5 VMs.

I modified the script to post additional diagnostics to see where the issue was (though it was rather obvious) and got this back from 2 runs, one with diags and one w/o.

>>>/usr/local/nagios/libexec>>./check_snmp_storage.pl -H APP-AS-01 -C ARM1 --v2c -m ^C: -w 90 -c 95 -f -v
Alarm at 60
Connecting to host
SNMP v2c login
Connected...
Got rid of UTF8 translation in case of accentuated caracters
Getting descr_table
Filter : ^C:
OID : 1.3.6.1.2.1.25.2.3.1.3.3, Desc : D:\
OID : 1.3.6.1.2.1.25.2.3.1.3.5, Desc : Physical Memory
OID : 1.3.6.1.2.1.25.2.3.1.3.2, Desc : C:\ Label: Serial Number e45adf46
Name : C:\ Label: Serial Number e45adf46, Index : 2
OID : 1.3.6.1.2.1.25.2.3.1.3.4, Desc : Virtual Memory
OID : 1.3.6.1.2.1.25.2.3.1.3.1, Desc : A:\
storages selected : 1
1.3.6.1.2.1.25.2.3.1.6.2 : 11746595
1.3.6.1.2.1.25.2.3.1.5.2 : 14391551
1.3.6.1.2.1.25.2.3.1.4.2 : 4096
Descr : C:\ Label: Serial Number e45adf46
Size : 14391551
Used : 11746595
Alloc : 4096
Perf data : 'C:\_Label:__Serial_Number_e45adf46'=45885MB;50595;53406;0;56217
C:\ Label: Serial Number e45adf46: 82%used(45885MB/56217MB) (<90%) : OK | 'C:\_Label:__Serial_Number_e45adf46'=45885MB;50595;53406;0;56217
>>>
>>>/usr/local/nagios/libexec>>./check_snmp_storage.pl -H APP-AS-01 -C ARM1 --v2c -m ^C: -w 90 -c 95 -f

ERROR: Description/Type table : No response from remote host "APP-AS-01".

The firewall is not running on the windows server being queried and, since it runs, SNMP seems to be configured correctly. It does also fail with diagnostics in this code block. It is failing on the line in red:

if (defined ($o_index)){
verb("Getting index_table");
if (version->parse(Net::SNMP->VERSION) < 4) {
$resultat = $session->get_table($index_table);
} else {
$resultat = $session->get_table(Baseoid => $index_table);
}
} else {
verb("Getting descr_table");
if (version->parse(Net::SNMP->VERSION) < 4) {
$resultat = $session->get_table($descr_table);
} else {
$resultat = $session->get_table(Baseoid => $descr_table);
}
}

Any ideas?

Also, adding -t 60 gives this:

Alarm at 60
Connecting to host
SNMP v2c login
Connected...
Got rid of UTF8 translation in case of accentuated caracters
Getting descr_table ....and 60 seconds later we get...
ERROR: General time-out (Alarm signal)

[ssax]

Does it eventually finish if you crank up the timeout and run it manually from the command line?

Code: Select all

/usr/local/nagios/libexec/check_snmp_storage.pl -H APP-AS-01 -C ARM1 --v2c -m ^C: -w 90 -c 95 -f -v -t 300

[medleyb]

300 does not appear to be a valid value for this script:

/usr/local/nagios/libexec>>./check_snmp_storage.pl -H APP-AS-01 -C ARM1 --v2c -m ^C: -w 90 -c 95 -f -v -t 300
Alarm at 60
Connecting to host
SNMP v2c login
ERROR: The timeout value 300 is out of range (1..60).

That error is coming from Net::SNMP I think. It is not tested for or generated from check_snmp_storage.pl itself.

[ssax]

When that is failing can you run a standard SNMP walk just fine?

Code: Select all

snmpwalk -v 2c -c ARM1 APP-AS-01:161

Do you have any other SNMP checks against those hosts that are working properly?

[medleyb]

snmpwalk -v2c -cARM1 APP-AS-01:161 1.3.6.1.2.1.25.2.3.1.3 -t60
Timeout: No Response from APP-AS-01:161

And none of the SNMP hooks are working despite SNMP being configured correctly on the target.

[ssax]

If the snmpwalk is not working then it's either a firewall/IPS or some other security device blocking it OR SNMP is not configured/working properly on the remote system.

What is the output of this command on the XI server:

Code: Select all

nmap -sU -p 161 APP-AS-01

[medleyb]

Seems to be working (name and part of the IP address changed)

nmap -sU -p 161 APP-AS-01

Starting Nmap 6.47 ( http://nmap.org ) at 2018-10-23 07:10 CDT
Nmap scan report for APP-AS-01 (xxx.xxx.125.182)
Host is up (0.0027s latency).
rDNS record for xxx.xxx.125.182: app-as-01.xxx.com
PORT STATE SERVICE
161/udp open|filtered snmp
MAC Address: 00:50:56:A5:32:D3 (VMware)

Nmap done: 1 IP address (1 host up) scanned in 0.41 seconds

[ssax]

Have you reached out to your security/firewall team to see if they are seeing anything blocking it like an IPS or another intrusion prevention device/piece of software?

[medleyb]

I have done so and am awaiting their reply. Thanks all for the input!!

[ssax]

Let us know what they say, the interesting part is the "Timeout: No Response from APP-AS-01:161" when running the snmpwalk, it indicates it's either a SNMP community issue, a load issue on the remote system that isn't letting the snmp daemon respond, or a firewall/security device interrupting the connection, or possibly a network issue but I'm sure you'd see more indicators of this.