Check_esx3.pl timeout - 07/14/16

[brian.diedrich]

I just built a new Nagios XI 5.2.8 server in Azure. I exported the data from a previous server to this new one, as I created a new CentOS 7.2 image that utilizes LVM for the file systems and includes swap space as well (using a D3_v2 VM to run Nagios XI).
With the advent of the Nagios XI migration, none of the checks for our existing VMWare hosts are working. I have pillaged the forums, and have already added the updated check_vmware_api.pl and renamed it to check_esx3.pl.
I have tried 4 different versions of the VMWare sdk. 5.1, 5.5, 5.5 update2, and 6 (5.5 is the version that was working on the previous install).
/usr/lib/vmware-vcli/apps/general/connect.pl works just fine.
/usr/lib/vmware-vcli/apps/host/hostinfo.pl times out
nagios.log only tells me check timed out after 60.02 seconds

System info:
CentOS Linux release 7.2.1511 (Core)
Nagios XI 5.2.8 - manual install, and using imported data from another fully functional Nagios XI server.
My guess at this point is there is likely a version issue involved somewhere, but whether it is a linux package or perl module I can't be certain.
Let me know what other information you may require, and I will provide it for you.

[tgriep]

Can you rerun the hostinfo.pl test with the --verbose option and post the output here?

Code: Select all

hostinfo.pl --url https://xxx.xxx.xxx.xxx/sdk/vimService --username <username> --password <password> --verbose

[brian.diedrich]

Code: Select all

You have mail in /var/spool/mail/root
[root@IN8MGTNAGIOS001 host]# ./hostinfo.pl --url https://<hostname>/sdk/vimService --username <username> --password <"password"> --verbose  

REQUEST: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                     xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Body>
<RetrieveServiceContent xmlns="urn:vim25"><_this type="ServiceInstance">ServiceInstance</_this>
</RetrieveServiceContent></soapenv:Body></soapenv:Envelope>
';

RESPONSE: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
 xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<RetrieveServiceContentResponse xmlns="urn:vim25"><returnval><rootFolder type="Folder">ha-folder-root</rootFolder><propertyCollector type="PropertyCollector">ha-property-collector</propertyCollector><viewManager type="ViewManager">ViewManager</viewManager><about><name>VMware ESXi</name><fullName>VMware ESXi 5.0.0 build-623860</fullName><vendor>VMware, Inc.</vendor><version>5.0.0</version><build>623860</build><localeVersion>INTL</localeVersion><localeBuild>000</localeBuild><osType>vmnix-x86</osType><productLineId>embeddedEsx</productLineId><apiType>HostAgent</apiType><apiVersion>5.0</apiVersion><licenseProductName>VMware ESX Server</licenseProductName><licenseProductVersion>5.0</licenseProductVersion></about><setting type="OptionManager">HostAgentSettings</setting><userDirectory type="UserDirectory">ha-user-directory</userDirectory><sessionManager type="SessionManager">ha-sessionmgr</sessionManager><authorizationManager type="AuthorizationManager">ha-authmgr</authorizationManager><perfManager type="PerformanceManager">ha-perfmgr</perfManager><eventManager type="EventManager">ha-eventmgr</eventManager><taskManager type="TaskManager">ha-taskmgr</taskManager><accountManager type="HostLocalAccountManager">ha-localacctmgr</accountManager><diagnosticManager type="DiagnosticManager">ha-diagnosticmgr</diagnosticManager><licenseManager type="LicenseManager">ha-license-manager</licenseManager><searchIndex type="SearchIndex">ha-searchindex</searchIndex><fileManager type="FileManager">ha-nfc-file-manager</fileManager><virtualDiskManager type="VirtualDiskManager">ha-vdiskmanager</virtualDiskManager><ovfManager type="OvfManager">ha-ovf-manager</ovfManager><dvSwitchManager type="DistributedVirtualSwitchManager">ha-dvsmanager</dvSwitchManager><localizationManager type="LocalizationManager">ha-l10n-manager</localizationManager><storageResourceManager type="StorageResourceManager">ha-storage-resource-manager</storageResourceManager><guestOperationsManager type="GuestOperationsManager">ha-guest-operations-manager</guestOperationsManager></returnval></RetrieveServiceContentResponse>
</soapenv:Body>
</soapenv:Envelope>';


REQUEST: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                     xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Body>
<Login xmlns="urn:vim25"><_this type="SessionManager">ha-sessionmgr</_this>
<userName>(username)</userName><password>(password)</password></Login></soapenv:Body></soapenv:Envelope>
';

RESPONSE: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
 xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<LoginResponse xmlns="urn:vim25"><returnval><key>52103a48-8846-c600-926a-90782fb46704</key><userName>briandiedrich</userName><fullName>briandiedrich</fullName><loginTime>1970-02-14T03:00:14.884087Z</loginTime><lastActiveTime>1970-02-14T03:00:14.884087Z</lastActiveTime><locale>en</locale><messageLocale>en</messageLocale><extensionSession>false</extensionSession></returnval></LoginResponse>
</soapenv:Body>
</soapenv:Envelope>';


REQUEST: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                     xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Body>
<RetrieveProperties xmlns="urn:vim25"><_this type="PropertyCollector">ha-property-collector</_this>
<specSet><propSet><type>HostSystem</type><all>0</all></propSet><objectSet><obj type="Folder">ha-folder-root</obj>
<skip>0</skip><selectSet xsi:type="TraversalSpec"><name>folderTraversalSpec</name><type>Folder</type><path>childEntity</path><skip>0</skip><selectSet><name>folderTraversalSpec</name></selectSet><selectSet><name>datacenterHostTraversalSpec</name></selectSet><selectSet><name>datacenterVmTraversalSpec</name></selectSet><selectSet><name>datacenterDatastoreTraversalSpec</name></selectSet><selectSet><name>datacenterNetworkTraversalSpec</name></selectSet><selectSet><name>computeResourceRpTraversalSpec</name></selectSet><selectSet><name>computeResourceHostTraversalSpec</name></selectSet><selectSet><name>hostVmTraversalSpec</name></selectSet><selectSet><name>resourcePoolVmTraversalSpec</name></selectSet></selectSet><selectSet xsi:type="TraversalSpec"><name>datacenterDatastoreTraversalSpec</name><type>Datacenter</type><path>datastoreFolder</path><skip>0</skip><selectSet><name>folderTraversalSpec</name></selectSet></selectSet><selectSet xsi:type="TraversalSpec"><name>datacenterNetworkTraversalSpec</name><type>Datacenter</type><path>networkFolder</path><skip>0</skip><selectSet><name>folderTraversalSpec</name></selectSet></selectSet><selectSet xsi:type="TraversalSpec"><name>datacenterVmTraversalSpec</name><type>Datacenter</type><path>vmFolder</path><skip>0</skip><selectSet><name>folderTraversalSpec</name></selectSet></selectSet><selectSet xsi:type="TraversalSpec"><name>datacenterHostTraversalSpec</name><type>Datacenter</type><path>hostFolder</path><skip>0</skip><selectSet><name>folderTraversalSpec</name></selectSet></selectSet><selectSet xsi:type="TraversalSpec"><name>computeResourceHostTraversalSpec</name><type>ComputeResource</type><path>host</path><skip>0</skip></selectSet><selectSet xsi:type="TraversalSpec"><name>computeResourceRpTraversalSpec</name><type>ComputeResource</type><path>resourcePool</path><skip>0</skip><selectSet><name>resourcePoolTraversalSpec</name></selectSet><selectSet><name>resourcePoolVmTraversalSpec</name></selectSet></selectSet><selectSet xsi:type="TraversalSpec"><name>resourcePoolTraversalSpec</name><type>ResourcePool</type><path>resourcePool</path><skip>0</skip><selectSet><name>resourcePoolTraversalSpec</name></selectSet><selectSet><name>resourcePoolVmTraversalSpec</name></selectSet></selectSet><selectSet xsi:type="TraversalSpec"><name>hostVmTraversalSpec</name><type>HostSystem</type><path>vm</path><skip>0</skip><selectSet><name>folderTraversalSpec</name></selectSet></selectSet><selectSet xsi:type="TraversalSpec"><name>resourcePoolVmTraversalSpec</name><type>ResourcePool</type><path>vm</path><skip>0</skip></selectSet></objectSet></specSet></RetrieveProperties></soapenv:Body></soapenv:Envelope>
';

RESPONSE: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
 xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<RetrievePropertiesResponse xmlns="urn:vim25"><returnval><obj type="HostSystem">ha-host</obj></returnval></RetrievePropertiesResponse>
</soapenv:Body>
</soapenv:Envelope>';


REQUEST: $VAR1 = '<?xml version="1.0" encoding="UTF-8"?>
   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                     xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Body>
<RetrieveProperties xmlns="urn:vim25"><_this type="PropertyCollector">ha-property-collector</_this>
<specSet><propSet><type>HostSystem</type><all>1</all></propSet><objectSet><obj type="HostSystem">ha-host</obj>
</objectSet></specSet></RetrieveProperties></soapenv:Body></soapenv:Envelope>
';

and this is where it hangs

[brian.diedrich]

I ran the command again with date before and after and just let it go. 3 minutes and 40 seconds to get a return. Far too long to be a useful check really.

[Box293]

Try connecting with the FQDN instead of the IP address.

[brian.diedrich]

using FQDN, shortname alias, or IP address the results are the same.

[Box293]

From what I understand port 443 is how the VMware SDK communicates.

What is the output of this command an your Nagios XI server:

Code: Select all

nmap -Pn -p T:443 xxx.xxx.xxx.xxx

This will test to see if a connection can be made.

[brian.diedrich]

I know the port is open and it works. Like I stated initially, I exported the config from my "old" working Nagios XI to a new and improved server. I expect there is some sort of version conflict somewhere.
[root@IN8MGTNAGIOS001 tmp]# nmap -Pn -p T:443 IP Address
Starting Nmap 6.47 ( http://nmap.org ) at 2016-07-14 22:04 AZOST
Nmap scan report for hostname (IP Address)
Host is up (0.081s latency).
PORT STATE SERVICE
443/tcp open https

Nmap done: 1 IP address (1 host up) scanned in 1.17 seconds
You have mail in /var/spool/mail/root

[Box293]

Can I get you to confirm that you installed the SDK following these steps:

https://assets.nagios.com/downloads/nag ... ios-XI.pdf

Note the CentOS 7 steps are different to the CentOS 6 steps.

I just tested this on CentOS 7.2 using VMware-vSphere-Perl-SDK-6.0.0-3561779.x86_64.tar.gz and the hostinfo.pl test ran fine.

[brian.diedrich]

I did indeed use the doc to install the SDK's, including the sed -i for Centos 7. Our ESXi hosts are running different versions, and when I set it up initially, the 5.5 SDK is the only version I could get to work with all of them. The hostinfo.pl does work, sort of. It just takes an exceedingly long time to return (better than 3 and a half minutes).