VMware ESXi SOAP request error

[marctonk]

Hi,

I am in an offline environment and have Nagios Xi setup. We are trying to get Nagios to monitor ESXi. When installing the vmware plugin it appears to work just fine, everything is monitoring. However we see critical alerts coming and going. Tailing the nagios.log we see a lot of "CHECK_VMWARE_API.PL Critical SOAP request errors - Possibly a protocol issue".

Here are the versions we are running:
vCenter 6.5.0
ESXi 6.0 / 5.5
NagiosXI 5.5.11 on Redhat 7.6
VMware-vSphere-Perl-SDK-6.5.0-4566394

Here is how it was installed:
export PERL_MM_USE_DEFAULT=1
yum install -y --nongpgcheck libxml2-devel libuuid-devel perl-xml-libxml xml2 perl-env openssl-devel
./vmware-install.pl

We did have a timeout issue where each ESXi service would timeout. We changed the timeout from 60 to 300 and that issue has went away.

Any help would be greatly appreciated.

[marctonk]

UPDATE:

I ran the following command as root and nagios to see if it was some type of permission issue.

As root:
/usr/local/nagios/libexec/check_vmware_api.pl -H "ipofesxihost" -f "/usr/local/nagiosxi/etc/components/vmware/esxiauthinfo.txt" -l "CPU" -v

CHECK_VMWARE_API.PL OK - cpu usage=5555.00 MHz (18.56%) | cpu_usagemhz=5555.00;; cpu_usage=18.56%;;

As nagios:
/usr/local/nagios/libexec/check_vmware_api.pl -H "ipofesxihost" -f "/usr/local/nagiosxi/etc/components/vmware/esxiauthinfo.txt" -l "CPU" -v

CHECK_VMWARE_API.PL OK - cpu usage=5861.00 MHz (19.58%) | cpu_usagemhz=5861.00;; cpu_usage=19.58%;;

I read from some older articles that perl-SOAP-Lite was potentially needed. I went ahead and installed that and reboot. After waiting for about 20 minutes the error "CHECK_VMWARE_API.PL Critical SOAP request errors - Possibly a protocol issue" is still there. I will monitor for any other error's that may help.

More to come.

[tgriep]

Sometimes the libwww perl module needs to be downgraded for compatibility issues.
Get this file and put it the /tmp folder on the nagios server.
http://search.cpan.org/CPAN/authors/id/ ... 837.tar.gz

Run the following as root in the Nagios server to downgrade that module.

Code: Select all

cd /tmp
tar xvfz libwww-perl-5.837.tar.gz
cd libwww-perl-5.837
perl Makefile.PL -l
make
make install

After this, run the plugin commands to see if they function.

[marctonk]

Sometimes the libwww perl module needs to be downgraded for compatibility issues.
Get this file and put it the /tmp folder on the nagios server.
http://search.cpan.org/CPAN/authors/id/ ... 837.tar.gz

Run the following as root in the Nagios server to downgrade that module.

Code: Select all

cd /tmp
tar xvfz libwww-perl-5.837.tar.gz
cd libwww-perl-5.837
perl Makefile.PL -l
make
make install

After this, run the plugin commands to see if they function.

Complete, I still see the error's after a reboot.

Doing a yum list | grep libwww I see perl-libwww-perl.noarch 6.05-2.el7. I am not sure how to see if this was successful on the downgrade. The output from the make install was successful.

[tgriep]

The errors are intermittent, correct?

[marctonk]

The errors are intermittent, correct?

Correct. I get the usual messages when an actual event happens but every few minutes (10-30 min) I get the SOAP request protocol error. This causes a critical alert to show up on the dashboard for a minute then it goes away.

[tgriep]

Let's see how long it takes to run the check.
Login to the server, switch to the nagios user and run this command. Run it a couple of times for 30 minutes to see it the time it takes to run varies.

Code: Select all

time /usr/local/nagios/libexec/check_vmware_api.pl -H "ipofesxihost" -f "/usr/local/nagiosxi/etc/components/vmware/esxiauthinfo.txt" -l "CPU" -v -t 300

[marctonk]

Hi,

I ran the command for 30 minutes, 3 times. Real, and User never went above 0m0.8XXs. Sys never went above 0m0.05X. I was tailing the nagios.log also and when the SOAP error hit I did not notice any uptick from any of the 3 outputs.

Error Message: SERVICE ALERT: hostname; service_being_monitored; CRTICAL; SOFT; 1; CHECK_VMWARE_API.PL CRITICAL - SOAP request error - possibly a protocol issue.

Thank you,
Marcus

[tgriep]

Thanks for running the commands and noting the time they took to run.
It's bad that it did not show anything.

Do you see any errors in the log files on the VMWare servers at the time the check fails?

[marctonk]

Hi,

I was hoping to see a long pause or wait that we could have pointed our efforts toward.

Looking at the ESXi logs here is what I notice:

vmksummary.log: Nothing to note.

vmkwarning.log: Nothing to note.

hostd.log: I see constant messages:
Accepted password for user root @ NagiosIP
User root @ NagiosIP logged in as VI Perl
User root @ NagiosIP logged out

Any other log you want me to look at?

Also do you have any other methods with documentation that I could monitor ESXi hosts with? Maybe I need to try a different approach since the plugin is not working as expected. Or do you have a complete offline build that has everything I need in it