Page 1 of 1
Storage Sizes not Updating CHECK_VMWARE_API
Posted: Thu Sep 23, 2021 9:29 am
by Sargento
We have a "Datastore usage for VMHost" check, we noticed in vmware that we had a LUN filling up but XI did not alert us. I waited a while, restarted the check a few dozen times and BAM, alerted, much too late and with too much manual intervention required.
Our team went in and resolved the sizing alert, yet the VMHost alert stayed red. I let it run over night, expecting another long delay, and I come to this this morning.
Code: Select all
'OUR LUN NAME'(free)=20560.00 MB (1.63%)
The actual usage on the disk is back to a safe amount (like 50% used or so) but XI does not reflect it. Why is this?
Here is the actual check.
Code: Select all
/usr/local/nagios/libexec/check_vmware_api.pl -H "host omitted" -f /path/to/creds.file -l "VMFS" -w 5%: -c 3%:
What is the missing piece here? We are trying to use XI and it's a hard sell sometimes when things don't seem to work right out of the box.
Thanks
Re: Storage Sizes not Updating CHECK_VMWARE_API
Posted: Fri Sep 24, 2021 10:56 am
by ssax
Does the check show anything different if you run the check from the CLI?
Code: Select all
/usr/local/nagios/libexec/check_vmware_api.pl -H "host omitted" -f /path/to/creds.file -l "VMFS" -w 5%: -c 3%:
If not, the sizes are being returned by your vmware system and vmware is serving the wrong information. Maybe you need to do this:
https://kb.vmware.com/s/article/52985
Re: Storage Sizes not Updating CHECK_VMWARE_API
Posted: Tue Sep 28, 2021 12:50 pm
by Sargento
When I ran this check we still got "CRITICAL" though the device in question is no longer at critical levels. This is occurring almost a week after the size was reclaimed.
Re: Storage Sizes not Updating CHECK_VMWARE_API
Posted: Tue Sep 28, 2021 5:22 pm
by ssax
Given the information is being served by VMware the recommendation would be for you to reach out to their support team to determine what the resolution is as it would be a VMware issue.
Re: Storage Sizes not Updating CHECK_VMWARE_API
Posted: Wed Sep 29, 2021 2:16 pm
by Sargento
Alrighty.
Re: Storage Sizes not Updating CHECK_VMWARE_API
Posted: Thu Sep 30, 2021 10:51 am
by Sargento
VMWare said that it is an issue with the CHECK_VMWARE_API code, which is not theirs and they do not support. They verified that the sizes are correct from the VMWare side and told me that you should support me to a resolution. I reran the command again..
Code: Select all
/usr/local/nagios/libexec/check_vmware_api.pl -H "host" -f/path/to/creds.file -l "VMFS" -w 5%: -c 3%:
Which again showed as critical. I went in to VSphere with the VMWare support team and they showed me that in fact nothing was critical. So the issue is with the check_vmware_api.pl script itself. It is getting numbers wrong. Is there possibly a timeout or another parameter that would force a refresh of data? I'm thinking perhaps something is getting cached to save on login attempts. Is there anything you can think of here?
The reason I ask is due to the output of
check_vmware_api.pl -h to get help. I see this output down the screen a ways.. (specifically the T (value) portions)
Code: Select all
vmfs - shows Datastore info
+ (name) - free space info for datastore with name (name)
o used - output used space instead of free
o brief - list only alerting volumes
o regexp - whether to treat name as regexp
o blacklistregexp - whether to treat blacklist as regexp
b - blacklist VMFS's
T (value) - timeshift to detemine if we need to refresh
^ all datastore info
o used - output used space instead of free
o brief - list only alerting volumes
o blacklistregexp - whether to treat blacklist as regexp
b - blacklist VMFS's
T (value) - timeshift to detemine if we need to refresh
Re: Storage Sizes not Updating CHECK_VMWARE_API
Posted: Thu Sep 30, 2021 4:39 pm
by benjaminsmith
Hi,
Thanks for the update on this, I'm going to reach out to one of our developers on this and, I will follow up shortly.
Regards,
Benjamin
Re: Storage Sizes not Updating CHECK_VMWARE_API
Posted: Mon Oct 04, 2021 7:28 am
by Sargento
Alright.
Re: Storage Sizes not Updating CHECK_VMWARE_API
Posted: Mon Oct 04, 2021 4:25 pm
by benjaminsmith
Hi,
The developer is looking into this, I should have an update shortly. Thanks, Benjamin
Re: Storage Sizes not Updating CHECK_VMWARE_API
Posted: Tue Oct 05, 2021 10:11 am
by swolf
Followed up via PM