Page 1 of 1

wmi_Plus Check Memory Above

Posted: Tue Apr 04, 2017 12:40 pm
by kgugerty
Good Afternoon,

I have to monitor processes for memory usage to ensure there is no spikes for specific services. I setup the WMI_Plus check to warn about any process exceeding 1GB of memory. The issue is when I use a '%' to catch ANY process, it includes the '_Total' value, which is ALWAYS over 1GB.
Has anyone found a way to successfully suppress the _Total value?

I also put in a request at http://edcint.co.nz/checkwmiplus/ to see if they had any way to suppress the value.

Re: wmi_Plus Check Memory Above

Posted: Tue Apr 04, 2017 12:47 pm
by mcapra
Can you share the specific syntax you're using with check_wmi_plus? It might help identify places in the plugin where changes could be made.

Re: wmi_Plus Check Memory Above

Posted: Tue Apr 04, 2017 1:42 pm
by kgugerty

Code: Select all

/usr/local/nagios/libexec/check_wmi_plus.pl -H host.address -u domain/user -p Password-m checkproc -s memoryabove -a % -w WorkingSet=1G -c WorkingSet=1500M -exc WorkingSet=@0:100M --NODATAEXIT 0 
WARNING - [Triggered by WorkingSet>1G] - Total Process Count=4 (Process details on next line)|'Process Count'=4; 'PrivateMemory_svchost#3'=272752640Bytes; 'TotalWorkingSet_svchost#3'=155025408Bytes;1073741824;1572864000; 'VirtualMemory_svchost#3'=715337728Bytes; 'PrivateMemory_Ssms#7'=105799680Bytes; 'TotalWorkingSet_Ssms#7'=113905664Bytes;1073741824;1572864000; 'VirtualMemory_Ssms#7'=453918720Bytes; 'PrivateMemory_Ssms#8'=138158080Bytes; 'TotalWorkingSet_Ssms#8'=147619840Bytes;1073741824;1572864000; 'VirtualMemory_Ssms#8'=476217344Bytes; 'PrivateMemory__Total'=2732363776Bytes; 'TotalWorkingSet__Total'=1246187520Bytes;1073741824;1572864000; 'VirtualMemory__Total'=13179531264Bytes;
OK - svchost#3: Private Memory=260.117MB, Working Set=147.844MB, Virtual Memory=682.199MB\nOK - Ssms#7: Private Memory=100.898MB, Working Set=108.629MB, Virtual Memory=432.891MB\nOK - Ssms#8: Private Memory=131.758MB, Working Set=140.781MB, Virtual Memory=454.156MB\nWARNING - [Triggered by WorkingSet>1G] - _Total: Private Memory=2.545GB, Working Set=1.161GB, Virtual Memory=12.274GB\n

Re: wmi_Plus Check Memory Above

Posted: Tue Apr 04, 2017 3:51 pm
by mcapra
Can you share that commands output with -d enabled to show the debug output? I'm wondering how -a ties into the WQL query run.

Re: wmi_Plus Check Memory Above

Posted: Wed Apr 05, 2017 11:01 am
by kgugerty
Can you share that commands output with -d enabled to show the debug output? I'm wondering how -a ties into the WQL query run.
This might answer your question a little better. In the code block below -a equates to ARG1, which is essentially the name of the process or processes you are looking for. It accepts % as a wildcard so specifying % should pull ALL processes. About halfway down the code block you will see the actual query defined. I did attempt to modify the query with

Code: Select all

 WHERE Name like "{_arg1}" and Name not like "_Total" 
to see if that would exclude the _Total results.

Code: Select all

#----------------------------------------------------------
[checkproc memoryabove]
requires=1.48
inihelp=<<EOT
Check for processes using more that a specified amount of memory. To make this work as intended you need to specify some
warning/critical criteria eg -w WorkingSet=50M for warning when a process uses more than 50M of WorkingSet memory. You probably also want to remove 
all processes with memory lower than the warning level. Do this using something like -exc WorkingSet=@0:40M (which will exclude processes that have a WorkingSet between 0 and 40M). We could have made that -exc WorkingSet=@0:50M but then the check will only show processes when there are some to warn against. That might be what you want. Instead of WorkingSet, you can also use PrivateBytes or VirtualBytes. You can in fact use more than one of them at the same time if you wish eg -exc WorkingSet=@0:50M -exc VirtualBytes=@0:50M.
ARG1  The processname to look for. Use % for wildcards.
   The process name typically only includes the actual file name minus its suffix eg firefox, svchost
   If there are multiple instances eg svchost, then some versions of Windows have them named all the same while others
   such as Windows 2008 Server, have them numbered eg svchost#1, svchost#2, svchost#3. To get all svchost processes you
   need to set ARG1 to svchost%
   To view all processes set ARG1 to "%" and the full process list will be included in the plugin output.

Note:  Use --nodatamode and/or NODATAEXIT settings to control what happens if no matching process is found.
EOT

aligndata=Name,IDProcess
query=select Name,IDProcess,PrivateBytes,VirtualBytes,WorkingSet from Win32_PerfFormattedData_PerfProc_Process WHERE Name like "{_arg1}"

# calculate sums for these fields
calc=PrivateBytes,VirtualBytes,WorkingSet

test=_ItemCount
test=PrivateBytes
test=WorkingSet
test=VirtualBytes

predisplay=_DisplayMsg||~|~| - ||
predisplay=_ItemCount||Total Process Count|||| (Process details on next line)\n

display=_DisplayMsg||~|~| - ||
display=PrivateBytes|#B|{Name}: Private Memory||||
display=WorkingSet|#B|Working Set||||
display=VirtualBytes|#B|Virtual Memory||~||\n

# need to include the {Name} so that performance data is unique to each instance
perf=_ItemCount||Process Count
perf=PrivateBytes|Bytes|PrivateMemory_{Name}
perf=WorkingSet|Bytes|TotalWorkingSet_{Name}
perf=VirtualBytes|Bytes|VirtualMemory_{Name}

#----------------------------------------------------------
After reading through the pl file it appears the _Total value is calculated, but I have not been able to figure out how to remove the calculation. I did comment out the final line below as a hopeful guess.

Code: Select all

# load some values to check warn/crit against
$collected_data[$last_wmi_data_index][0]{'_NumGood'}=$num_ok;
$collected_data[$last_wmi_data_index][0]{'_NumBad'}=$num_bad;
$collected_data[$last_wmi_data_index][0]{'_NumExcluded'}=$num_excluded;
$collected_data[$last_wmi_data_index][0]{'_Total'}=$num_ok+$num_bad;

Re: wmi_Plus Check Memory Above

Posted: Wed Apr 05, 2017 4:17 pm
by mcapra
You might try altering the memoryabove query in check_wmi_plus.ini to look like this:

Code: Select all

select Name,IDProcess,PrivateBytes,VirtualBytes,WorkingSet from Win32_PerfFormattedData_PerfProc_Process WHERE Name like "{_arg1}" AND Name != "_Total"
That seems to be working for me:

Code: Select all

[root@nls1 libexec]# python send_wql.py -H 192.168.67.99 -u admin -p welcome123 -q 'select Name,IDProcess,PrivateBytes,VirtualBytes,WorkingSet from Win32_PerfFormattedData_PerfProc_Process WHERE Name like "%" AND Name != "_Total";'
{'WorkingSet': 17608704, 'VirtualBytes': 1235480576, 'Name': 'certsrv', 'PrivateBytes': 12447744, 'IDProcess': 1400}
{'WorkingSet': 4489216, 'VirtualBytes': 52207616, 'Name': 'csrss', 'PrivateBytes': 2052096, 'IDProcess': 372}
{'WorkingSet': 60428288, 'VirtualBytes': 246972416, 'Name': 'csrss#1', 'PrivateBytes': 2387968, 'IDProcess': 5048}
{'WorkingSet': 3702784, 'VirtualBytes': 44339200, 'Name': 'csrss#2', 'PrivateBytes': 1298432, 'IDProcess': 6108}
{'WorkingSet': 33767424, 'VirtualBytes': 698740736, 'Name': 'dfsrs', 'PrivateBytes': 28676096, 'IDProcess': 1488}
{'WorkingSet': 6844416, 'VirtualBytes': 32653312, 'Name': 'dfssvc', 'PrivateBytes': 3153920, 'IDProcess': 2644}
{'WorkingSet': 10612736, 'VirtualBytes': 50049024, 'Name': 'dllhost', 'PrivateBytes': 3309568, 'IDProcess': 3568}
{'WorkingSet': 11046912, 'VirtualBytes': 98881536, 'Name': 'dllhost#1', 'PrivateBytes': 2338816, 'IDProcess': 5808}
{'WorkingSet': 81850368, 'VirtualBytes': 213770240, 'Name': 'dwm', 'PrivateBytes': 26505216, 'IDProcess': 5132}
{'WorkingSet': 20463616, 'VirtualBytes': 96591872, 'Name': 'dwm#1', 'PrivateBytes': 13942784, 'IDProcess': 6844}
{'WorkingSet': 172806144, 'VirtualBytes': 759500800, 'Name': 'explorer', 'PrivateBytes': 88629248, 'IDProcess': 5608}
{'WorkingSet': 24576, 'VirtualBytes': 65536, 'Name': 'Idle', 'PrivateBytes': 0, 'IDProcess': 0}
{'WorkingSet': 12783616, 'VirtualBytes': 54800384, 'Name': 'inetinfo', 'PrivateBytes': 6270976, 'IDProcess': 1516}
{'WorkingSet': 88219648, 'VirtualBytes': 811147264, 'Name': 'InetMgr', 'PrivateBytes': 55275520, 'IDProcess': 10476}
{'WorkingSet': 4468736, 'VirtualBytes': 26390528, 'Name': 'ismserv', 'PrivateBytes': 1683456, 'IDProcess': 1536}
{'WorkingSet': 12541952, 'VirtualBytes': 111091712, 'Name': 'jucheck', 'PrivateBytes': 2895872, 'IDProcess': 5308}
{'WorkingSet': 14270464, 'VirtualBytes': 104579072, 'Name': 'jusched', 'PrivateBytes': 4407296, 'IDProcess': 5264}
{'WorkingSet': 25919488, 'VirtualBytes': 142282752, 'Name': 'LogonUI', 'PrivateBytes': 9977856, 'IDProcess': 6700}
{'WorkingSet': 62189568, 'VirtualBytes': 1286012928, 'Name': 'lsass', 'PrivateBytes': 49721344, 'IDProcess': 532}
{'WorkingSet': 31404032, 'VirtualBytes': 614727680, 'Name': 'MediaStreamingProvider', 'PrivateBytes': 39219200, 'IDProcess': 1304}
{'WorkingSet': 63733760, 'VirtualBytes': 607174656, 'Name': 'Microsoft.ActiveDirectory.WebServices', 'PrivateBytes': 51351552, 'IDProcess': 1344}
{'WorkingSet': 7241728, 'VirtualBytes': 42856448, 'Name': 'msdtc', 'PrivateBytes': 2613248, 'IDProcess': 3696}
{'WorkingSet': 41164800, 'VirtualBytes': 120856576, 'Name': 'ncpa_listener', 'PrivateBytes': 33878016, 'IDProcess': 4668}
{'WorkingSet': 32481280, 'VirtualBytes': 106237952, 'Name': 'ncpa_passive', 'PrivateBytes': 25980928, 'IDProcess': 660}
{'WorkingSet': 25530368, 'VirtualBytes': 124960768, 'Name': 'nscp', 'PrivateBytes': 9842688, 'IDProcess': 4064}
{'WorkingSet': 13303808, 'VirtualBytes': 89735168, 'Name': 'nxlog', 'PrivateBytes': 9969664, 'IDProcess': 7992}
{'WorkingSet': 5525504, 'VirtualBytes': 32464896, 'Name': 'ocspsvc', 'PrivateBytes': 1355776, 'IDProcess': 1856}
{'WorkingSet': 57335808, 'VirtualBytes': 613888000, 'Name': 'ProviderRegistryService', 'PrivateBytes': 45957120, 'IDProcess': 1552}
{'WorkingSet': 11202560, 'VirtualBytes': 87412736, 'Name': 'rdpclip', 'PrivateBytes': 2011136, 'IDProcess': 4784}
{'WorkingSet': 42491904, 'VirtualBytes': 324980736, 'Name': 'SearchIndexer', 'PrivateBytes': 37740544, 'IDProcess': 4040}
{'WorkingSet': 12328960, 'VirtualBytes': 40304640, 'Name': 'services', 'PrivateBytes': 6955008, 'IDProcess': 524}
{'WorkingSet': 164524032, 'VirtualBytes': 808427520, 'Name': 'SharedServiceHost', 'PrivateBytes': 157544448, 'IDProcess': 2528}
{'WorkingSet': 53559296, 'VirtualBytes': 657874944, 'Name': 'SharedServiceHost#1', 'PrivateBytes': 52256768, 'IDProcess': 2572}
{'WorkingSet': 130535424, 'VirtualBytes': 798470144, 'Name': 'SharedServiceHost#2', 'PrivateBytes': 126357504, 'IDProcess': 2404}
{'WorkingSet': 10633216, 'VirtualBytes': 49348608, 'Name': 'silsvc', 'PrivateBytes': 4243456, 'IDProcess': 1896}
{'WorkingSet': 5496832, 'VirtualBytes': 256921600, 'Name': 'smbhash', 'PrivateBytes': 50868224, 'IDProcess': 2052}
{'WorkingSet': 1155072, 'VirtualBytes': 4337664, 'Name': 'smss', 'PrivateBytes': 286720, 'IDProcess': 264}
{'WorkingSet': 35352576, 'VirtualBytes': 552644608, 'Name': 'SMSvcHost', 'PrivateBytes': 34787328, 'IDProcess': 1592}
{'WorkingSet': 15978496, 'VirtualBytes': 104853504, 'Name': 'spoolsv', 'PrivateBytes': 6426624, 'IDProcess': 1308}
{'WorkingSet': 449966080, 'VirtualBytes': 1100648448, 'Name': 'storageservice', 'PrivateBytes': 490692608, 'IDProcess': 2616}
{'WorkingSet': 14221312, 'VirtualBytes': 691929088, 'Name': 'svchost', 'PrivateBytes': 9924608, 'IDProcess': 3528}
{'WorkingSet': 4640768, 'VirtualBytes': 21512192, 'Name': 'svchost#1', 'PrivateBytes': 1159168, 'IDProcess': 3548}
{'WorkingSet': 20008960, 'VirtualBytes': 1209384960, 'Name': 'svchost#2', 'PrivateBytes': 10637312, 'IDProcess': 1008}
{'WorkingSet': 54988800, 'VirtualBytes': 156901376, 'Name': 'svchost#3', 'PrivateBytes': 52031488, 'IDProcess': 3512}
{'WorkingSet': 14647296, 'VirtualBytes': 90112000, 'Name': 'svchost#4', 'PrivateBytes': 8347648, 'IDProcess': 944}
{'WorkingSet': 8806400, 'VirtualBytes': 39575552, 'Name': 'svchost#5', 'PrivateBytes': 3809280, 'IDProcess': 3344}
{'WorkingSet': 10510336, 'VirtualBytes': 66252800, 'Name': 'svchost#6', 'PrivateBytes': 3862528, 'IDProcess': 1380}
{'WorkingSet': 11091968, 'VirtualBytes': 47849472, 'Name': 'svchost#7', 'PrivateBytes': 6713344, 'IDProcess': 2252}
{'WorkingSet': 11808768, 'VirtualBytes': 51302400, 'Name': 'svchost#8', 'PrivateBytes': 7581696, 'IDProcess': 788}
{'WorkingSet': 177106944, 'VirtualBytes': 420274176, 'Name': 'svchost#9', 'PrivateBytes': 166756352, 'IDProcess': 920}
{'WorkingSet': 47554560, 'VirtualBytes': 180101120, 'Name': 'svchost#10', 'PrivateBytes': 55304192, 'IDProcess': 880}
{'WorkingSet': 9338880, 'VirtualBytes': 38559744, 'Name': 'svchost#11', 'PrivateBytes': 4370432, 'IDProcess': 712}
{'WorkingSet': 8077312, 'VirtualBytes': 441434112, 'Name': 'svchost#12', 'PrivateBytes': 57958400, 'IDProcess': 1900}
{'WorkingSet': 13148160, 'VirtualBytes': 44478464, 'Name': 'svchost#13', 'PrivateBytes': 6041600, 'IDProcess': 668}
{'WorkingSet': 5402624, 'VirtualBytes': 23678976, 'Name': 'System', 'PrivateBytes': 155648, 'IDProcess': 4}
{'WorkingSet': 4694016, 'VirtualBytes': 30183424, 'Name': 'taskeng', 'PrivateBytes': 1196032, 'IDProcess': 28052}
{'WorkingSet': 11231232, 'VirtualBytes': 272408576, 'Name': 'taskhostex', 'PrivateBytes': 3608576, 'IDProcess': 5340}
{'WorkingSet': 8253440, 'VirtualBytes': 46215168, 'Name': 'vds', 'PrivateBytes': 2048000, 'IDProcess': 2116}
{'WorkingSet': 10641408, 'VirtualBytes': 68894720, 'Name': 'VGAuthService', 'PrivateBytes': 4591616, 'IDProcess': 2132}
{'WorkingSet': 4280320, 'VirtualBytes': 46694400, 'Name': 'vmacthlp', 'PrivateBytes': 1130496, 'IDProcess': 824}
{'WorkingSet': 61808640, 'VirtualBytes': 151113728, 'Name': 'vmtoolsd', 'PrivateBytes': 55132160, 'IDProcess': 2232}
{'WorkingSet': 11984896, 'VirtualBytes': 94396416, 'Name': 'vmtoolsd#1', 'PrivateBytes': 3469312, 'IDProcess': 6128}
{'WorkingSet': 11382784, 'VirtualBytes': 56397824, 'Name': 'w3wp', 'PrivateBytes': 5087232, 'IDProcess': 31092}
{'WorkingSet': 10977280, 'VirtualBytes': 56397824, 'Name': 'w3wp#1', 'PrivateBytes': 4562944, 'IDProcess': 33984}
{'WorkingSet': 6508544, 'VirtualBytes': 39034880, 'Name': 'wbengine', 'PrivateBytes': 1605632, 'IDProcess': 1168}
{'WorkingSet': 3837952, 'VirtualBytes': 41644032, 'Name': 'wininit', 'PrivateBytes': 872448, 'IDProcess': 436}
{'WorkingSet': 4767744, 'VirtualBytes': 54595584, 'Name': 'winlogon', 'PrivateBytes': 1028096, 'IDProcess': 4300}
{'WorkingSet': 5734400, 'VirtualBytes': 50896896, 'Name': 'winlogon#1', 'PrivateBytes': 1265664, 'IDProcess': 1424}
{'WorkingSet': 68288512, 'VirtualBytes': 682078208, 'Name': 'WMIExplorer', 'PrivateBytes': 42455040, 'IDProcess': 12760}
{'WorkingSet': 16310272, 'VirtualBytes': 171307008, 'Name': 'WmiPrvSE', 'PrivateBytes': 10522624, 'IDProcess': 8408}
{'WorkingSet': 39821312, 'VirtualBytes': 96186368, 'Name': 'WmiPrvSE#1', 'PrivateBytes': 32481280, 'IDProcess': 20080}
{'WorkingSet': 30744576, 'VirtualBytes': 130625536, 'Name': 'WmiPrvSE#2', 'PrivateBytes': 18210816, 'IDProcess': 3140}
{'WorkingSet': 5636096, 'VirtualBytes': 33796096, 'Name': 'WSSBackup', 'PrivateBytes': 1466368, 'IDProcess': 2284}

Code: Select all

[root@xi-stable ~]# /usr/local/nagios/libexec/check_wmi_plus.pl -H 192.168.67.99 -u admin -p welcome123 -m checkproc -s memoryabove -a % -w WorkingSet=1G -c WorkingSet=1500M -exc WorkingSet=@0:100M
OK - Total Process Count=5 (Process details on next line)|'Process Count'=5; 'PrivateMemory_explorer'=88551424Bytes; 'TotalWorkingSet_explorer'=172789760Bytes;1073741824;1572864000; 'VirtualMemory_explorer'=758968320Bytes; 'PrivateMemory_SharedServiceHost'=157474816Bytes; 'TotalWorkingSet_SharedServiceHost'=164495360Bytes;1073741824;1572864000; 'VirtualMemory_SharedServiceHost'=806313984Bytes; 'PrivateMemory_SharedServiceHost#2'=126320640Bytes; 'TotalWorkingSet_SharedServiceHost#2'=130519040Bytes;1073741824;1572864000; 'VirtualMemory_SharedServiceHost#2'=797413376Bytes; 'PrivateMemory_storageservice'=490561536Bytes; 'TotalWorkingSet_storageservice'=449896448Bytes;1073741824;1572864000; 'VirtualMemory_storageservice'=1099051008Bytes; 'PrivateMemory_svchost#9'=171175936Bytes; 'TotalWorkingSet_svchost#9'=182317056Bytes;1073741824;1572864000; 'VirtualMemory_svchost#9'=420274176Bytes;
OK - explorer: Private Memory=84.449MB, Working Set=164.785MB, Virtual Memory=723.809MB\nOK - SharedServiceHost: Private Memory=150.18MB, Working Set=156.875MB, Virtual Memory=768.961MB\nOK - SharedServiceHost#2: Private Memory=120.469MB, Working Set=124.473MB, Virtual Memory=760.473MB\nOK - storageservice: Private Memory=467.836MB, Working Set=429.055MB, Virtual Memory=1.024GB\nOK - svchost#9: Private Memory=163.246MB, Working Set=173.871MB, Virtual Memory=400.805MB\n
If that doesn't work, can you send over a copy of these guys:

Code: Select all

check_wmi_plus.pl
check_wmi_plus.ini
check_wmi_plus.conf

Re: wmi_Plus Check Memory Above

Posted: Thu Apr 06, 2017 2:06 pm
by kgugerty
That worked! Thanks!
I was close... but wrong syntax!


This issue can be closed from my point of view.