Page 1 of 1

check_wmi_plus / wmic NT code 0xc002001b

Posted: Wed Aug 02, 2017 4:30 am
by mdahms
Hello,

I've got problems with check_wmi_plus / wmic when checking Server 2008 R2 machines with Nagios Core 4.

I get following error when using check_wmi_plus. Ok i have seen this one already

Code: Select all

./check_wmi_plus.pl -d 4 -t 30 -H xxx.xx.xxx.xx -u DOMAIN/USER -p PASSWORD -m checkservice -w 0 -c 1 -a auto
Command Line (v1.62): ./check_wmi_plus.pl -d 4 -t 30 -H xxx.xx.xxx.xx -u USER -p PASS -m checkservice -w 0 -c 1 -a auto
Base Dir: /opt/openitc/nagios/libexec
Conf File Dir: /etc/check_wmi_plus
Loaded Conf File /etc/check_wmi_plus/check_wmi_plus.conf
WMI Ini Dir: /etc/check_wmi_plus/check_wmi_plus.d
Round #1 of 1
QUERY: /usr/bin/wmic '-U' 'USER%PASS' '--namespace' 'root/cimv2' '//XXX.XX.XXX.XX' 'select name, displayname, Started, StartMode, State, Status FROM Win32_Service where StartMode="auto"'
UNKNOWN - Plugin Timed out (30 sec). There are multiple possible reasons for this, some of them include - The host XXX.XX.XXX.XX might just be really busy, it might not even be running Windows.
But here things are getting strange.
When i use wmic on the host it shows services but stucks after RemoteRegistry

Code: Select all

wmic -d 7 '-U' 'DOMAIN/USER%PASSWORD' '--namespace' 'root/cimv2' '//XXX.XX.XXX.XX' 'select name, displayname, Started, StartMode, State, Status FROM Win32_Service where StartMode="auto"'
[param/loadparm.c:587:init_globals()] Initialising global parameters
[param/loadparm.c:2462:lp_load()] lp_load: refreshing parameters from /dev/null
[param/params.c:556:pm_process()] params.c:pm_process() - Processing configuration file "/dev/null"
[param/loadparm.c:2471:lp_load()] pm_process() returned Yes
[param/loadparm.c:1343:lp_add_hidden()] adding hidden service IPC$
[param/loadparm.c:1343:lp_add_hidden()] adding hidden service ADMIN$
[auth/credentials/credentials_krb5.c:171:cli_credentials_set_ccache()] failed to get principal from default ccache: No such file or directory: open(/tmp/krb5cc_0): No such file or directory
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'schannel' registered
[auth/gensec/gensec.c:1205:gensec_register()] gensec subsystem gssapi_spnego is disabled
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'gssapi_krb5' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'gssapi_krb5_sasl' registered
[auth/auth.c:447:auth_register()] AUTH backend 'sam' registered
[auth/auth.c:447:auth_register()] AUTH backend 'sam_ignoredomain' registered
[auth/auth.c:447:auth_register()] AUTH backend 'anonymous' registered
[auth/auth.c:447:auth_register()] AUTH backend 'unix' registered
[auth/auth.c:447:auth_register()] AUTH backend 'winbind_samba3' registered
[auth/auth.c:447:auth_register()] AUTH backend 'winbind' registered
[auth/auth.c:447:auth_register()] AUTH backend 'name_to_ntstatus' registered
[auth/auth.c:447:auth_register()] AUTH backend 'fixed_challenge' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'krb5' registered
[auth/gensec/gensec.c:1205:gensec_register()] gensec subsystem fake_gssapi_krb5 is disabled
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'ntlmssp' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'spnego' registered
[lib/com/dcom/main.c:528:dcom_determine_rpc_binding()] Using binding ncacn_ip_tcp:XXX.XX.XXX.XX
[librpc/rpc/dcerpc_connect.c:513:continue_map_binding()] Mapped to DCERPC endpoint 135
[lib/com/dcom/main.c:413:determine_rpc_binding_continue2()] dcerpc_ndr_request_recv returned NT_STATUS_OK
[lib/com/dcom/main.c:417:determine_rpc_binding_continue2()] IObjectExporter::ServerAlive returned NT_STATUS_OK
[auth/gensec/gensec.c:599:gensec_start_mech()] Starting GENSEC mechanism spnego
[auth/gensec/gensec.c:599:gensec_start_mech()] Starting GENSEC submechanism gssapi_krb5
[auth/gensec/gensec_gssapi.c:304:gensec_gssapi_client_start()] Cannot do GSSAPI to an IP address
[auth/gensec/gensec.c:606:gensec_start_mech()] Failed to start GENSEC client mech gssapi_krb5: NT_STATUS_INVALID_PARAMETER
[auth/gensec/gensec.c:599:gensec_start_mech()] Starting GENSEC submechanism ntlmssp
[auth/ntlmssp/ntlmssp_client.c:128:ntlmssp_client_challenge()] Got challenge flags:
[auth/ntlmssp/ntlmssp.c:72:debug_ntlmssp_flags()] Got NTLMSSP neg_flags=0x62898205
  NTLMSSP_NEGOTIATE_UNICODE
  NTLMSSP_REQUEST_TARGET
  NTLMSSP_NEGOTIATE_NTLM
  NTLMSSP_NEGOTIATE_ALWAYS_SIGN
  NTLMSSP_NEGOTIATE_NTLM2
  NTLMSSP_CHAL_TARGET_INFO
  NTLMSSP_NEGOTIATE_128
  NTLMSSP_NEGOTIATE_KEY_EXCH
[auth/credentials/credentials_ntlm.c:130:cli_credentials_get_ntlm_response()] NTLMSSP challenge set by NTLM2
[auth/credentials/credentials_ntlm.c:131:cli_credentials_get_ntlm_response()] challenge is:
[000] A1 14 37 78 C7 17 34 49                           ..7x..4I
[auth/ntlmssp/ntlmssp_client.c:242:ntlmssp_client_challenge()] NTLMSSP: Set final flags:
[auth/ntlmssp/ntlmssp.c:72:debug_ntlmssp_flags()] Got NTLMSSP neg_flags=0x60088205
  NTLMSSP_NEGOTIATE_UNICODE
  NTLMSSP_REQUEST_TARGET
  NTLMSSP_NEGOTIATE_NTLM
  NTLMSSP_NEGOTIATE_ALWAYS_SIGN
  NTLMSSP_NEGOTIATE_NTLM2
  NTLMSSP_NEGOTIATE_128
  NTLMSSP_NEGOTIATE_KEY_EXCH
[librpc/ndr/ndr_string.c:214:ndr_pull_string()] long string ''
[lib/com/dcom/main.c:570:complete_activation()] Negotiated COM version: 5.1 using binding ncacn_ip_tcp:XXX.XX.XXX.XX[135]
[lib/com/dcom/main.c:1172:bind_new_pipe()] lib/com/dcom/main.c:1172: dcom_get_pipe: host=XXX.XX.XXX.XX, similar=XXX.XX.XXX.XX[49155]
[auth/gensec/gensec.c:599:gensec_start_mech()] Starting GENSEC mechanism ntlmssp
[auth/ntlmssp/ntlmssp_client.c:128:ntlmssp_client_challenge()] Got challenge flags:
[auth/ntlmssp/ntlmssp.c:72:debug_ntlmssp_flags()] Got NTLMSSP neg_flags=0x62898215
  NTLMSSP_NEGOTIATE_UNICODE
  NTLMSSP_REQUEST_TARGET
  NTLMSSP_NEGOTIATE_SIGN
  NTLMSSP_NEGOTIATE_NTLM
  NTLMSSP_NEGOTIATE_ALWAYS_SIGN
  NTLMSSP_NEGOTIATE_NTLM2
  NTLMSSP_CHAL_TARGET_INFO
  NTLMSSP_NEGOTIATE_128
  NTLMSSP_NEGOTIATE_KEY_EXCH
[auth/credentials/credentials_ntlm.c:130:cli_credentials_get_ntlm_response()] NTLMSSP challenge set by NTLM2
[auth/credentials/credentials_ntlm.c:131:cli_credentials_get_ntlm_response()] challenge is:
[000] 41 38 DE 18 35 96 33 7C                           A8..5.3|
[auth/ntlmssp/ntlmssp_client.c:242:ntlmssp_client_challenge()] NTLMSSP: Set final flags:
[auth/ntlmssp/ntlmssp.c:72:debug_ntlmssp_flags()] Got NTLMSSP neg_flags=0x60088215
  NTLMSSP_NEGOTIATE_UNICODE
  NTLMSSP_REQUEST_TARGET
  NTLMSSP_NEGOTIATE_SIGN
  NTLMSSP_NEGOTIATE_NTLM
  NTLMSSP_NEGOTIATE_ALWAYS_SIGN
  NTLMSSP_NEGOTIATE_NTLM2
  NTLMSSP_NEGOTIATE_128
  NTLMSSP_NEGOTIATE_KEY_EXCH
[auth/ntlmssp/ntlmssp_sign.c:318:ntlmssp_sign_init()] NTLMSSP Sign/Seal - Initialising with flags:
[auth/ntlmssp/ntlmssp.c:72:debug_ntlmssp_flags()] Got NTLMSSP neg_flags=0x60088215
  NTLMSSP_NEGOTIATE_UNICODE
  NTLMSSP_REQUEST_TARGET
  NTLMSSP_NEGOTIATE_SIGN
  NTLMSSP_NEGOTIATE_NTLM
  NTLMSSP_NEGOTIATE_ALWAYS_SIGN
  NTLMSSP_NEGOTIATE_NTLM2
  NTLMSSP_NEGOTIATE_128
  NTLMSSP_NEGOTIATE_KEY_EXCH
[librpc/ndr/ndr_string.c:214:ndr_pull_string()] long string ''
[wmi/wmic.c:196:main()] OK   : Login to remote object.
[librpc/ndr/ndr_string.c:214:ndr_pull_string()] long string ''
[wmi/wmic.c:200:main()] OK   : WMI query execute.
[librpc/ndr/ndr_string.c:214:ndr_pull_string()] long string ''
[wmi/wmic.c:203:main()] OK   : Reset result of WMI query.
[librpc/ndr/ndr_string.c:214:ndr_pull_string()] long string ''
[librpc/ndr/ndr_string.c:214:ndr_pull_string()] long string ''
[librpc/ndr/ndr_string.c:214:ndr_pull_string()] long string ''
[wmi/wmic.c:212:main()] OK   : Retrieve result data.
CLASS: Win32_Service
DisplayName|Name|Started|StartMode|State|Status
ACMP Client|ACMPClient|True|Auto|Running|OK
Autodesk Lizenz Server|Autodesk Lizenz Server|True|Auto|Running|OK
Basisfiltermodul|BFE|True|Auto|Running|OK
Computerbrowser|Browser|True|Auto|Running|OK
CBIOS Server|CBIOSServer|True|Auto|Running|OK
[wmi/wmic.c:212:main()] OK   : Retrieve result data.
Microsoft .NET Framework NGEN v4.0.30319_X86|clr_optimization_v4.0.30319_32|False|Auto|Stopped|OK
Microsoft .NET Framework NGEN v4.0.30319_X64|clr_optimization_v4.0.30319_64|False|Auto|Stopped|OK
Kryptografiedienste|CryptSvc|True|Auto|Running|OK
DCOM-Server-Prozessstart|DcomLaunch|True|Auto|Running|OK
DHCP-Client|Dhcp|True|Auto|Running|OK
[wmi/wmic.c:212:main()] OK   : Retrieve result data.
Diagnostics Tracking Service|DiagTrack|True|Auto|Running|OK
DNS-Client|Dnscache|True|Auto|Running|OK
Diagnoserichtliniendienst|DPS|True|Auto|Running|OK
Verschl³sselndes Dateisystem (EFS)|EFS|True|Auto|Running|OK
Windows-Ereignisprotokoll|eventlog|True|Auto|Running|OK
[wmi/wmic.c:212:main()] OK   : Retrieve result data.
COM+-Ereignissystem|EventSystem|True|Auto|Running|OK
Windows-Dienst f³r Schriftartencache|FontCache|True|Auto|Running|OK
Gruppenrichtlinienclient|gpsvc|True|Auto|Running|OK
IKE- und AuthIP IPsec-Schl³sselerstellungsmodule|IKEEXT|True|Auto|Running|OK
IP-Hilfsdienst|iphlpsvc|True|Auto|Running|OK
[wmi/wmic.c:212:main()] OK   : Retrieve result data.
Server|LanmanServer|True|Auto|Running|OK
Arbeitsstationsdienst|LanmanWorkstation|True|Auto|Running|OK
lmadmin_ptc|lmadmin_ptc|True|Auto|Running|OK
TCP/IP-NetBIOS-Hilfsdienst|lmhosts|True|Auto|Running|OK
Windows-Firewall|MpsSvc|True|Auto|Running|OK
[wmi/wmic.c:212:main()] OK   : Retrieve result data.
Distributed Transaction Coordinator|MSDTC|True|Auto|Running|OK
Windows Installer|msiserver|False|Auto|Stopped|OK
NLA (Network Location Awareness)|NlaSvc|True|Auto|Running|OK
Netzwerkspeicher-Schnittstellendienst|nsi|True|Auto|Running|OK
[wmi/wmic.c:212:main()] OK   : Retrieve result data.
Password Depot Server 6|PD_Service_6|True|Auto|Running|OK
Plug & Play|PlugPlay|True|Auto|Running|OK
Stromversorgung|Power|True|Auto|Running|OK
Benutzerprofildienst|ProfSvc|True|Auto|Running|OK
Remoteregistrierung|RemoteRegistry|True|Auto|Running|OK
[wmi/wmic.c:212:main()] ERROR: Retrieve result data.
NTSTATUS: NT code 0xc002001b - NT code 0xc002001b
Any chance to see at which service the check is stuck / why it is stuck?
Other modes like checkcpu work btw.

Regards

Re: check_wmi_plus / wmic NT code 0xc002001b

Posted: Wed Aug 02, 2017 4:49 pm
by tgriep
You can run these commands on the Windows system to show service's that are on the system and display the status.

Code: Select all

sc query state= all
sc query type= service state= all
Maybe you can find the troublesome service.

Re: check_wmi_plus / wmic NT code 0xc002001b

Posted: Thu Aug 03, 2017 1:36 am
by mdahms
I tried other StarModes like manual and it always stops after the 35th service

Re: check_wmi_plus / wmic NT code 0xc002001b

Posted: Thu Aug 03, 2017 12:46 pm
by tgriep
You may need to reboot the Windows server to free up the stuck service.

Re: check_wmi_plus / wmic NT code 0xc002001b

Posted: Thu Aug 03, 2017 12:56 pm
by mcapra
You might also try altering your check_wmi_plus command to leverage the --forcetruncateoutput argument. By default, this plugin will truncate the data received at 8192 bytes. You might bump it up to something like 65536 instead. I don't have a lab setup readily available, but it might look something like this:

Code: Select all

./check_wmi_plus.pl -d 4 -t 30 -H xxx.xx.xxx.xx -u DOMAIN/USER -p PASSWORD -m checkservice -w 0 -c 1 -a auto --forcetruncateoutput 65536
From this article:
https://support.nagios.com/kb/article.php?id=579

Given that the plugin is timing out, I think the stuck service is most likely the issue. You might make sure your system is not impacted by known issues with WMI in Server 2008 as well:
https://support.microsoft.com/en-us/hel ... entation-s

Re: check_wmi_plus / wmic NT code 0xc002001b

Posted: Thu Aug 03, 2017 4:20 pm
by tgriep
Thanks @mcapra for the help.

Re: check_wmi_plus / wmic NT code 0xc002001b

Posted: Fri Aug 04, 2017 12:13 am
by mdahms
Thank you tgriep and mcapra for your help. I don't know why but suddenly it is working after several winmgmt restarts. I will keep your hints in mind.