Page 2 of 2
Re: Auth file suddenly stopped working
Posted: Mon Feb 25, 2019 3:12 pm
by N4g10s4dm1n
There are no logon restrictions on the domain account I'm using. As I said in my original post I've had this working for about a year without issue.
At this point I have rolled back both the Nagios server and test Windows servers to a point in time where I know everything was working, and still got the same errors. Beyond that, I rebuilt another Nagios server from scratch, and a set up another test Windows server with a clean install of Server 2016. I am still getting the same errors. I am able to query all of my servers fine using PowerShell, and/or WBEMTEST from Windows servers on the same network. Additionally, I set up a different Windows based service monitoring tool on one of my Windows servers, that also uses WMI, and it is working fine.
I have a few questions for you. From your screenshots of your testing it appears you're using Server 2012, correct? Are you able to set up a DC that's Server 2016, a Windows box that's Server 2016, fully update them with Windows updates (all of my servers are current as of today), and then test Nagios/Check_WMI_Plus?
For now, I'm using the aforementioned Windows based service monitoring tool as a stop-gap until I can get Nagios working again. The one major drawback is that there is no performance, or disk monitoring capabilities with this other service monitoring tool. I really wish I had my Nagios solution back...it was really nice while I had it working!
Re: Auth file suddenly stopped working
Posted: Tue Feb 26, 2019 5:05 pm
by scottwilkerson
Can you re-verify the permissions on the server and user as outlined in this document
https://assets.nagios.com/downloads/nag ... ios-XI.pdf
Re: Auth file suddenly stopped working
Posted: Wed Feb 27, 2019 3:58 pm
by N4g10s4dm1n
Thanks for your reply @scottwilkerson
Yes, all the proper permissions are in place. I'm able to use the same account to query WMI on my Windows servers from other Windows servers using PS and WBEMTEST.
I did manually verify though that all permissions are still in place.
Re: Auth file suddenly stopped working
Posted: Thu Feb 28, 2019 4:49 pm
by scottwilkerson
I know cdienger had you do this before but he had the slashed wrong, can you run this from the command line?
Code: Select all
/usr/local/nagios/libexec/check_wmi_plus.pl -d -z -H myserver.mydomain -m checkcpu -u domainname\/username -p password
replacing the myserver.mydomain, domainname, username and password
Thx
Re: Auth file suddenly stopped working
Posted: Thu Feb 28, 2019 5:08 pm
by N4g10s4dm1n
Same result.
Code: Select all
[root@myserver ~]# /usr/local/nagios/libexec/check_wmi_plus.pl -d -z -H myserver.mydomain -m checkcpu -u mydomain\/useraccount -p password --extrawmicarg "--debuglevel=4"
Command Line (v1.6): /usr/local/nagios/libexec/check_wmi_plus.pl -d -z -H myserver.mydomain -m checkcpu -u mydomain/useraccount -p password --extrawmicarg --debuglevel=4
Base Dir: /usr/local/nagios/libexec
Conf File Dir: /usr/local/nagios/libexec
Loaded Conf File /usr/local/nagios/libexec/check_wmi_plus.conf
Extra Wmic Arguments specified:--debuglevel=4
Starting Keep State Mode
STATE FILE: /tmp/cwpss_checkcpu__10106617___.state
Checking previous data's expiry - Timestamp 1549868800 vs Expiry After 1550686706 (Keep State Expiry setting is 3600sec)
Data has expired - getting data again
Round #1 of 1
QUERY: /usr/bin/wmic '--debuglevel=4' '-U' 'mydomain/useraccount%password' '--namespace' 'root/cimv2' '//myserver.mydomain' 'select PercentProcessorTime,Timestamp_Sys100NS from Win32_PerfRawData_PerfOS_Processor where Name="_Total"'
OUTPUT: [param/loadparm.c:587:init_globals()] Initialising global parameters
[param/loadparm.c:2462:lp_load()] lp_load: refreshing parameters from /dev/null
[param/params.c:556:pm_process()] params.c:pm_process() - Processing configuration file "/dev/null"
[param/loadparm.c:2471:lp_load()] pm_process() returned Yes
[param/loadparm.c:1343:lp_add_hidden()] adding hidden service IPC$
[param/loadparm.c:1343:lp_add_hidden()] adding hidden service ADMIN$
[auth/kerberos/krb5_init_context.c:388:smb_krb5_init_context()] krb5_init_context failed (Invalid argument)
[auth/auth.c:447:auth_register()] AUTH backend 'winbind_samba3' registered
[auth/auth.c:447:auth_register()] AUTH backend 'winbind' registered
[auth/auth.c:447:auth_register()] AUTH backend 'name_to_ntstatus' registered
[auth/auth.c:447:auth_register()] AUTH backend 'fixed_challenge' registered
[auth/auth.c:447:auth_register()] AUTH backend 'unix' registered
[auth/auth.c:447:auth_register()] AUTH backend 'anonymous' registered
[auth/auth.c:447:auth_register()] AUTH backend 'sam' registered
[auth/auth.c:447:auth_register()] AUTH backend 'sam_ignoredomain' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'krb5' registered
[auth/gensec/gensec.c:1205:gensec_register()] gensec subsystem fake_gssapi_krb5 is disabled
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'schannel' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'spnego' registered
[auth/gensec/gensec.c:1205:gensec_register()] gensec subsystem gssapi_spnego is disabled
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'gssapi_krb5' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'gssapi_krb5_sasl' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'ntlmssp' registered
[lib/com/dcom/main.c:528:dcom_determine_rpc_binding()] Using binding ncacn_ip_tcp:myserver.mydomain
[librpc/rpc/dcerpc_connect.c:513:continue_map_binding()] Mapped to DCERPC endpoint 135
[lib/com/dcom/main.c:413:determine_rpc_binding_continue2()] dcerpc_ndr_request_recv returned NT_STATUS_OK
[lib/com/dcom/main.c:417:determine_rpc_binding_continue2()] IObjectExporter::ServerAlive returned NT_STATUS_OK
[auth/gensec/gensec_gssapi.c:304:gensec_gssapi_client_start()] Cannot do GSSAPI to an IP address
[auth/gensec/gensec.c:606:gensec_start_mech()] Failed to start GENSEC client mech gssapi_krb5: NT_STATUS_INVALID_PARAMETER
[auth/ntlmssp/ntlmssp_client.c:128:ntlmssp_client_challenge()] Got challenge flags:
[auth/ntlmssp/ntlmssp.c:72:debug_ntlmssp_flags()] Got NTLMSSP neg_flags=0x62898205
NTLMSSP_NEGOTIATE_UNICODE
NTLMSSP_REQUEST_TARGET
NTLMSSP_NEGOTIATE_NTLM
NTLMSSP_NEGOTIATE_ALWAYS_SIGN
NTLMSSP_NEGOTIATE_NTLM2
NTLMSSP_CHAL_TARGET_INFO
NTLMSSP_NEGOTIATE_128
NTLMSSP_NEGOTIATE_KEY_EXCH
[auth/ntlmssp/ntlmssp_client.c:242:ntlmssp_client_challenge()] NTLMSSP: Set final flags:
[auth/ntlmssp/ntlmssp.c:72:debug_ntlmssp_flags()] Got NTLMSSP neg_flags=0x60088205
NTLMSSP_NEGOTIATE_UNICODE
NTLMSSP_REQUEST_TARGET
NTLMSSP_NEGOTIATE_NTLM
NTLMSSP_NEGOTIATE_ALWAYS_SIGN
NTLMSSP_NEGOTIATE_NTLM2
NTLMSSP_NEGOTIATE_128
NTLMSSP_NEGOTIATE_KEY_EXCH
[librpc/rpc/dcerpc_util.c:1290:dcerpc_pipe_auth_recv()] Failed to bind to uuid REDACTED - NT_STATUS_NET_WRITE_FAULT
[librpc/rpc/dcerpc_connect.c:790:dcerpc_pipe_connect_b_recv()] failed NT status (c0000022) in dcerpc_pipe_connect_b_recv
[wmi/wmic.c:196:main()] ERROR: Login to remote object.
NTSTATUS: NT_STATUS_ACCESS_DENIED - Access denied
Could not find the CLASS: line - an error occurred
WMI DATA:$VAR1 = undef;
UNKNOWN - The WMI query had problems. You might have your username/password wrong or the user's access level is too low. Your Authentication File might be incorrectly formatted or inaccessible. Wmic error text on the next line.
[param/loadparm.c:587:init_globals()] Initialising global parameters
[param/loadparm.c:2462:lp_load()] lp_load: refreshing parameters from /dev/null
[param/params.c:556:pm_process()] params.c:pm_process() - Processing configuration file "/dev/null"
[param/loadparm.c:2471:lp_load()] pm_process() returned Yes
[param/loadparm.c:1343:lp_add_hidden()] adding hidden service IPC$
[param/loadparm.c:1343:lp_add_hidden()] adding hidden service ADMIN$
[auth/kerberos/krb5_init_context.c:388:smb_krb5_init_context()] krb5_init_context failed (Invalid argument)
[auth/auth.c:447:auth_register()] AUTH backend 'winbind_samba3' registered
[auth/auth.c:447:auth_register()] AUTH backend 'winbind' registered
[auth/auth.c:447:auth_register()] AUTH backend 'name_to_ntstatus' registered
[auth/auth.c:447:auth_register()] AUTH backend 'fixed_challenge' registered
[auth/auth.c:447:auth_register()] AUTH backend 'unix' registered
[auth/auth.c:447:auth_register()] AUTH backend 'anonymous' registered
[auth/auth.c:447:auth_register()] AUTH backend 'sam' registered
[auth/auth.c:447:auth_register()] AUTH backend 'sam_ignoredomain' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'krb5' registered
[auth/gensec/gensec.c:1205:gensec_register()] gensec subsystem fake_gssapi_krb5 is disabled
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'schannel' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'spnego' registered
[auth/gensec/gensec.c:1205:gensec_register()] gensec subsystem gssapi_spnego is disabled
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'gssapi_krb5' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'gssapi_krb5_sasl' registered
[auth/gensec/gensec.c:1229:gensec_register()] GENSEC backend 'ntlmssp' registered
[lib/com/dcom/main.c:528:dcom_determine_rpc_binding()] Using binding ncacn_ip_tcp:myserver.mydomain
[librpc/rpc/dcerpc_connect.c:513:continue_map_binding()] Mapped to DCERPC endpoint 135
[lib/com/dcom/main.c:413:determine_rpc_binding_continue2()] dcerpc_ndr_request_recv returned NT_STATUS_OK
[lib/com/dcom/main.c:417:determine_rpc_binding_continue2()] IObjectExporter::ServerAlive returned NT_STATUS_OK
[auth/gensec/gensec_gssapi.c:304:gensec_gssapi_client_start()] Cannot do GSSAPI to an IP address
[auth/gensec/gensec.c:606:gensec_start_mech()] Failed to start GENSEC client mech gssapi_krb5: NT_STATUS_INVALID_PARAMETER
[auth/ntlmssp/ntlmssp_client.c:128:ntlmssp_client_challenge()] Got challenge flags:
[auth/ntlmssp/ntlmssp.c:72:debug_ntlmssp_flags()] Got NTLMSSP neg_flags=0x62898205
NTLMSSP_NEGOTIATE_UNICODE
NTLMSSP_REQUEST_TARGET
NTLMSSP_NEGOTIATE_NTLM
NTLMSSP_NEGOTIATE_ALWAYS_SIGN
NTLMSSP_NEGOTIATE_NTLM2
NTLMSSP_CHAL_TARGET_INFO
NTLMSSP_NEGOTIATE_128
NTLMSSP_NEGOTIATE_KEY_EXCH
[auth/ntlmssp/ntlmssp_client.c:242:ntlmssp_client_challenge()] NTLMSSP: Set final flags:
[auth/ntlmssp/ntlmssp.c:72:debug_ntlmssp_flags()] Got NTLMSSP neg_flags=0x60088205
NTLMSSP_NEGOTIATE_UNICODE
NTLMSSP_REQUEST_TARGET
NTLMSSP_NEGOTIATE_NTLM
NTLMSSP_NEGOTIATE_ALWAYS_SIGN
NTLMSSP_NEGOTIATE_NTLM2
NTLMSSP_NEGOTIATE_128
NTLMSSP_NEGOTIATE_KEY_EXCH
[librpc/rpc/dcerpc_util.c:1290:dcerpc_pipe_auth_recv()] Failed to bind to uuid REDACTED - NT_STATUS_NET_WRITE_FAULT
[librpc/rpc/dcerpc_connect.c:790:dcerpc_pipe_connect_b_recv()] failed NT status (c0000022) in dcerpc_pipe_connect_b_recv
[wmi/wmic.c:196:main()] ERROR: Login to remote object.
NTSTATUS: NT_STATUS_ACCESS_DENIED - Access denied
Re: Auth file suddenly stopped working
Posted: Thu Feb 28, 2019 5:33 pm
by scottwilkerson
I just verified and tested this against a Windows 2016 server with an domain admin account and got the expected results.
Additionally I tested it with an auth file in the same format you showed you were using earlier. Again, worked as expected.
Does your username, password or domain have any spaces or special chars that might need to be escaped?
Re: Auth file suddenly stopped working
Posted: Fri Mar 01, 2019 2:49 pm
by N4g10s4dm1n
There's nothing fancy about the credentials. I tested multiple different passwords looking for some character that suddenly wasn't being passed correctly, and found nothing of the sort. Even using a minimally complex password returns the same errors.
Is there anything in the query output or wmic error text that sticks out to you?
Also, was the 2016 server you tested against fully updated, and what OS was the DC running and was it fully updated? I'm wondering if recent Windows updates might have caused this.
Re: Auth file suddenly stopped working
Posted: Fri Mar 01, 2019 4:31 pm
by scottwilkerson
I know the server and DC I tested with isn't fully up to date, so this could be a possibility. I can't perform that action at present to test if it is related to a Windows update unfortunately.
Re: Auth file suddenly stopped working
Posted: Mon Mar 04, 2019 4:55 pm
by scottwilkerson
Had another user on our customer forum come across the same issue and another tech found this to be helpful
KB4487026
Addresses an issue that fails to set the LmCompatibilityLevel value correctly. LmCompatibilityLevel specifies the authentication mode and session security.
What do the NTLM options in highlighted in
https://www.rootusers.com/implement-ntl ... rver-2016/ and the "LAN Manager authentication level" option look like now? Try setting the LAN Manager setting to the different options and testing them with the check_wmi_plus plugin.
The end user said setting check_wmi_plus to use NTLMv2 seems to clear up the issue