Page 1 of 1

Check_nt failing for only one server

Posted: Thu May 09, 2013 1:28 pm
by linuxnewbie
Hello everyone!

I was wondering if anyone could help me troubleshoot why only one of the servers I setup is giving me a socket time out error.

Code: Select all

CRITICAL - Socket timeout after 10 seconds
I've been using the same code for all of my servers, just changing the needed info for each.

Code: Select all

define service{
	use			generic-service
	host_name		Chan-server
	service_description	Drive Space - C
	check_command		check_nt!USEDDISKSPACE!-l c -w 95 -c 97
	}
To test my connection via the command line I tried this: (with the correct ip of course)

Code: Select all

$ ./check_nt -H ***.***.***.*** -p 12489 -v UPTIME
CRITICAL - Socket timeout after 10 seconds
I also tried changing the port:

Code: Select all

$ ./check_nt -H ***.***.***.*** -p 5666 -v UPTIME
No data was received from host!
On the windows server I have stopped and started the NSclient service, I've checked the ports, and made sure the firewall has exceptions... What have I not tried, what am I over looking?

**edit**
I should also mention I have 'check event log' services checking that server as well and they are reporting back normally.

Re: Check_nt failing for only one server

Posted: Thu May 09, 2013 1:51 pm
by lmiltchev
Can you telnet from the nagios server into port 12489 on the windows box?

Code: Select all

telnet <client_ip> 12489
Is the nagios server's ip added to the "allowed_hosts=" line in the nsclinet++ configuration file (NSC.ini or nsclient.ini)? Did you restart the NSClient++ service, after you made changes in the *.ini file? It may help if you posted the NSC.ini file (hide sensitive data).

BTW, you don't run check_nt on port 5666. Also, you didn't use "-s" flag in your check, so I assume you are not using a password with NSClient++ (or you deleted it for privacy). If you ran the check without "-s <password>", you would get:

Code: Select all

NSClient - ERROR: Invalid password.

Re: Check_nt failing for only one server

Posted: Thu May 09, 2013 4:12 pm
by linuxnewbie
lmiltchev-

First off, thanks for the help!
Second, I'm still trying to teach my self nagios and linux so if I'm making a simple mistake thats prob why.
Currently we are using a password for Rsync backups but not for nagios, now that I know about the password I will go about setting that up as well.

Thanks for confirming that I cannot run checks on port 5666, I just wanted to make sure I was being as through as possible.

Telnet test using port 12489 (below under the edit you will see the result for a telnet test via port 5666)

Code: Select all

$ telnet ***.***.***.*** 12489
Trying ***.***.***.***...
telnet: Unable to connect to remote host: Connection timed out
As requested this is the NSC.ini that i am using, i have commented out my server's IP.
Before I made the edit the ini file had the 'allowed_hosts=' was set for my nagios server correctly. We use this same ini file for 10 other servers and none of them are having issues like this server is.

Code: Select all

[modules]
;# NSCLIENT++ MODULES
;# A list with DLLs to load at startup.
;  You will need to enable some of these for NSClient++ to work.
; ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !
; *                                                               *
; * N O T I C E ! ! ! - Y O U   H A V E   T O   E D I T   T H I S *
; *                                                               *
; ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !
FileLogger.dll
CheckSystem.dll
CheckDisk.dll
NSClientListener.dll
NRPEListener.dll
;SysTray.dll
CheckEventLog.dll
CheckHelpers.dll
;CheckWMI.dll
;
; Script to check external scripts and/or internal aliases.
CheckExternalScripts.dll
;
; NSCA Agent if you enable this NSClient++ will talk to NSCA hosts repeatedly (so dont enable unless you want to use NSCA)
;NSCAAgent.dll
;
; LUA script module used to write your own "check deamon".
;LUAScript.dll
;
; RemoteConfiguration IS AN EXTREM EARLY IDEA SO DONT USE FOR PRODUCTION ENVIROMNEMTS!
;RemoteConfiguration.dll
; Check other hosts through NRPE extreme beta and probably a bit dangerous! :)
;NRPEClient.dll
; Extreamly early beta of a task-schedule checker
;CheckTaskSched.dll



[Settings]
;# OBFUSCATED PASSWORD
;  This is the same as the password option but here you can store the password in an obfuscated manner.
;  *NOTICE* obfuscation is *NOT* the same as encryption, someone with access to this file can still figure out the 
;  password. Its just a bit harder to do it at first glance.
;obfuscated_password=Jw0KAUUdXlAAUwASDAAB
;
;# PASSWORD
;  This is the password (-s) that is required to access NSClient remotely. If you leave this blank everyone will be able to access the daemon remotly.
;password=secret-password
;
;# ALLOWED HOST ADDRESSES
;  This is a comma-delimited list of IP address of hosts that are allowed to talk to the all daemons.
;  If leave this blank anyone can access the deamon remotly (NSClient still requires a valid password).
;  The syntax is host or ip/mask so 192.168.0.0/24 will allow anyone on that subnet access
allowed_hosts=***.***.***.***
;
;# USE THIS FILE
;  Use the INI file as opposed to the registry if this is 0 and the use_reg in the registry is set to 1 
;  the registry will be used instead.
use_file=1
;
; # USE SHARED MEMORY CHANNELS
;  This is the "new" way for using the system tray based on an IPC framework on top shared memmory channels and events.
;  It is brand new and (probably has bugs) so dont enable this unless for testing!
;  If set to 1 shared channels will be created and system tray icons created and such and such...
;shared_session=0


[log]
;# LOG DEBUG
;  Set to 1 if you want debug message printed in the log file (debug messages are always printed to stdout when run with -test)
;debug=1
;
;# LOG FILE
;  The file to print log statements to
;file=nsclient.log
;
;# LOG DATE MASK
;  The format to for the date/time part of the log entry written to file.
;date_mask=%Y-%m-%d %H:%M:%S
;
;# LOG ROOT FOLDER
;  The root folder to use for logging.
;  exe = the folder where the executable is located
;  local-app-data = local application data (probably a better choice then the old default)
;root_folder=exe


[NSClient]
;# ALLOWED HOST ADDRESSES
;  This is a comma-delimited list of IP address of hosts that are allowed to talk to NSClient deamon.
;  If you leave this blank the global version will be used instead.
;allowed_hosts=
;
;# NSCLIENT PORT NUMBER
;  This is the port the NSClientListener.dll will listen to.
port=12489
;
;# BIND TO ADDRESS
;  Allows you to bind server to a specific local address. This has to be a dotted ip adress not a hostname.
;  Leaving this blank will bind to all avalible IP adresses.
;bind_to_address=
;
;# SOCKET TIMEOUT
;  Timeout when reading packets on incoming sockets. If the data has not arrived withint this time we will bail out.
;socket_timeout=30

[NRPE]
;# NRPE PORT NUMBER
;  This is the port the NRPEListener.dll will listen to.
port=5666
;
;# COMMAND TIMEOUT
;  This specifies the maximum number of seconds that the NRPE daemon will allow plug-ins to finish executing before killing them off.
;command_timeout=60
;
;# COMMAND ARGUMENT PROCESSING
;  This option determines whether or not the NRPE daemon will allow clients to specify arguments to commands that are executed.
allow_arguments=1
;
;# COMMAND ALLOW NASTY META CHARS
;  This option determines whether or not the NRPE daemon will allow clients to specify nasty (as in |`&><'"\[]{}) characters in arguments.
allow_nasty_meta_chars=1
;
;# USE SSL SOCKET
;  This option controls if SSL should be used on the socket.
;use_ssl=1
;
;# BIND TO ADDRESS
;  Allows you to bind server to a specific local address. This has to be a dotted ip adress not a hostname.
;  Leaving this blank will bind to all avalible IP adresses.
; bind_to_address=
;
;# ALLOWED HOST ADDRESSES
;  This is a comma-delimited list of IP address of hosts that are allowed to talk to NRPE deamon.
;  If you leave this blank the global version will be used instead.
;allowed_hosts=
;
;# SCRIPT DIRECTORY
;  All files in this directory will become check commands.
;  *WARNING* This is undoubtedly dangerous so use with care!
;script_dir=scripts\
;
;# SOCKET TIMEOUT
;  Timeout when reading packets on incoming sockets. If the data has not arrived withint this time we will bail out.
;socket_timeout=30

[EventLog]
buffer_size=250000

[Check System]
;# CPU BUFFER SIZE
;  Can be anything ranging from 1s (for 1 second) to 10w for 10 weeks. Notice that a larger buffer will waste memory 
;  so don't use a larger buffer then you need (ie. the longest check you do +1).
;CPUBufferSize=1h
;
;# CHECK RESOLUTION
;  The resolution to check values (currently only CPU).
;  The value is entered in 1/10:th of a second and the default is 10 (which means ones every second)
;CheckResolution=10
;
;# CHECK ALL SERVICES
;  Configure how to check services when a CheckAll is performed.
;  ...=started means services in that class *has* to be running.
;  ...=stopped means services in that class has to be stopped.
;  ...=ignored means services in this class will be ignored.
;check_all_services[SERVICE_BOOT_START]=ignored
;check_all_services[SERVICE_SYSTEM_START]=ignored
;check_all_services[SERVICE_AUTO_START]=started
;check_all_services[SERVICE_DEMAND_START]=ignored
;check_all_services[SERVICE_DISABLED]=stopped

[External Script]
;# COMMAND TIMEOUT
;  This specifies the maximum number of seconds that the NRPE daemon will allow plug-ins to finish executing before killing them off.
;command_timeout=60
;
;# COMMAND ARGUMENT PROCESSING
;  This option determines whether or not the NRPE daemon will allow clients to specify arguments to commands that are executed.
allow_arguments=1
;
;# COMMAND ALLOW NASTY META CHARS
;  This option determines whether or not the NRPE daemon will allow clients to specify nasty (as in |`&><'"\[]{}) characters in arguments.
allow_nasty_meta_chars=1
;
;# COMMAND ALLOW NASTY META CHARS
;  This option determines whether or not the NRPE daemon will allow clients to specify nasty (as in |`&><'"\[]{}) characters in arguments.
;script_dir=c:\my\script\dir

[External Scripts]
;check_es_long=scripts\long.bat
;check_es_ok=scripts\ok.bat
;check_es_nok=scripts\nok.bat
;check_vbs_sample=cscript.exe //T:30 //NoLogo scripts\check_vb.vbs
;check_powershell_warn=cmd /c echo scripts\powershell.ps1 | powershell.exe -command -

[External Alias]
alias_cpu=checkCPU warn=80 crit=90 time=5m time=1m time=30s
alias_cpu_ex=checkCPU warn=$ARG1$ crit=$ARG2$ time=5m time=1m time=30s
alias_disk=CheckDriveSize MinWarn=10% MinCrit=5% CheckAll FilterType=FIXED
alias_service=checkServiceState CheckAll
alias_process=checkProcState $ARG1$=started
alias_mem=checkMem MaxWarn=80% MaxCrit=90% ShowAll type=physical
alias_up=checkUpTime MinWarn=1d MinWarn=1h
alias_file_age=checkFile2 filter=out "file=$ARG1$" filter-written=>1d MaxWarn=1 MaxCrit=1 "syntax=%filename% %write%"
alias_file_size=checkFile2 filter=out "file=$ARG1$" filter-size=>$ARG2$ MaxWarn=1 MaxCrit=1 "syntax=%filename% %size%"
alias_file_size_in_dir=checkFile2 filter=out pattern=*.txt "file=$ARG1$" filter-size=>$ARG2$ MaxWarn=1 MaxCrit=1 "syntax=%filename% %size%"
alias_event_log=CheckEventLog file=application file=system filter=new filter=out MaxWarn=1 MaxCrit=1 filter-generated=>2d filter-severity==success filter-severity==informational truncate=1023 unique descriptions  "syntax=%severity%: %source%: %message% (%count%)"
alias_event_log2=CheckEventLog file=application file=system filter=new filter=in MaxWarn=1 MaxCrit=1 filter+generated=<2d "filter+eventSource==Service Control Manager" filter+severity==error truncate=1023 unique descriptions  "syntax=%severity%: %source%: %message% (%count%)"


; [includes]
;# The order when used is "reversed" thus the last included file will be "first"
;# Included files can include other files (be carefull only do basic recursive checking)
;
; myotherfile.ini
; real.ini


[NSCA Agent]
;# CHECK INTERVALL (in seconds)
;   How often we should run the checks and submit the results.
;interval=5
;
;# ENCRYPTION METHOD
;   This option determines the method by which the send_nsca client will encrypt the packets it sends 
;   to the nsca daemon. The encryption method you choose will be a balance between security and 
;   performance, as strong encryption methods consume more processor resources.
;   You should evaluate your security needs when choosing an encryption method.
;
; Note: The encryption method you specify here must match the decryption method the nsca daemon uses 
;       (as specified in the nsca.cfg file)!!
; Values:
;	0 = None	(Do NOT use this option)
;	1 = Simple XOR  (No security, just obfuscation, but very fast)
;   2 = DES
;   3 = 3DES (Triple DES)
;	4 = CAST-128
;	6 = xTEA
;	8 = BLOWFISH
;	9 = TWOFISH
;	11 = RC2
;	14 = RIJNDAEL-128 (AES)
;	20 = SERPENT
;encryption_method=14
;
;# ENCRYPTION PASSWORD
;  This is the password/passphrase that should be used to encrypt the sent packets. 
;password=
;
;# BIND TO ADDRESS
;  Allows you to bind server to a specific local address. This has to be a dotted ip adress not a hostname.
;  Leaving this blank will bind to "one" local interface.
; -- not supported as of now -- 
;bind_to_address=
;
;# LOCAL HOST NAME
;  The name of this host (if empty "computername" will be used.
;hostname=
;
;# NAGIOS SERVER ADDRESS
;  The address to the nagios server to submit results to.
;nsca_host=192.168.0.1
;
;# NAGIOS SERVER PORT
;  The port to the nagios server to submit results to.
;nsca_port=5667
;

;# CHECK COMMAND LIST
;  The checks to run everytime we submit results back to nagios
;  Any command(alias/key) starting with a host_ is sent as HOST_COMMAND others are sent as SERVICE_COMMANDS
;  where the alias/key is used as service name.
;
[NSCA Commands]
;my_cpu_check=checkCPU warn=80 crit=90 time=20m time=10s time=4
;my_mem_check=checkMem MaxWarn=80% MaxCrit=90% ShowAll type=page
;my_svc_check=checkServiceState CheckAll exclude=wampmysqld exclude=MpfService
;host_check=check_ok

[NRPE Handlers]
;# COMMAND DEFINITIONS
;# Command definitions that this daemon will run.
;# Can be either NRPE syntax:
;command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
;# Or simplified syntax:
;test=c:\test.bat foo $ARG1$ bar
;check_disk1=/usr/local/nagios/libexec/check_disk -w 5 -c 10
;# Or even loopback (inject) syntax (to run internal commands)
;# This is a way to run "NSClient" commands and other internal module commands such as check eventlog etc.
;check_cpu=inject checkCPU warn=80 crit=90 5 10 15
;check_eventlog=inject CheckEventLog Application warn.require.eventType=error warn.require.eventType=warning critical.require.eventType=error critical.exclude.eventType=info truncate=1024 descriptions
;check_disk_c=inject CheckFileSize ShowAll MaxWarn=1024M MaxCrit=4096M File:WIN=c:\ATI\*.*
;# But be careful:
; dont_check=inject dont_check This will "loop forever" so be careful with the inject command...
;# Check some escapings...
; check_escape=inject CheckFileSize ShowAll MaxWarn=1024M MaxCrit=4096M "File: foo \" WIN=c:\\WINDOWS\\*.*"
;# Some real world samples
;nrpe_cpu=inject checkCPU warn=80 crit=90 5 10 15
;nrpe_ok=scripts\ok.bat
;check_multi_line=scripts\multi_line.bat
;#
;# The sample scripts
;#
;check_long=scripts\long.bat
;check_ok=scripts\ok.bat
;check_nok=scripts\xlong.bat
;check_vbs=cscript.exe //T:30 //NoLogo scripts\check_vb.vbs

;# REMOTE NRPE PROXY COMMANDS
;  A list of commands that check other hosts.
;  Used by the NRPECLient module
[NRPE Client Handlers]
check_other=-H 192.168.0.1 -p 5666 -c remote_command -a arguments

;# LUA SCRIPT SECTION
;  A list of all Lua scripts to load.
;[LUA Scripts]
;scripts\test.lua
**Edit**
With my good friend google I found someone trying a test via port 5666.

Code: Select all

$ telnet ***.***.***.*** 5666
Trying ***.***.***.***...
Connected to ***.***.***.***
Escape character is '^]'.
Connection closed by foreign host.

Re: Check_nt failing for only one server

Posted: Fri May 10, 2013 9:43 am
by sreinhardt
Welcome to nagios! 5666 and 5667 specifically are for nrdp and nsca, two protocols that nslcient does support however require additional configuration and are not used with check_nt. Just in case you are not aware, to explain the telnet command you ran previously this just checks if the port is open for nagios to connect to and in this case does not appear to be so. I would suggest a couple things since you already showed that nsclientlistner is not commented out in your configuration.

1) restart the nsclient service on your remote windows machine.
2) verify the windows firewall allows your nagios server or all internal lan traffic to access TCP port 12489 incoming.
3) verify that you do not have any transparent firewalls or routers between your and the remote device. These could also limit what ports are accessable, such situations would be if this machine is in a DMZ or other network segment from the nagios server.
4) Once those are done, lets check the port again with another tool just to verify. On the nagios server run:

Code: Select all

nmap [Windows IP Address] -p 12489
so for me it might look like:
nmap 192.168.168.2 -p 12489
Edit, I did see your notes above about checking the firewall, but it never hurts to verify.