Setting Up NSCA Client with NSClient and Nagios XI

[phil821]

Hello all,

I am trying to set up passive checks in Nagios XI. The checks are coming from a Windows machine so I'm trying to do this through NSClient ++ 0.4.3.143.

I think I have everything set up properly on the Nagios XI server side (Linux). I've made changes to /usr/local/nagios/etc/nsca.cfg (put in password, encryption/decryption methods). *NSCA is running under xinetd.

I've also traveled to /etc/xinetd.d/nsca and set disabled=no, and restarted xinted.

On the client side, I'm not as confident everything is set up properly. The error I'm getting from my log is

Code: Select all

2015-08-10 08:48:22: error:D:\source\nscp\modules\Scheduler\Scheduler.cpp:122: Failed to submit check_work: Error: Failed to connect to: 192.168.253.107:5667 :A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

I'm assuming this is a firewall issue, however on my Windows machine I have completed disabled the firewall and the Linux server is opened up as well. What's interesting is that, in Windows, when I list the ports, there is nothing listening on 5667. It is only listening on 5666 (for my active nrpe checks) and 12489 (for the nsclient basic checks)

Here is my nsclient.ini file. (I have removed the password)

*Sorry for the unorganized files, you can see entries pertaining to NSCA near the middle and at the very end of the file.

Code: Select all

# If you want to fill this file with all avalible options run the following command:
#   nscp settings --generate --add-defaults --load-all
# If you want to activate a module and bring in all its options use:
#   nscp settings --activate-module <MODULE NAME> --add-defaults
# For details run: nscp settings --help


; Undocumented section
[/settings/default]

; PASSWORD - Password used to authenticate against server
password = 
; ALLOWED HOSTS - A comaseparated list of allowed hosts. You can use netmasks (/ syntax) or * to create ranges.
allowed hosts = 127.0.0.1,nagios.visualfrenzy.com,vfmsrv107.visualfrenzy.com,192.168.253.107

[log]
debug = 1
file = nsclient.log



; Undocumented section
[/settings/NRPE/server]
allow arguments = true




; ALLOW INSECURE CHIPHERS and ENCRYPTION - Only enable this if you are using legacy check_nrpe client.
insecure = true

; Undocumented key
allow nasty characters = false


allowed hosts = 127.0.0.1,nagios.visualfrenzy.com,vfmsrv107.visualfrenzy.com,192.168.253.107
port = 5666


; Undocumented section
[/modules]

; NRPEServer - A server that listens for incoming NRPE connection and processes incoming requests.
NRPEServer = enabled

; CheckSystem - Various system related checks, such as CPU load, process state, service state memory usage and PDH 

counters.
CheckSystem = enabled

; NSClientServer - A server that listens for incoming check_nt connection and processes incoming requests.
NSClientServer = enabled

; CheckExternalScripts - Execute external scripts
CheckExternalScripts = enabled

; CheckHelpers - Various helper function to extend other checks.
CheckHelpers = 1

; CheckEventLog - Check for errors and warnings in the event log.
CheckEventLog = 1

; CheckNSCP - Use this module to check the healt and status of NSClient++ it self
CheckNSCP = 1

; CheckDisk - CheckDisk can check various file and disk related things.
CheckDisk = 1

Scheduler=enabled

NSCAClient=enabled

[/settings/scheduler/schedules/default]
interval=5m


[/settings/external scripts]
allow arguments = true


[/settings/external scripts/scripts]
check_updates = cscript.exe //NoLogo //T:40 scripts\check_updates.wsf

windows_updates = c:\windows\system32\cscript.exe //NoLogo //T:120 scripts\check_windows_updates.wsf /w:0 /c:1

check_work = cmd /c echo scripts\check_work.ps1 "$ARG1$"; exit($lastexitcode) | powershell.exe scripts

\check_work.ps1 "$ARG1$"

check_veeam_backups = cmd /c echo scripts\check_veeam_backups.ps1 "$ARG1$" "$ARG2$" "$ARG3$"; exit($lastexitcode) | 

powershell.exe scripts\check_veeam_backups.ps1 "$ARG1$" "$ARG2$" "$ARG3$"

check_backupexec_backups = cmd /c echo scripts\check_backupexec_backups.ps1 "$ARG1$" "$ARG2$" "$ARG3$"; exit

($lastexitcode) | powershell.exe scripts\check_backupexec_backups.ps1 "$ARG1$" "$ARG2$" "$ARG3$"

check_windowsupdates=cmd /c echo scripts\RoughDraftUpdates.ps1; exit $LastExitCode | powershell.exe -command -

; A list of templates for wrapped scripts.
[/settings/external scripts/wrappings]

; POWERSHELL WRAPPING - 
ps1 = cmd /c echo scripts\\%SCRIPT% %ARGS%; exit($lastexitcode) | powershell.exe -command -

; BATCH FILE WRAPPING - 
bat = scripts\\%SCRIPT% %ARGS%

; VISUAL BASIC WRAPPING - 
vbs = cscript.exe //T:30 //NoLogo scripts\\lib\\wrapper.vbs %SCRIPT% %ARGS%

; A list of aliases available.
An alias is an internal command that has been predefined to provide a single command without arguments. Be careful 

so you don't create loops (ie check_loop = check_a, check_a=check_loop)


[/settings/NSCA/server]
encryption = 0

port = 5667

performance data = true 
 
password = 
[/settings/external scripts/alias]

; alias_volumes_loose - Alias for alias_volumes_loose. To configure this item add a section called: 

/settings/external scripts/alias/alias_volumes_loose
alias_volumes_loose = check_drivesize

alias_check_work = check_work "yoasdasdlo"

; alias_volumes - Alias for alias_volumes. To configure this item add a section called: /settings/external 

scripts/alias/alias_volumes
alias_volumes = check_drivesize

; alias_sched_all - Alias for alias_sched_all. To configure this item add a section called: /settings/external 

scripts/alias/alias_sched_all
alias_sched_all = check_tasksched show-all "syntax=${title}: ${exit_code}" "crit=exit_code ne 0"

; alias_process_stopped - Alias for alias_process_stopped. To configure this item add a section called: 

/settings/external scripts/alias/alias_process_stopped
alias_process_stopped = check_process "process=$ARG1$" "crit=state != 'stopped'"

; alias_service - Alias for alias_service. To configure this item add a section called: /settings/external 

scripts/alias/alias_service
alias_service = check_service

; alias_process_hung - Alias for alias_process_hung. To configure this item add a section called: 

/settings/external scripts/alias/alias_process_hung
alias_process_hung = check_process "filter=is_hung" "crit=count>0"

; alias_process_count - Alias for alias_process_count. To configure this item add a section called: 

/settings/external scripts/alias/alias_process_count
alias_process_count = check_process "process=$ARG1$" "warn=count > $ARG2$" "crit=count > $ARG3$"

; alias_process - Alias for alias_process. To configure this item add a section called: /settings/external 

scripts/alias/alias_process
alias_process = check_process "process=$ARG1$" "crit=state != 'started'"

; alias_mem - Alias for alias_mem. To configure this item add a section called: /settings/external 

scripts/alias/alias_mem
alias_mem = check_memory

; alias_file_size - Alias for alias_file_size. To configure this item add a section called: /settings/external 

scripts/alias/alias_file_size
alias_file_size = check_files "path=$ARG1$" "crit=size > $ARG2$" "top-syntax=${list}" "detail-syntax=${filename] 

${size}" max-dir-depth=10

; alias_disk - Alias for alias_disk. To configure this item add a section called: /settings/external 

scripts/alias/alias_disk
alias_disk = check_drivesize

; alias_cpu_ex - Alias for alias_cpu_ex. To configure this item add a section called: /settings/external 

scripts/alias/alias_cpu_ex
alias_cpu_ex = check_cpu "warn=load > $ARG1$" "crit=load > $ARG2$" time=5m time=1m time=30s

; alias_file_age - Alias for alias_file_age. To configure this item add a section called: /settings/external 

scripts/alias/alias_file_age
alias_file_age = check_files "path=$ARG1$" "crit=written > $ARG2$" "top-syntax=${list}" "detail-syntax=${filename] 

${written}" max-dir-depth=10

; alias_cpu - Alias for alias_cpu. To configure this item add a section called: /settings/external 

scripts/alias/alias_cpu
alias_cpu = check_cpu

; alias_event_log - Alias for alias_event_log. To configure this item add a section called: /settings/external 

scripts/alias/alias_event_log
alias_event_log = check_eventlog

; alias_service_ex - Alias for alias_service_ex. To configure this item add a section called: /settings/external 

scripts/alias/alias_service_ex
alias_service_ex = check_service "exclude=Net Driver HPZ12" "exclude=Pml Driver HPZ12" exclude=stisvc

; alias_up - Alias for alias_up. To configure this item add a section called: /settings/external 

scripts/alias/alias_up
alias_up = check_uptime

; alias_disk_loose - Alias for alias_disk_loose. To configure this item add a section called: /settings/external 

scripts/alias/alias_disk_loose
alias_disk_loose = check_drivesize

; alias_sched_task - Alias for alias_sched_task. To configure this item add a section called: /settings/external 

scripts/alias/alias_sched_task
alias_sched_task = check_tasksched show-all "filter=title eq '$ARG1$'" "detail-syntax=${title} (${exit_code})" 

"crit=exit_code ne 0"

; alias_sched_long - Alias for alias_sched_long. To configure this item add a section called: /settings/external 

scripts/alias/alias_sched_long
alias_sched_long = check_tasksched "filter=status = 'running'" "detail-syntax=${title} (${most_recent_run_time})" 

"crit=most_recent_run_time < -$ARG1$"

[/settings/scheduler/schedules]
check_work = alias_check_work

[/settings/NSCA/client]
hostname = vfm-dsk-193

[/settings/NSCA/client/targets/default]
address = 192.168.253.107
password = 

[tgriep]

One thing I see that is missing is the encryption isn't set in the /settings/NSCA/client/targets/default section.
You may want to add the following to that section but change the type of encryption you are using.

Code: Select all

encryption=des

Lets test to see if the Nagios system is seeing packets from your remote system.
Run the following and post back the results.

Code: Select all

tcpdump port 5667

Let it run for 10 minutes.

[phil821]

Thanks for the reply.

I made the changes and ran tcp dump

Here is the output

Code: Select all

listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:32:33.991800 IP 192.168.250.152.49502 > 192.168.253.107.nsca: Flags [S], seq                             2269368154, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
11:32:36.991702 IP 192.168.250.152.49502 > 192.168.253.107.nsca: Flags [S], seq                             2269368154, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
11:32:42.987521 IP 192.168.250.152.49502 > 192.168.253.107.nsca: Flags [S], seq                             2269368154, win 8192, options [mss 1460,nop,nop,sackOK], length 0

The above IP's are both the server and the machine so that's a good sign

It still is not showing up in Nagios, however.

Here is output from nsclient.log

Code: Select all

2015-08-11 11:33:03: error:D:\source\nscp\modules\Scheduler\Scheduler.cpp:122: Failed to submit check_work: Error: Failed to connect to: 192.168.253.107:5667 :A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

NMAP is still saying that from server to client, 5667 is closed (again 5666, 12489 is open) Firewall on my Windows 7 machine is turned off, 5667 is currently not listening. Server side firewall allows all both directions.

Code: Select all

PORT     STATE  SERVICE
5667/tcp closed unknown

This is the output I get from nscp test when I check the command currently scheduled

Code: Select all

D ext-script Command line: cmd /c echo scripts\check_work.ps1 "$ARG1$"; exit($lastexitcode) | powershell.exe scripts\che
ck_work.ps1 "$ARG1$"
L        cli Long message
OK: yes this workasdasdasds
$[0]
L        cli  Performance data: 'test'=123

I'm starting to think this is a server side problem

I have tripled checked iptables

Here is the output from /etc/xinetd.d/nsca

Code: Select all

service nsca
{
        flags           = REUSE
        socket_type     = stream
        wait            = no
        user            = nagios
        group           = nagios
        server          = /usr/local/nagios/bin/nsca
        server_args     = -c /usr/local/nagios/etc/nsca.cfg --inetd
        log_on_failure  += USERID
        disable         = no
}

/usr/local/nagios/etc/nsca.cfg is pretty standard, I just inputted my password and uncommented "only_from" line



My shell seems to hang everytime I cat /usr/local/nagios/var/rw/nagios.cmd

ps -ef | grep nsca | grep -v grep returns nothing

[jolson]

My shell seems to hang everytime I cat /usr/local/nagios/var/rw/nagios.cmd

nagios.cmd is a named pipe, and cannot be read via cat as a normal file can. What you're looking to do is follow the nagios.log file:

Code: Select all

tail -f /usr/local/nagios/var/nagios.log

The above file should display any passive checks that come in.

Based on the facts that tcpdump is seeing information and you have verified your iptables configuration, I am thinking that nsca may not be listening properly. Let's verify that the daemon is functioning properly.

First, we'll restart xinetd:

Code: Select all

service xinetd restart

Next, ensure that port 5667 is listening:

Code: Select all

netstat -na | grep 5667

Let's check out your configuration file for nsca:

Code: Select all

cat /usr/local/nagios/etc/nsca

Your NSClient configuration looks fine to me.

Xinetd (known as a super-daemon) handles the responsibility of starting and stopping nsca on demand - I am hoping that restarting xinetd may do the trick. If not, I'll test this further on my end. Let us know!

[phil821]

Thanks for your reply

Code: Select all

####################################################
# Sample NSCA Daemon Config File
# Written by: Ethan Galstad ([email protected])
#
# Last Modified: 04-03-2006
####################################################


# PID FILE
# The name of the file in which the NSCA daemon should write it's process ID
# number.  The file is only written if the NSCA daemon is started by the root
# user as a single- or multi-process daemon.

pid_file=/var/run/nsca.pid



# PORT NUMBER
# Port number we should wait for connections on.
# This must be a non-priveledged port (i.e. > 1024).

server_port=5667



# SERVER ADDRESS
# Address that NSCA has to bind to in case there are
# more as one interface and we do not want NSCA to bind
# (thus listen) on all interfaces.

#server_address=192.168.1.1



# NSCA USER
# This determines the effective user that the NSCA daemon should run as.
# You can either supply a username or a UID.
#
# NOTE: This option is ignored if NSCA is running under either inetd or xinetd

nsca_user=nagios



# NSCA GROUP
# This determines the effective group that the NSCA daemon should run as.
# You can either supply a group name or a GID.
#
# NOTE: This option is ignored if NSCA is running under either inetd or xinetd

nsca_group=nagios



# NSCA CHROOT
# If specified, determines a directory into which the nsca daemon
# will perform a chroot(2) operation before dropping its privileges.
# for the security conscious this can add a layer of protection in
# the event that the nagios daemon is compromised.
#
# NOTE: if you specify this option, the command file will be opened
#       relative to this directory.

#nsca_chroot=/var/run/nagios/rw



# DEBUGGING OPTION
# This option determines whether or not debugging
# messages are logged to the syslog facility.
# Values: 0 = debugging off, 1 = debugging on

debug=1



# COMMAND FILE
# This is the location of the Nagios command file that the daemon
# should write all service check results that it receives.

command_file=/usr/local/nagios/var/rw/nagios.cmd


# ALTERNATE DUMP FILE
# This is used to specify an alternate file the daemon should
# write service check results to in the event the command file
# does not exist.  It is important to note that the command file
# is implemented as a named pipe and only exists when Nagios is
# running.  You may want to modify the startup script for Nagios
# to dump the contents of this file into the command file after
# it starts Nagios.  Or you may simply choose to ignore any
# check results received while Nagios was not running...

alternate_dump_file=/usr/local/nagios/var/rw/nsca.dump



# AGGREGATED WRITES OPTION
# This option determines whether or not the nsca daemon will
# aggregate writes to the external command file for client
# connections that contain multiple check results.  If you
# are queueing service check results on remote hosts and
# sending them to the nsca daemon in bulk, you will probably
# want to enable bulk writes, as this will be a bit more
# efficient.
# Values: 0 = do not aggregate writes, 1 = aggregate writes

aggregate_writes=0



# APPEND TO FILE OPTION
# This option determines whether or not the nsca daemon will
# will open the external command file for writing or appending.
# This option should almost *always* be set to 0!
# Values: 0 = open file for writing, 1 = open file for appending

append_to_file=0



# MAX PACKET AGE OPTION
# This option is used by the nsca daemon to determine when client
# data is too old to be valid.  Keeping this value as small as
# possible is recommended, as it helps prevent the possibility of
# "replay" attacks.  This value needs to be at least as long as
# the time it takes your clients to send their data to the server.
# Values are in seconds.  The max packet age cannot exceed 15
# minutes (900 seconds).  If this variable is set to zero (0), no
# packets will be rejected based on their age.

max_packet_age=30



# DECRYPTION PASSWORD
# This is the password/passphrase that should be used to descrypt the
# incoming packets.  Note that all clients must encrypt the packets
# they send using the same password!
# IMPORTANT: You don't want all the users on this system to be able
# to read the password you specify here, so make sure to set
# restrictive permissions on this config file!

password=asdasdasdasdasdasd



# DECRYPTION METHOD
# This option determines the method by which the nsca daemon will
# decrypt the packets it receives from the clients.  The decryption
# method you choose will be a balance between security and performance,
# as strong encryption methods consume more processor resources.
# You should evaluate your security needs when choosing a decryption
# method.
#
# Note: The decryption method you specify here must match the
#       encryption method the nsca clients use (as specified in
#       the send_nsca.cfg file)!!
# Values:
#
#       0 = None        (Do NOT use this option)
#       1 = Simple XOR  (No security, just obfuscation, but very fast)
#
#       2 = DES
#       3 = 3DES (Triple DES)
#       4 = CAST-128
#       5 = CAST-256
#       6 = xTEA
#       7 = 3WAY
#       8 = BLOWFISH
#       9 = TWOFISH
#       10 = LOKI97
#       11 = RC2
#       12 = ARCFOUR
#
#       14 = RIJNDAEL-128
#       15 = RIJNDAEL-192
#       16 = RIJNDAEL-256
#
#       19 = WAKE
#       20 = SERPENT
#
#       22 = ENIGMA (Unix crypt)
#       23 = GOST
#       24 = SAFER64
#       25 = SAFER128
#       26 = SAFER+
#

decryption_method=0

Again, I barely touched the file, but there it is.

Code: Select all

[root@server ~]# netstat -na | grep 5667
tcp        0      0 :::5667                     :::*                        LISTEN

Here is the output from my windows client

Code: Select all

 TCP    0.0.0.0:2049           0.0.0.0:0              LISTENING       3776
 TCP    0.0.0.0:3389           0.0.0.0:0              LISTENING       1464
 TCP    0.0.0.0:5666           0.0.0.0:0              LISTENING       1588
 TCP    0.0.0.0:5666           0.0.0.0:0              LISTENING       1588
 TCP    0.0.0.0:6160           0.0.0.0:0              LISTENING       3836
....
....
...
 TCP    0.0.0.0:10001          0.0.0.0:0              LISTENING       5732
 TCP    0.0.0.0:10003          0.0.0.0:0              LISTENING       1800
 TCP    0.0.0.0:12489          0.0.0.0:0              LISTENING       1588
 TCP    0.0.0.0:12489          0.0.0.0:0              LISTENING       1588
 TCP    0.0.0.0:49152          0.0.0.0:0              LISTENING       680

[Box293]

I have some steps here that might help with testing:

http://sites.box293.com/nagios/guides/n ... core-4-0-x

Specifically you can use the send_nsca command on the Nagios server itself to send an NSCA result, this then removes NSClient++ from the picture.

Also, is there a router/firewall that separates the Windows server from the Nagios XI sever? If yes, is that allowing port 5667 through?

These steps may help with testing from the Windows side:

http://sites.box293.com/nagios/guides/n ... ient-0-4-x

[phil821]

The tests I ran were successful on the server side, I got the exact same output as expected.

So its safe to say this is a problem with the client?

My thinking is that it is not listening properly, when I scan ports on the windows machines only 5666 and 12489 are listening.

When I restart NSClient, there is a line about binding to both 5666 and 12489 but again, no 5667

Code: Select all

2015-08-07 15:28:28: debug:D:\source\nscp\include\socket/server.hpp:95: Binding to: 0.0.0.0:12489(ipv4), reopen: true, reuse: true

2015-08-07 15:28:28: debug:D:\source\nscp\include\socket/server.hpp:95: Binding to: 0.0.0.0:5666(ipv4), reopen: true, reuse: true

[phil821]

Would you say that the steps so set up a passive check in the GUI as outlined here are still accurate?

https://assets.nagios.com/downloads/nag ... ios-XI.pdf

This is more or less what I have followed. Do you need to do anything specifically to tie the specific passive service on the Windows Machine to the check set up in the GUI? As far as I can tell, you only specify the source address of the client. I'm not sure how Nagios would organize multiple passive checks from the same host. Unless the service name has to be the exact same as what NSClient is sending (mine is currently not btw but Im going to try now)

[tgriep]

In your nsclient.ini file what did you add for the encryption= line?
If you used my example from 2 days ago, you will have to match that in the nsca.cfg file for the decryption_method=2

[phil821]

nsclient.ini has

Code: Select all

[/settings/NSCA/client/targets/default]

address = 192.168.253.107

port = 5667

password = vgt8BaWWCfj0a4r8kzif

encryption = des

use ssl = false

nsca.cfg has

Code: Select all

decryption_method=2