Page 1 of 1

NSClient++ check_logfile NRPE not alerting

Posted: Wed Feb 05, 2020 1:43 pm
by mannyjacobs73
Hi,

I'm having an issue getting a Nagios alert generated when wanting to add monitoring on a file on a remote Windows system.

I have followed documentation, in particular "Demo 3: Passive realtime checks via NRPE" here: https://www.medin.name/blog/2012/10/26/ ... sentation/ and put in the configuration below, but for the moment I'm just not understanding why I retrieve the check and file extract (when adding 'show-all') on my Nagios server but receive no alerts for the lines.

Sample file content:
INFORMATION : LOG : Creating Log on 05/02/2020 at 08:10:00
INFORMATION : Application : Auto import enabled! at 08:10:01
INFORMATION : Error : Unable to connect at 08:10:04

There are time entries in the log file (for example 08:10:01), but I have set the "column separator / column split to " : " to overcome this.

My Nagios Server - v4.4.5
NSClient++ on Windows client - vNSCP-0.5.2.35-x64

Disk, CPU and Memory checks have already been working fine for the client for many months.

On the Client, I have added the sections below in the nsclient.ini file and restarted services with each change:

; MODULES
[/modules]
.......

; Load Check_LogFile Module
CheckLogFile = enabled
SimpleCache = enabled
SimpleFileWriter = enabled

; ACTIVATE REAL-TIME SUPERVISION
[/settings/logfile/real-time]
enabled = true

; CREATE SEARCH FILTER
[/settings/logfile/real-time/checks/logfile]
file = C:\test\logfile.txt
destination = CACHE,FILE
column-split=' : '
filter = column1 like 'INFORMATION'
warn = column2 like 'Error'


From my Nagios server command line, if I run /usr/local/nagios/libexec/check_nrpe -H <IP> -c check_logfile -n -a file='C:\test\logfile.txt' column-split=' : ' filter="column1 like 'INFORMATION'" warn="column2 like 'Error'"

I receive output below indicating one warning threshold found, so I believe the nrpe check from my server is working:
217/222 (INFORMATION )


My check command in my nagios config is:
check_command check_nrpe!check_logfile -n -a file='C:\test\logfile.txt' column-split="' : '" filter="column1 like 'INFORMATION'" warn="column2 like 'ERROR'" show-all


Any pointers in the right direction would be appreciated.


Thank you

Re: NSClient++ check_logfile NRPE not alerting

Posted: Wed Feb 05, 2020 6:54 pm
by Box293
mannyjacobs73 wrote:From my Nagios server command line, if I run /usr/local/nagios/libexec/check_nrpe -H <IP> -c check_logfile -n -a file='C:\test\logfile.txt' column-split=' : ' filter="column1 like 'INFORMATION'" warn="column2 like 'Error'"
After running that from the command line, immediately execute this command:

Code: Select all

echo $?
What output do you receive? This is the exit code which tells Nagios if the result is OK (0) Warning (1) Critical (2) Unknown (3).

I know you pointed to some documentation however it looks like it's been created in 2012. I know the author of NSClient++ is constantly making improvements so I would check his offical documentation to see if there are any newer examples.

On a completely separate note, I don't recommend using Nagios XI for log file monitoring. Nagios Log Server is a much better solution. The reason is that when using Nagios XI you are constantly polling the server and the same logs. With Nagios Log Server, the client sends the logs to Nagios Log Server and then any queries or reporting are performed on Nagios Log Server and the client is not polled all the time.

Re: NSClient++ check_logfile NRPE not alerting

Posted: Thu Feb 06, 2020 4:15 am
by mannyjacobs73
Hi,

I tested the output from the command line with "echo $?" and do in fact receive (1) or (2), depending if I use warn or crit, so this definitely seems to be working from my Nagios server.

I'll look to find updated documentation and perhaps post a query on a NSClient++ dedicated forum, but the one at https://forums.nsclient.org/ looks down and I don't think this is an issue to post a query to their github.

Apologies but I probably should have mentioned my Nagios server is running Core, not XI.


Alternatively, I'll need to look for a different solution but it's a pity because I'm 99% confident my config is ok, but just don't understand right now why no alerts are generated :(

Oh, and just to add, I do have alerting set and the check period is active within my Nagios config

Re: NSClient++ check_logfile NRPE not alerting

Posted: Thu Feb 06, 2020 5:19 pm
by Box293
What is being generated in your nagios.log file when the service is triggering a warning or critical state? Remember alerts won't be sent until max_check_attempts is reached once a service enters a SOFT state.

I would enable debug logging to see what commands and arguments are being sent to NSClient and what is received back.

Try setting the debug level on and then restart Nagios.

Code: Select all

sed -i 's/.*debug_level=.*/debug_level=-1/g' /usr/local/nagios/etc/nagios.cfg
service nagios restart
Force an immediate check of the service and then inspect /usr/local/nagios/var/nagios.debug for what happened. I recommend taking a copy of the nagios.debug file once the result is received for NSClient as the file generates a lot of data and you may miss the lines if they are rotated over too quickly.

When you are finished this turns debugging off:

Code: Select all

sed -i 's/.*debug_level=.*/debug_level=0/g' /usr/local/nagios/etc/nagios.cfg
service nagios restart

Re: NSClient++ check_logfile NRPE not alerting

Posted: Tue Feb 11, 2020 3:27 am
by mannyjacobs73
Thank you for those tips!

I've continued to test this and following (http://nagios.sourceforge.net/download/ ... _Setup.pdf), also setup and configured NSCA on my Nagios server (which wasn't configured previously) - I thought configuring for passive checks might be an easier way to get it working.


I've set the config below, have the check on Nagios, which shows
" logfile UNKNOWN 02-11-2020 09:17:11 0d 0h 2m 0s 1/1 Need to specify at least one file: file=foo.txt "

I think it's clear to me it's obviously not a bug and just me not completely understanding how both sides should work together ;-(


On My Nagios Server, I have defined a passive service:
--------------------------------------------------------------------
define service {
use generic-service
name passive-service
.....
check_command logfile!0
....
}


a command definition: --> I have set a 'logfile = check_logfile' in my NSClient++ configuration below

define command {
command_name logfile
command_line $USER1$/check_logfile $ARG1$
}


A service check:

define service {
service_description logfile
host_name <Hostname of my Windows system>
use passive-service
}
--------------------------------------------------------------------



The related Windows system NSClient++ config looks like:
--------------------------------------------------------------------

[/modules]
....
NRPEServer = enabled

CheckExternalScripts = enabled
CheckLogFile = enabled

NSCAClient = enabled

SimpleCache = enabled
SimpleFileWriter = enabled

Scheduler = enabled

[/settings/scheduler/schedules/default]
interval = 1m

[/settings/scheduler/schedules]
logfile = check_logfile

[/settings/NSCA/client]
hostname = <My Windows Server Hostname - same as on Nagios Server>

[/settings/NSCA/client/targets/default]
address = nsca://<Nagios Server IP>:5667
encryption = 0
password = <password -- same as in nsca.cfg on my Nagios Server>
use ssl = 0
verify mode = none

[/settings/logfile/real-time]
enabled = true

[/settings/logfile/real-time/checks/logfile]
file = C:\test\logfile.txt
read entire file = true
destination = NSCA,CACHE,FILE
column split=' : '
filter = column1 like 'INFORMATION'
critical = column2 like 'LOG'
--------------------------------------------------------------------


I need to work on something else for the moment, so will look to restart the config from scratch and re-look at this again next week.

With a clean config, and what I've learned the past few days, I hope to get it right then. I'll update my post though, either with my resolution, or with more educated questions.

Re: NSClient++ check_logfile NRPE not alerting

Posted: Wed Feb 12, 2020 11:12 am
by lmiltchev
With a clean config, and what I've learned the past few days, I hope to get it right then. I'll update my post though, either with my resolution, or with more educated questions.
Sounds good! We will keep the thread open for the time being.

Since, you are trying to set up real-time checks with NSClient++, please review the guide below:

https://outsideit.net/real-time-eventlog-monitoring/

Let us know if you get stuck on something.