NCPA Unknown Error Occurred while running the plugin

[matthewk2]

I have a number of services which retrieve statistics from Windows clients running NCPA version 1.8.1. They generally are calling a powershell script as a plugin.
Occasionally the service check generated an;

UNKNOWN: Error occurred while running the plugin. Use the verbose flag for more detail.

With the 'SuperVerbose' switch I get the following output;

The stack trace:Traceback (most recent call last):
File "/usr/local/nagios/libexec_ncpa.py", line 266, in main
info_json = get_json(options)
File "/usr/local/nagios/libexec_ncpa.py", line 220, in get_json
filename, _ = urlretrieve(url)
File "/usr/lib64/python2.6/urllib.py", line 93, in urlretrieve
return _urlopener.retrieve(url, filename, reporthook, data)
File "/usr/lib64/python2.6/urllib.py", line 267, in retrieve
block = fp.read(bs)
File "/usr/lib64/python2.6/socket.py", line 383, in read
return self.read(buflen)
File "/usr/lib64/python2.6/ssl.py", line 215, in recv
return self.read(buflen)
File "/usr/lib64/python2.6/ssl.py", line 136, in read
return self._sslobj.read(len)
error: [Errno 104] Connection reset by peer

I think that it maybe caused by 1 or more NCPA checks running at the same time, but cannot determine if this is definitely the case.

Thanks,


Matthew

[lmiltchev]

What type of Windows servers are you monitoring - 2003, 2008, etc.? Is the "NCPA Listener" set up with "Startup type" = Automatic (Delayed Start)?
Can you post the entire ncpa_listener.log?

Run the following command on the Nagios XI server and show the output:

Code: Select all

/usr/local/nagios/libexec/check_ncpa.py -V

[matthewk2]

The types of machines are a mix of 2003 and 2008 and the startup type is set to Automatic.

Please see attached a copy of the NCPA log file. During this log file I had a lot of the NCPA Unknown Error messages as I had just clicked 'update all services for this host'

The output from check_ncpa.py -V is '0.3.5'

Thanks,


Matthew

[lmiltchev]

I think that it maybe caused by 1 or more NCPA checks running at the same time, but cannot determine if this is definitely the case.

This is a logical explanation, especially if you can recreate the issue by forcing an immediate check for the host and all its services. We don't really know how long does it usually take to run these powershell scripts. It could take a while to run some of them, for example "EventLogCheck.ps1".

One thing you can try is to increase the timeout. The "default" timeout for "check_ncpa.py" is 15 sec.

For example, you can add "-T 59" to your "check_xi_ncpa_agent" command:

Code: Select all

$USER1$/check_ncpa.py -H $HOSTADDRESS$ -T 59 $ARG1$

Note: I used "59" as the "default" service check timeout in the nagios.cfg is 60 seconds.

Code: Select all

service_check_timeout=60

Let us know if this helped.

[matthewk2]

Hi there,

Thanks you for the reply. Each of these powershell scripts actually report back their run time in the status back to Nagios and each run's for between 0-4 seconds EventLogCheck.ps1. I have increased the timeout on the check_ncpa.py command already, but still get the issue.

I have also had it error when I have been running it within an SSH session and the unknown error message certainly comes back within 60 seconds.

I was hoping that there would be some way to ask check_ncpa.py to retry if it got a failure rather than error, I know this can be handled by Nagios retires, but you still end up with an unknown status in the meantime and potentially a flap condition.

Thanks,


Matthew

[ssax]

I'm wondering if you are running into this here:

http://scie.nti.st/2008/3/14/amazon-s3- ... t-by-peer/

Check your kernel version (and other settings) by running this command:

Code: Select all

uname -a

[matthewk2]

Hi there,

I have run the command and the output is;

2.6.32-573.8.1.el6.x86_64

Thanks,


Matthew

[rkennedy]

Did you try the instructions listed, since you're running 2.6?

Put the following in /etc/sysctl.conf:
# Workaround for TCP Window Scaling bugs in other ppl's equipment:
net.ipv4.tcp_wmem = 4096 16384 512000
net.ipv4.tcp_rmem = 4096 87380 512000

Then run:
sudo sysctl -p

Did this help at all?

[matthewk2]

Hello there,

I have made the change as requested and rebooted. Unfortunately I continue to see the Unknown Error occured while running the plugin message.

Thanks,


Matthew

[ssax]

On one of the windows machines can you try installing an earlier version to see if you still experience it:

https://assets.nagios.com/downloads/ncp ... -1.7.2.exe