NCPA plugins causing Python errors

[onegative]

G 'Day Nagios Support,

So I have a Windows 2012 R2 system that has just installed the NCPA 2.2.0 version on which when executing PowerShell scripts fail. The failure occurs whether you use the agent's GUI or passive configuration. It all appears to have issues with the compiled binary which is for whatever reason unable to determine its embedded/internal Python interpreter. I can successfully run the PowerShell scripts from the command line but not through the agent. This is the very first time I have encountered an agent having this issue even though I have it running on nearly 1000 hosts.

Here is an example of the ERROR. Obviously the references of "C:\Python27" & "C:\ncpa" has to come from the compiled binary because this box doesn't have Python installed and ncpa is not installed on this path. Everything else functions on the agent, just plugins fail.

2020-04-03 12:06:51,994:ERROR:app:Exception on /api/plugins/check_disk_nagios.ps1/ [GET]
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\flask\app.py", line 1817, in wsgi_app
File "C:\Python27\lib\site-packages\flask\app.py", line 1477, in full_dispatch_request
File "C:\Python27\lib\site-packages\flask\app.py", line 1381, in handle_user_exception
File "C:\Python27\lib\site-packages\flask\app.py", line 1475, in full_dispatch_request
File "C:\Python27\lib\site-packages\flask\app.py", line 1461, in dispatch_request
File "C:\ncpa\agent\listener\server.py", line 185, in token_auth_decoration
File "C:\ncpa\agent\listener\server.py", line 946, in api
File "C:\ncpa\agent\listener\nodes.py", line 300, in run_check
File "C:\ncpa\agent\listener\pluginnodes.py", line 107, in execute_plugin
File "C:\Python27\lib\site-packages\gevent\subprocess.py", line 585, in __init__
File "C:\Python27\lib\site-packages\gevent\subprocess.py", line 554, in __init__
File "C:\Python27\lib\site-packages\gevent\subprocess.py", line 898, in _execute_child
WindowsError: [Error 2] The system cannot find the file specified

So I am looking for some way to understand how/why the agent cannot determine its own Python implementation?

Please advise and thanks for your support,
Danny

[ssax]

Those always differ from the actual in the traceback.

Do you have any anti-virus or security software on the system that could be blocking access to the plugin? Did you restart the ncpa_listener service?

Does it happen with a different plugin? (please try).

Triple check the plugin name is propper and try with IP address instead of DNS name (if using DNS name).

Please go to this page after restarting the ncpa_listener service:

Code: Select all

https://NCPAHOST:5693/gui/api

Select plugins from the dropdown, does it show the plugins?

[onegative]

@ssax

Yes I have restarted the listener multiple times.

Yes there is both anti-virus (Sophos) and security software (Symantec) installed like all other hosts, nothing different that I am aware.

The error occurs with any plugin I attempt to utilize.
Here is a different one and displays the same error.
2020-04-06 13:43:13,061:ERROR:app:Exception on /api/plugins/getCPUUtil.ps1 [GET]
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\flask\app.py", line 1817, in wsgi_app
File "C:\Python27\lib\site-packages\flask\app.py", line 1477, in full_dispatch_request
File "C:\Python27\lib\site-packages\flask\app.py", line 1381, in handle_user_exception
File "C:\Python27\lib\site-packages\flask\app.py", line 1475, in full_dispatch_request
File "C:\Python27\lib\site-packages\flask\app.py", line 1461, in dispatch_request
File "C:\ncpa\agent\listener\server.py", line 185, in token_auth_decoration
File "C:\ncpa\agent\listener\server.py", line 948, in api
File "C:\ncpa\agent\listener\pluginnodes.py", line 53, in walk
File "C:\ncpa\agent\listener\pluginnodes.py", line 107, in execute_plugin
File "C:\Python27\lib\site-packages\gevent\subprocess.py", line 585, in __init__
File "C:\Python27\lib\site-packages\gevent\subprocess.py", line 554, in __init__
File "C:\Python27\lib\site-packages\gevent\subprocess.py", line 898, in _execute_child
WindowsError: [Error 2] The system cannot find the file specified

I guess my first thought is why does Python not resolve as I believe this error from other posts I've seen seem to indicate it cannot determine python.exe.

When I get the chance I will try changing the loglevel for both the listener and passive services to debug...and see what other logging might be available because currently I am unsure where to actually look for the possible failure.

Thanks,
Danny

Both using the fqdn or the ipAddr results in the plugins being defined in the API EndPoint and populates the pull-down as well as shown in the png shown below.

fqdn-ncpa.png

ipAddr-ncpa.png

[onegative]

@ssax

So changing the loglevel = debug for both services did not result in any additional logging details about the Error...

Danny

[onegative]

Earlier I was referring to the following as a possible cause...but I am unsure.

https://github.com/NagiosEnterprises/ncpa/issues/478

Danny

[ssax]

I'll reach out to the dev on this and see what he says, will update when I know more. If you don't hear from me by midday tomorrow, please reply to the topic so that it pops up on our dashboard.

Edit: Please check your sophos/security software logs just in case as well.

[ssax]

Please attach your ncpa.cfg with the token removed so we can review it.

When you select the plugin from the web interface and run it, does it say the same message?

Please turn on debugging again and attach the debug info that is returned so we can see what's occurring.

Can you try running it from your XI server or does it have to use check_nrpe.exe from a windows machine?

[onegative]

@ssax

So I perform a couple more test. I executed a .bat script and a .vbs script and both functioned. So that tells me that the problem has to be associated with PowerShell alone...

My guess there is some system restriction that is preventing the ncpa agent from calling ".ps1 = powershell -ExecutionPolicy Bypass -File $plugin_name $plugin_args"

I will try diving deeper, please leave the ticket open until I resolve and I will post what I determine.

Results from the other type of scripts:

bat-ncpa.png

vbs-ncpa.png

[lmiltchev]

I will try diving deeper, please leave the ticket open until I resolve and I will post what I determine.

Sure thing - we will keep the topic open.

Next time you post on the forum, can you attach your ncpa.cfg with the token removed so we can review it? Thanks!

[onegative]

Here ya go...