NCPA

ignacio.sanchez · Post by **ignacio.sanchez** » Fri Oct 04, 2024 6:11 am

Hello,

We have an issue with NCPA (tried with v2 and v3)

If I restart the service, everything starts working well.

2024-10-04 08:07:49,881 root DEBUG Started log <RootLogger root (DEBUG)>! Handlers: [<RotatingFileHandler C:\Program Files\Nagios\NCPA\var\log\ncpa_listener.log (DEBUG)>]
2024-10-04 08:07:49,881 root INFO main - Python version: 3.12.6 (tags/v3.12.6:a4a2d2b, Sep  6 2024, 20:11:23) [MSC v.1940 64 bit (AMD64)]
2024-10-04 08:07:49,881 root INFO main - SSL version: OpenSSL 3.0.15 3 Sep 2024
2024-10-04 08:07:49,881 root INFO main - ZLIB version: 1.3.1
2024-10-04 08:07:49,881 parent DEBUG ---------------- Winservice.initialize()
2024-10-04 08:07:49,881 parent DEBUG get_options()
2024-10-04 08:07:49,881 parent DEBUG get_configuration()
2024-10-04 08:07:49,881 parent DEBUG get_filename(etc\ncpa.cfg)
2024-10-04 08:07:49,881 parent DEBUG get_filename(etc\ncpa.cfg.d\*.cfg)
2024-10-04 08:07:49,881 parent DEBUG get_filename(plugins/)
2024-10-04 08:07:49,881 parent DEBUG Looking for plugins at: C:\Program Files\Nagios\NCPA\plugins
2024-10-04 08:07:49,881 parent DEBUG get_filename(var/log/ncpa_listener.log)
2024-10-04 08:07:49,881 listener DEBUG Winservice.init_logger() - Name: listener, File: C:\Program Files\Nagios\NCPA\var\log\ncpa_listener.log
2024-10-04 08:07:49,881 listener DEBUG Started log <Logger listener (DEBUG)>! Handlers: [<RotatingFileHandler C:\Program Files\Nagios\NCPA\var\log\ncpa_listener.log (DEBUG)>]
2024-10-04 08:07:49,881 listener DEBUG Started log <Logger listener (DEBUG)>! Handlers: [<RotatingFileHandler C:\Program Files\Nagios\NCPA\var\log\ncpa_listener.log (DEBUG)>]
2024-10-04 08:07:54,350 listener DEBUG Started log <Logger listener (DEBUG)>! Handlers: [<RotatingFileHandler C:\Program Files\Nagios\NCPA\var\log\ncpa_listener.log (DEBUG)>]
2024-10-04 08:07:54,350 listener INFO run()
2024-10-04 08:07:54,350 listener DEBUG delay_start: 0
2024-10-04 08:07:54,350 listener DEBUG address1: X.X.X.X
2024-10-04 08:07:54,350 listener DEBUG port: 5693
2024-10-04 08:07:54,350 listener DEBUG ssl_str_ciphers: 
2024-10-04 08:07:54,350 listener DEBUG Using SSL version TLSv1_2
2024-10-04 08:07:54,350 listener DEBUG max_connections: 200
2024-10-04 08:07:54,350 listener DEBUG Start create cert
2024-10-04 08:07:54,350 listener DEBUG Cert created
2024-10-04 08:07:54,350 listener DEBUG run() - define http_server
2024-10-04 08:07:54,350 listener DEBUG run() - start http_server

Code: Select all

2024-10-04 08:08:24,392 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:24,392 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/memory/virtual/?token=********&warning=85&critical=90&units=Gi&check=1
2024-10-04 08:08:24,392 listener DEBUG     before_request() - request.path: /api/memory/virtual/
2024-10-04 08:08:24,392 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:24,392 listener DEBUG     before_request() - request.view_args: {'accessor': 'memory/virtual/'}
2024-10-04 08:08:24,392 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:25,519 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:25,519 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:30,324 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:30,324 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/cpu/percent/?token=********&warning=85&critical=95&check=1&aggregate=avg
2024-10-04 08:08:30,324 listener DEBUG     before_request() - request.path: /api/cpu/percent/
2024-10-04 08:08:30,324 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:30,324 listener DEBUG     before_request() - request.view_args: {'accessor': 'cpu/percent/'}
2024-10-04 08:08:30,324 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:31,433 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:31,433 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:34,906 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:34,906 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/memory/swap/?token=********&warning=85&critical=90&units=Gi&check=1
2024-10-04 08:08:34,906 listener DEBUG     before_request() - request.path: /api/memory/swap/
2024-10-04 08:08:34,906 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:34,906 listener DEBUG     before_request() - request.view_args: {'accessor': 'memory/swap/'}
2024-10-04 08:08:34,906 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:36,016 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:36,016 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:36,484 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:36,484 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/services/?token=********&check=1&service=SentinelAgent&status=running
2024-10-04 08:08:36,484 listener DEBUG     before_request() - request.path: /api/services/
2024-10-04 08:08:36,484 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:36,484 listener DEBUG     before_request() - request.view_args: {'accessor': 'services/'}
2024-10-04 08:08:36,484 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:36,625 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:36,641 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:37,611 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:37,611 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/system/version/?token=********&check=1
2024-10-04 08:08:37,611 listener DEBUG     before_request() - request.path: /api/system/version/
2024-10-04 08:08:37,611 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:37,611 listener DEBUG     before_request() - request.view_args: {'accessor': 'system/version/'}
2024-10-04 08:08:37,611 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:37,720 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:37,720 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:37,720 listener INFO Did not receive normal values. Unable to find meaningful check.
2024-10-04 08:08:39,115 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:39,115 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/services/?token=********&check=1&service=NCPA&status=running
2024-10-04 08:08:39,115 listener DEBUG     before_request() - request.path: /api/services/
2024-10-04 08:08:39,115 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:39,115 listener DEBUG     before_request() - request.view_args: {'accessor': 'services/'}
2024-10-04 08:08:39,115 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:39,209 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:39,209 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:42,576 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:42,576 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/system/uptime/?token=********&critical=3888000&check=1
2024-10-04 08:08:42,576 listener DEBUG     before_request() - request.path: /api/system/uptime/
2024-10-04 08:08:42,576 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:42,576 listener DEBUG     before_request() - request.view_args: {'accessor': 'system/uptime/'}
2024-10-04 08:08:42,576 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:43,697 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:43,697 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:43,947 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:43,947 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/services/?token=********&check=1&service=TermService&status=running
2024-10-04 08:08:43,947 listener DEBUG     before_request() - request.path: /api/services/
2024-10-04 08:08:43,947 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:43,947 listener DEBUG     before_request() - request.view_args: {'accessor': 'services/'}
2024-10-04 08:08:43,947 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:44,087 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:44,087 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:44,150 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:44,150 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/services/?token=********&check=1&service=SentinelStaticEngine&status=running
2024-10-04 08:08:44,150 listener DEBUG     before_request() - request.path: /api/services/
2024-10-04 08:08:44,150 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:44,150 listener DEBUG     before_request() - request.view_args: {'accessor': 'services/'}
2024-10-04 08:08:44,150 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:44,259 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:44,275 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:47,224 listener DEBUG     before_request() - type(request.view_args): <class 'dict'>
2024-10-04 08:08:47,224 listener INFO before_request() - request.url: https://X.X.X.X:5693/api/system/agent_version/?token=********&check=1
2024-10-04 08:08:47,224 listener DEBUG     before_request() - request.path: /api/system/agent_version/
2024-10-04 08:08:47,224 listener DEBUG     before_request() - request.url_rule: /api/<path:accessor>
2024-10-04 08:08:47,224 listener DEBUG     before_request() - request.view_args: {'accessor': 'system/agent_version/'}
2024-10-04 08:08:47,224 listener DEBUG     before_request() - request.routing_exception: None
2024-10-04 08:08:47,333 listener DEBUG Imported windowscounters into the API tree.
2024-10-04 08:08:47,333 listener DEBUG Imported windowslogs into the API tree.
2024-10-04 08:08:47,333 listener INFO Did not receive normal values. Unable to find meaningful check.

But after some time, it stops working (nothing in the logs), and we start receiving:

Code: Select all

UNKNOWN: Execution exceeded timeout threshold of 55s

After that:

Code: Select all

UNKNOWN: An error occurred connecting to API. (Connection error: '[Errno 111] Connection refused')

If I restart the service on the server, it starts working again.

It's nonsense!

Can someone please give me some light on this?

Thank you

bbahn · Post by **bbahn** » Mon Oct 07, 2024 11:22 am

Hello @ignacio.sanchez,

It seems that one of your checks is returning invalid values, which is likely causing the API and then by extent the NCPA listener to crash.
Which version of NCPA 3 did you try? There have been several improvements to the API to reduce crashes in recent versions that may help.
Do you have any plugins that might be returning bad values?

ignacio.sanchez · Post by **ignacio.sanchez** » Wed Oct 16, 2024 3:13 am

Hello @bbahn,

We currently use the latest version (3.1.1) and are not checking anything unusual.

server.png

Is there any way to see what could be causing this?

Code: Select all

2024-10-04 08:08:47,333 listener INFO Did not receive normal values. Unable to find meaningful check.

bbahn · Post by **bbahn** » Wed Oct 16, 2024 12:57 pm

I'll create an issue and investigate as soon as I can. Unfortunately with the way NCPA was written, there isn't a way to know for certain what caused the error listed aside from that there is a type error in the values returned from a check. I have made a quick MR that will make the logs give some better information when in debug mode.

Nagios Support Forum

NCPA

NCPA

Re: NCPA

Re: NCPA

Re: NCPA