Powershell.exe processes not closing.

[WillemDH]

Hey,

I'm working on making my check_ms_netstat plugin better, as it seems when connection times out, the powershell.exe process sometimes don't close properly. This has caused CPU related issues on one of my Exchange servers, as it had several powershell.exe and conhost.exe processes, which seemed to hang.
As this Exchange server had 5k+ connections, the plugin timed out (after 120 seconds).
So I solved this by putting get-process in an array before going through all the Netstat connections and then comparing each pid with the array instead of doing a get-process every time. Besides the above, the plugin will only do a get-process if the -P parameter is provided. This shortens the execution of the plugin a lot and it now takes 15 seconds with about 4% cpu on one of my test servers, which has 9k+ connections (SAP).
Did anyone here ran into problems with powershell.exe processes not closing properly? I'm using NSClient++ 0.4.1.101 mostly.
Also made a post about it on the NSclient forum (https://www.nsclient.org/forums/topic/i ... n-timeout/)

Grtz

Willem

[rajasegar]

I had similar problem in one of my server. Could not figure out what was wrong since when I am monitoring it is always ok.

1) Checked to ensure all the batch files and powershell scripts had an exit statement.
2) Setup a monitor for powershell process. More than 5 processes active for 30 minutes to alert the sysadmin.

Never had the problem again which means the root cause could be something else.

[sreinhardt]

I can't say I use powershell with nsclient enough to weigh in on it's effects. However commenting on process exit control, I certainly can! Personally, I have moved to writing scripts that act as a master for a secondary script executed in a runspace. Runspaces allow me to thread powershell in a very fast effective way. I can also apply timeouts at the runspace level and master thread level. So that if the runspace times out, it will report as unknown via the master thread, however if the connection to nagios is closed or command has taken too long and not timed out properly, the master thread will close it's worker runspace and cleanup what needs to before exiting its self. This works really well, as the master thread just relays information from worker to nagios or your agent, and allows the scripts themselves to exit properly in all cases.

I'm actually working on extending this for people just like you willem, that do lots of cool powershell stuff, but want it faster and more cleanly! It's no where near production, and I won't really discuss it too much more yet, but I'll definitely reach out when I deem it testable!

[WillemDH]

Spenser,

Looking forward to it Spenser, but my problem at the moment is that we are getting very integrated with NSClient (mainly for its realtime eventlog capability). I fear your solution will be a part of NCPA? Or will it be usable with NSclient? Have there been tests with running two agents one a server?

Grtz

Willem

[sreinhardt]

I am not making it agent specific. It could be run standalone or with an agent. Nsclient may have to add an interface to communicate with me, but otherwise it should be more than happy to work with both.

I've definitely run both on a single system, its not a lot of overhead, but does seem redundant. I totally get already having nsclient installed and not wanting to switch, I'd much rather integrate with everything than force you to change. Some features will likely be better integrated with ncpa, but otherwise nothing should force you into ncpa as I plan it.

[WillemDH]

Thanks Spenser. Let me know if you would like someone to test it on Nsclient.

Thread can be closed.

Willem