Powershell exiting OK even with errors?

[redskins1234]

Reposting this in the correct fourm

Implementing a new script on some servers with check_nrpe!script.ps1, and made a mistake while copying it over to everyone. I had a misplaced " somewhere which caused the script to break and exit with an error.

But, I noticed that this was still returning an OK, even though powershell obviously did not exit with a exit code of 0? if a script exits with errors, why is it returning OK?

nsclient.ini

Code: Select all

;Powershell wrapping
ps1=cmd /c echo scripts\\%SCRIPT% %ARGS%; exit($lastexitcode) | powershell.exe -command -

Example: I forced an error by turning execution policy off. It was normally a different error. I am just concerned that it is returning ok, when clearly, the script did not run ok.

Code: Select all

Current Status:   
  OK  
(for 0d 0h 41m 28s)
Status Information:   File C:Program FilesNSClient++scriptscheck_windows_updates.ps1 cannot be lo
aded because the execution of scripts is disabled on this system. Please see "g
et-help about_signing" for more details.
At line:1 char:34
+ scripts\check_windows_updates.ps1 <<<< ; exit($lastexitcode)
+ CategoryInfo : NotSpecified: (:) [], PSSecurityException
+ FullyQualifiedErrorId : RuntimeException

[benjaminsmith]

Hi @redskins1234,

The Powershell wrapping looks correct. Can you PM us the powershell script so we can test it here? I think what's happening is the powershell is returning the exit code of powershell itself and not the code of your script. See this.

[redskins1234]

Apologies for the very late response. Read your link and the error there was the wrapping, which as you said looks fine here..

The script I'm running is actually a slightly modified version of this one. None of the modifications we made change the exit code codes.

https://exchange.nagios.org/directory/P ... ll/details

Our modified version hosted here to avoid clutter: https://pastebin.com/f4khwV5s

I've examined it now myself and theres nothing fancy going on in terms of an exit statement. Just exits with anywhere from 0-3

Will turn on notifications so I can respond quicker this time Thanks for the help

[redskins1234]

I actually wanted to test something this morning, and completely deleted the script off a server. I then forced the check, and it STILL came back okay

Code: Select all

Current Status:	
  OK  
 (for 4d 17h 0m 30s)
Status Information:	scripts\check_windows_updates.ps1 : The module 'scripts' could not be loaded.
For more information, run 'Import-Module scripts'.
At line:1 char:1
+ scripts\\check_windows_updates.ps1 ; exit($lastexitcode)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (scripts\\check_windows_updates.
ps1:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CouldNotAutoLoadModule 

So I think this means it's definitely not the script itself? That points to the wrapping but as we saw it seems okay...

EDIT:

I've done some looking into $lastexitcode and I wonder if thats not the issue?

This post says that it only returns the exit code of the last executable that ran, i.e., nothing runs, it'll be 0
https://social.technet.microsoft.com/Fo ... forum=ITCG

Tested using the wrapper and it does look like the issue could be from this..

With $lastexitcode

Code: Select all

C:\Program Files\NSClient++>echo scripts\\check_windows_updates.ps1; exit ($lastexitcode) | powershell
Windows PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.

PS C:\Program Files\NSClient++> scripts\\check_windows_updates.ps1; exit ($lastexitcode)
Updates: 0 Critical, 2 Optional, 1 'Fresh' Updates
| critical=0, optional=2, hidden=0, new=1

C:\Program Files\NSClient++>echo Exit Code is %errorlevel%
Exit Code is 0

C:\Program Files\NSClient++>echo "I am moving the file"
"I am moving the file"

C:\Program Files\NSClient++>echo scripts\\check_windows_updates.ps1; exit ($lastexitcode) | powershell
Windows PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.

PS C:\Program Files\NSClient++> scripts\\check_windows_updates.ps1; exit ($lastexitcode)
scripts\\check_windows_updates.ps1 : The module 'scripts' could not be loaded. For more information, run
'Import-Module scripts'.
At line:1 char:1
+ scripts\\check_windows_updates.ps1; exit ($lastexitcode)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (scripts\\check_windows_updates.ps1:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CouldNotAutoLoadModule


C:\Program Files\NSClient++>echo Exit Code is %errorlevel%
Exit Code is 0

Without $lastexitcode

Code: Select all

C:\Program Files\NSClient++>echo scripts\\check_windows_updates.ps1 | powershell
Windows PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.

PS C:\Program Files\NSClient++> scripts\\check_windows_updates.ps1
scripts\\check_windows_updates.ps1 : The module 'scripts' could not be loaded. For more information, run
'Import-Module scripts'.
At line:1 char:1
+ scripts\\check_windows_updates.ps1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (scripts\\check_windows_updates.ps1:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CouldNotAutoLoadModule

PS C:\Program Files\NSClient++>
C:\Program Files\NSClient++>echo Exit Code is %errorlevel%
Exit Code is 1

C:\Program Files\NSClient++>echo "moving file back"
"moving file back"

C:\Program Files\NSClient++>echo scripts\\check_windows_updates.ps1 | powershell
Windows PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.

PS C:\Program Files\NSClient++> scripts\\check_windows_updates.ps1
Updates: 0 Critical, 2 Optional, 1 'Fresh' Updates
| critical=0, optional=2, hidden=0, new=1
PS C:\Program Files\NSClient++>
C:\Program Files\NSClient++>echo Exit Code is %errorlevel%
Exit Code is 0

So something funky seems to be going on with the wrapping? $lastexitcode doesn't seem necessary, but the documentation/your original link seems to say that it is to capture the exit code..Any comments/ideas?

[scottwilkerson]

Looking this over and doing some testing, it does appear that powershell exits 0 if the the file doesn't exist.

Below is a wrapper I would propose to catch such a situation

Code: Select all

ps1=cmd /c echo if (test-path scripts\\%SCRIPT%){scripts\\%SCRIPT% %ARGS%}else{Write-Host "Plugin %SCRIPT% NOT FOUND!";Exit 3;}; exit($lastexitcode) | powershell.exe -command -

[redskins1234]

That looks good to me, but quickly testing that, this wrapper isn't a catch all for a script not running at all. I.e. if the script has errors like the original post.

I understand at a certain level its the responsibility of us to make sure these scripts run and are actually there, but on wide scale deployments I've sometimes even had bad network transfers ruin scripts before (i.e. copying a script to say 100 servers, one of them got it mangled, I'll never know without manually verifying). So ideally the wrapper should be have some kind of if (! script_ran) exit 3 else echo scripts\\%SCRIPT% %ARGS%; exit ($lastexitcode) by default?

Sorry if that doesn't make sense or is too demanding, a bit new to all this.

Edit: And if you dont mind further expanding on what ; exit ($lastexitcode) accomplishes, I'm a bit confused why it's needed.. I get return codes when I expect them without it, and going by that original link (https://forums.nsclient.org/t/return-co ... olved/2504), the $lastexit code on the wrapping is necessary because of running a sub-instance of powershell..but, from what I can see, the wrapping without last exit code, i.e.

Code: Select all

;Powershell wrapping
ps1=cmd /c echo scripts\\%SCRIPT% %ARGS%| powershell.exe -command -

is just running powershell IN that cmd terminal. So the lastexitcode is unnecessary and actually causing this problem?? Does that sound right?

[scottwilkerson]

Well, that would be one for a powershell guru, I honestly don't know any way of making powershell do that other than somehow parsing the standard error.

Generally I would say to add some exceptions to the script to catch errors, but that wouldn't help the scenario where the script got mangled in transit

[redskins1234]

Maybe a try/catch is the best case here? This looks like the best of both worlds, we keep around $lastexitcode for when it's needed, but if the script encounters any problem starting (i.e. missing, mangled script etc), the catch will run and exit with a 3.

Tested with my script and it still captures any exit code I put in there.

Code: Select all

ps1=cmd /c echo Try {scripts\\%SCRIPT% %ARGS%; exit ($lastexitcode)} Catch {echo "Error Running Script"; exit 3} | powershell.exe -command -

I think personally this is the solution to our problem. Hopefully this can help others if they've had this issue. Respectfully, I may submit a PR to the nsclient docs repo on github to suggest this as the wrapping example there. For such a small change it can save a lot of headache.

Either way, thank you for the help and guidance!

[scottwilkerson]

Maybe a try/catch is the best case here? This looks like the best of both worlds, we keep around $lastexitcode for when it's needed, but if the script encounters any problem starting (i.e. missing, mangled script etc), the catch will run and exit with a 3.

Tested with my script and it still captures any exit code I put in there.

Code: Select all

ps1=cmd /c echo Try {scripts\\%SCRIPT% %ARGS%; exit ($lastexitcode)} Catch {echo "Error Running Script"; exit 3} | powershell.exe -command -

I think personally this is the solution to our problem. Hopefully this can help others if they've had this issue. Respectfully, I may submit a PR to the nsclient docs repo on github to suggest this as the wrapping example there. For such a small change it can save a lot of headache.

Either way, thank you for the help and guidance!

sounds great!