Python plugin using winexe hangs when run as service

[yuletak]

Hi, I'm using Python to create a plugin to execute a Win7 command remotely with winexe. The plugin runs fine manually on the command line with the complete path to the file and as user nagios, though I AM getting the following error messages after getting output back from the remote Win7 server. This is the debug output from winexe:

[output from "dir c:\" snipped]
09/18/2013 01:20 PM 209,814,312 utilities.zip
01/07/2011 01:54 PM <DIR> utils
02/20/2014 05:42 PM <DIR> Windows
5 File(s) 212,003,517 bytes
12 Dir(s) 10,333,868,032 bytes free
ERROR: smb_raw_read_recv - NT_STATUS_PIPE_DISCONNECTED
ERROR: smb_raw_read_recv - NT_STATUS_PIPE_DISCONNECTED
ERROR: smb_raw_read_recv - NT_STATUS_PIPE_DISCONNECTED
ERROR: smb_raw_read_recv - NT_STATUS_PIPE_DISCONNECTED
ERROR: on_ctrl_pipe_error - NT_STATUS_PIPE_DISCONNECTED
OpenService - NT_STATUS_OK
StopService - NT_STATUS_OK
DeleteService - NT_STATUS_OK
CloseServiceHandle - NT_STATUS_OK
CloseSCMHandle - NT_STATUS_OK
Delete winexesvc.exe - NT_STATUS_OK
Closing ADMIN$ - NT_STATUS_OK

When scheduling a check of the service from Nagios, I get this:

This program may be freely redistributed under the terms of the GNU GPLv3
svc_UploadService: Installing 64bit winexesvc.exe
IN: async_open(\pipe\ahexec, 2)
IN: async_open_recv
CTRL: Sending command: get version
CTRL: Sending command: run cmd.exe /C dir c:\
CTRL: Recieved command: std_io_err 03100001
IN: async_open(\pipe\ahexec_stdin03100001, 2)
IN: async_open(\pipe\ahexec_stdout03100001, 2)
IN: async_open(\pipe\ahexec_stderr03100001, 2)
IN: async_open_recv
tevent: EPOLL_CTL_ADD failed (Operation not permitted) - falling back to select()

I'm using using winexe Version 4.0.0alpha11-GIT-UNKNOWN. Python is 2.6.6. I've tried just creating a sh script, but it hangs also. Has anyone run into this problem before?

[sreinhardt]

Could you post a link to the plugins, and a copy of your service and command definitions please?

[yuletak]

Here's my python code. Please ignore the print out for "Performance Data"

Code: Select all

#!/bin/env /usr/bin/python

import sys, getopt, shlex, subprocess

LOG='/usr/lib64/nagios/plugins/server.log'
cmd = '/usr/bin/winexe -d 1 --interactive=0 --uninstall -U <USER removed>%<PASSWD removed> //<server> \'cmd.exe /C dir c:\\\''

def main():
    oput = err = None
    log = open(LOG, 'w')
    p = subprocess.Popen(cmd, shell=True, stdin=None, stdout=log, stderr=log)
    log.close() 
    print 'status'
    print '|cmd:{2} *** oput:{0} err:{1}'.format(oput,err,cmd)
    sys.exit(0)

if __name__ == "__main__":
    main()    

Here's the command definition. For right now, the script isn't reading the input arguments since that's not my problem.

# 'check_ecxtwqlkp.py' command definition
define command{
command_name check_ecxtwqlkup.py
command_line $USER1$/check_ecxtwqlkup.py $USER5$ $USER6$ $USER7$ $USER8$ $USER9$ $USER10$ $ARG1$ $ARG2$
}

Service definition:

define service{
use generic-service
host_name prosrv2
service_description TEST PDF Q_Lookup TW01
check_command check_ecxtwqlkup.py!tw1!1
}

Thanks.

[slansing]

Did you follow a specific guide for this that you could link us to? We'd like to reproduce this if possible.

[yuletak]

I didn't have a guide to follow. I was just trying to automate our recovery process for known problems. We have existing servers, service groups, etc. already set up. Is there something specific that I can provide that might help? I kept an eye on the Nagios log as well, but nothing helpful showed up.

In the python code, I originally had stdout=PIPE and stderr=stdout, so all messages were going to one place. Currently, I'm recompiling winexe to see if that helps, but I'm very curious to find out why winexe hangs when run from Nagios but is fine when run from the CLI.

Thanks.

[yuletak]

Sorry, forgot the version of Nagios we're using.

-bash-4.1$ ./nagios --version

Nagios Core 3.5.0
Copyright (c) 2009-2011 Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 03-15-2013
License: GPL

[tmcdonald]

Just so we can confirm and have it spelled out for us, can you do the following?

- Run the command as root from the command line with all arguments you normally would use
- Switch to the nagios user and run it from the command line the same way
- run ls -l on the python plugin and the winexe file

Post all the commands and their output here, ideally just the whole terminal session copy+pasted. Be sure to wrap it all in [code][/code] tags.

[yuletak]

RESOLVED

The problem was stdin for winexe. I didn't think that was the problem since I had set stdin=None when using subprocess (which in itself has another hanging problem w/ too much input). Here is the link that I referenced:

http://sourceforge.net/p/winexe/bugs/31/?limit=25

First:

"This bug report is about poll/select bug, indicated by the following winexe message:
tevent: EPOLL_CTL_ADD failed (Operation not permitted) - falling back to select()
Your issue is totaly different, powershell is a console application and do not works correctly with simple stdin/out/error pipes provided by winexe."

Which led me to refer to an earlier post:

"Try running "cat </dev/null | winexe..." instead of "winexe..."."

As I said earlier, stdin was already set to None, so I didn't think to try this. But after several failed attempts to build the current version of winexe and a whole lot of mucking around, I decided to give this a try, and sure enough, it worked. If I can find an explanation and remember to do so, I will come back to post.

Thanks.

[sreinhardt]

Cool, thanks for letting us know!