Just upgraded to nagios 5.6 and it broke all python scripts

[acentek]

Below is a transcript of the upgrade 5.6 and our python errors.

This is a VM in ESXi.

Linux Distribution and version?
32 or 64bit?
VMware Image or Manual Install of XI?
Are there special configurations on your system, ie; is Gnome installed? Are you using a proxy? Are you using SSL?
**If you are encountering multiple issues that may not be related, start a thread for each issue

1) CentOS Linux release 7.6.1810 (Core)
2) 64bit
3) Manual Install of XI
4) SSL is being used for https


Chat on nagios.com
Conversation started on : Friday, April 26, 2019, at 19:58 (GMT+0)
[19:58] Nagios: Hi there! We're happy to help answer any questions you have about Nagios. Just ask!
[19:59] V1529074156483989: Just upgraded to nagios 5.6 and it broke all python scripts
[20:00] Brian has joined the conversation
[20:00] Brian: Hi, I'm Brian and I'm here to help. With whom am I speaking?
[20:00] V1529074156483989: gary
[20:01] Brian: Hi Gary, just for clarity, are you saying it broke the nagios plugins which are python scripts?
[20:02] V1529074156483989: We have a lot of services calling python scripts, that we have written
[20:02] Brian: As in custom plugins?
[20:04] V1529074156483989: Some may be from plugons but mainly we wrote the python scripts and created a nagios command to call the scripts and then have services using the nagios commands in the CCM
[20:05] V1529074156483989: File : https://tawk.link/587930f25bcc2b263bda7 ... /image.png

[20:05] Brian: I'll need a minute to check on something
[20:06] V1529074156483989: File : https://tawk.link/587930f25bcc2b263bda7 ... /image.png

[20:06] V1529074156483989: Posted command and service examples
[20:06] Brian: In what way are the scripts broken?
[20:07] Brian: AAlso, are you a current Nagios customer with support and maintenance?
[20:07] V1529074156483989: Traceback (most recent call last):
File "unifi_sw_lanesboro.py", line 55, in <module>
devNETGUARD.check_port_status(devNETGUARD.oid_alarm_status)
File "unifi_sw_lanesboro.py", line 18, in check_port_status
payload = alarm.poll_device_on_oid(self, oid)
File "/usr/local/alarm_scripts/unifi_v1/../alarm.py", line 12, in poll_device_on_oid
cmdGen = cmdgen.CommandGenerator()
File "/usr/lib/python2.7/site-packages/pysnmp/entity/rfc3413/oneliner/cmdgen.py", line 168, in __init__
self.snmpEngine = snmpEngine or SnmpEngine()
File "/usr/lib/python2.7/site-packages/pysnmp/entity/engine.py", line 92, in __init__
'__SNMP-FRAMEWORK-MIB', 'snmpEngineMaxMessageSize')
File "/usr/lib/python2.7/site-packages/pysnmp/smi/builder.py", line 400, in importSymbols
self.loadModules(modName, **userCtx)
File "/usr/lib/python2.7/site-packages/pysnmp/smi/builder.py", line 362, in loadModules
self.loadModule(modName, **userCtx)
File "/usr/lib/python2.7/site-packages/pysnmp/smi/builder.py", line 330, in loadModule
'MIB module \"%s\" load error: %s' % (modPath, traceback.format_exception(*sys.exc_info()))
pysnmp.smi.error.MibLoadError: MIB module "/usr/lib/python2.7/site-packages/pysnmp/smi/mibs/instances/__SNMP-FRAMEWORK-MIB.pyc" load error: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/pysnmp/smi/builder.py", line 325, in loadModule\n exec (modData, g)\n', ' File "/tmp/pip-build-E8J2rJ/pysnmp/pysnmp/smi/mibs/instances/__SNMP-FRAMEWORK-MIB.py", line 24, in <module>\n', ' File "/tmp/pip-build-E8J2rJ/pysnmp/pysnmp/smi/mibs/SNMP-FRAMEWORK-MIB.py", line 86, in clone\n', 'TypeError: clone() takes at most 2 arguments (4 given)\n']
[20:08] V1529074156483989: I am going to post the impact screen next :
[20:08] V1529074156483989: File : https://tawk.link/587930f25bcc2b263bda7 ... /image.png

[20:08] V1529074156483989: We rely heavily on scripting
[20:09] V1529074156483989: AAlso, are you a current Nagios customer with support and maintenance? Yes
[20:10] V1529074156483989: File : [REDACTED]

[20:11] Brian: You're best bet would be to open a ticket. The support people work closely with the developers and would be best equipped to resolve your issue.
[20:11] V1529074156483989: Ok, we will have to roll back 5.11 soon. So you have no suggestions?

[scottwilkerson]

Do you have /tmp in your python path?

Code: Select all

sudo su nagios -c 'python -c "import sys; print(sys.path)"'

[acentek]

No it doesn't look like I do.

# sudo su nagios -c 'python -c "import sys; print(sys.path)"'
['', '/usr/lib/python2.7/site-packages/ucsmsdk-0.9.3.1-py2.7.egg', '/usr/lib/python2.7/site-packages/imcsdk-0.9.3.0-py2.7.egg', '/usr/lib64/python27.zip', '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2', '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', '/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages', '/usr/lib/python2.7/site-packages']

[scottwilkerson]

I'm not clear why your python code would be pulling in the file
/tmp/pip-build-E8J2rJ/pysnmp/pysnmp/smi/mibs/SNMP-FRAMEWORK-MIB.py

There isn't anything in the upgrade that I know of that manipulates pysnmp at all

Is it possible to share unifi_sw_lanesboro.py ?

And
/usr/local/alarm_scripts/unifi_v1/../alarm.py ?

[acentek]

#####################################################################################################################
# Here is the unifi_lanesboro.py Script
#####################################################################################################################
#!/usr/bin/python
import sys
import os
# Used to map to alarm file
sys.path.append(os.path.join(os.path.dirname(os.path.realpath(__file__)), os.pardir))
from alarm import alarm

class netguard_alarm(alarm):
def __init__(self, host, read_community):
self.host = host
self.Read_Community = read_community
self.oid_alarm_name = '.1.3.6.1.4.1.8072.1.3.2.4.1.2.7.97.112.115.100.111.119.110.1'
self.oid_alarm_status = '.1.3.6.1.4.1.8072.1.3.2.3.1.4.7.97.112.115.100.111.119.110'
self.down_message = ''
self.is_blank = False

def check_port_status(self, oid):
payload = alarm.poll_device_on_oid(self, oid)
if isinstance(payload, list):
for name, val in payload:
# Service in bad state
#Jeremy this is you:
#print val
offset = 0
if int(val) == 0+offset:
self.down_message = 'Normal'
elif int(val) == 1+offset:
self.down_message = 'Warning'
else:
self.down_message = 'Critical'
else:
print(payload)
sys.exit(3)

def get_aux_comment(self):
payload = alarm.poll_device_on_oid(self, self.oid_alarm_name)
for name, val in payload:
return val

def output(self):
if(self.down_message == 'Normal'):
print("OK - No APs Down - Normal.")
sys.exit(0)
elif(self.down_message == 'Warning'):
print("Warning - %s (%s)." % (self.get_aux_comment(), self.down_message))
sys.exit(1)
elif(self.down_message == 'Critical'):
print("Critical - %s (%s)." % (self.get_aux_comment(), self.down_message))
sys.exit(2)
else:
print("UKNOWN - returned blank result.")
sys.exit(3)

if __name__ == '__main__':
if(len(sys.argv) == 3):
devNETGUARD = netguard_alarm(sys.argv[1], sys.argv[2])
devNETGUARD.check_port_status(devNETGUARD.oid_alarm_status)
devNETGUARD.output()
else:
print("Usage: unifi_v1.py <host> <SNMP_Community>")
sys.exit(3)

#####################################################################################################################
# Here is the alarm.py Script
#####################################################################################################################
from pysnmp.entity.rfc3413.oneliner import cmdgen

class alarm(object):

def poll_device_on_oid(self, oid):
cmdGen = cmdgen.CommandGenerator()

errorIndication, errorStatus, errorIndex, varBinds = cmdGen.getCmd(
cmdgen.CommunityData(self.Read_Community),
cmdgen.UdpTransportTarget((self.host, 161)),
oid
)

if errorIndication:
return errorIndication
else:
return varBinds

def convert_list_to_dict(self, alarm_list):
result = {}

result = dict(itertools.izip_longest(*[iter(alarm_list)] * 2, fillvalue=""))
return result

def print_down_message(self):
print(self.output)

[scottwilkerson]

I see this

Code: Select all

sys.path.append(os.path.join(os.path.dirname(os.path.realpath(__file__)), os.pardir))

Are you running this from the /tmp directory?

[acentek]

No, we are not running this from the /tmp directory. Here are both of the script locations:

find . -name unifi_lanesboro.py
./usr/local/alarm_scripts/unifi_v1/unifi_lanesboro.py

find . -name alarm.py
./root/scripts/alarm.py
./usr/local/alarm_scripts/alarm.py

Thanks!

[scottwilkerson]

Looking into this there are several people reporting that there is a problem with some versions of pysnmp (we do not distribute /upgrade this python package)
https://github.com/mushorg/conpot/issues/337

You may want to verify which version you are using currently, and then possibly attempt to upgrade the package to a known working version

Code: Select all

pip install pysnmp==4.3.9 -U

[acentek]

I updated to pysnmp version 4.3.9 (was on version 4.3.5).

Also did yum update, which updated a bunch of python items. We had done yum update previously but didn't help.

After that do you suggest attempting the Nagios upgrade again?

Also we'd try 5.6.1 version which rolled out last week.

[scottwilkerson]

I updated to pysnmp version 4.3.9 (was on version 4.3.5).

Also did yum update, which updated a bunch of python items. We had done yum update previously but didn't help.

After that do you suggest attempting the Nagios upgrade again?

Also we'd try 5.6.1 version which rolled out last week.

I'm not really sure what to suggest because this has to do with the specific plugin you are using and this plugin isn't part of the Nagios XI distribution (We did not create it)