SNMPTT Problem

[arnab.roy]

Hi Guys,

Sorry that I am having to raise this again...

But we are having serious issues with snmptrap handling. The snmptrapd service hangs and we see something like this

KARMA[~]> service snmptrapd restart
Stopping snmptrapd: [ OK ]
Starting snmptrapd: [FAILED]
KARMA[~]> ps aux | grep snmpt
root 15880 0.0 0.0 108196 1240 ? S Sep25 0:00 sh -c (/usr/local/sbin/snmptt) < "/var/run/net-snmp/snmp-tmp-NpaPyb"
root 15881 0.0 0.2 178900 28348 ? S Sep25 0:01 /usr/bin/perl /usr/local/sbin/snmptt
root 15897 0.0 0.0 121824 3132 ? S Sep25 0:00 python /usr/local/bin/snmptraphandling.py 192.168.163.40 SNMP Traps Normal 1348578255 wlsxTrapTime (OCTETSTR):2012-9-25,14:11:35.0,+1:0 wlsxTrapTargetAPBSSID.0 (OCTETSTR):0:26:cb:10:27:44 wlsxTrapTargetAPSSID.0 (OCTETSTR):Imperial-WPA wlsxTrapAPMacAddress.0 (OCTETSTR):0:24:6c:c1:78:9c wlsxTrapAPRadioNumber.0 (INTEGER32):1 wlsxTrapAPLocation.0 (OCTETSTR):FW.4.11.P.G wlsxTrapAPChannel.0 (INTEGER32):11 ??? 2012-9-25,14:11:35.0,+1:0 0:26:cb:10:27:44 Imperial-WPA 0:24:6c:c1:78:9c 1 FW.4.11.P.G 11
root 25712 0.0 0.0 103248 852 pts/1 S+ 12:54 0:00 grep snmpt

So it looks like the snmptraphandling.py script gets stuck and cause the snmptrapd to get stuck... Can someone please help us as we failed to detect significant network failover because the process was stuck

[scottwilkerson]

Sorry I didn't update the other thread.

We have a developer reworking the installer for this addon, however this appears to be unrelated as this appears to be a hung process from almost 2 weeks ago.

I would kill these off so you can restart the service.

Were you seeing any errors in syslog? perhaps segfaults?

[arnab.roy]

Hi Scott,

This is what is happening I have killed off and restarted and is working now. But very soon it will get stuck and hung again. As a work around can we detect this situation(may be cron) and restart or kill services as required ?

Many Thanks
Arnab

[jsmurphy]

We ran into this as well regularly back when we were running in standalone mode... there was a time when I knew exactly what caused this, but instead you will have to deal with the version where my memory has recycled this information.

It had something to do with the length of time the snmptrapd service had a file handle open (or maybe it was number of open file handlers?) while trying to pass it off to the snmptt handler, it's more or less related to the issue in the previous post I made. There was a more knowledgeable guy than me on a forum somewhere who explained (from the vague depths of my memory) it was an oversight in the snmptrapd's preservation coding to stop snmptrapd from killing your box.