Windows 2012 SNMP Memory Leak?

[jriker1]

I have a Windows 2012 R2 Essentials server that I am monitoring with Nagios. I notice high SNMP service utilization to the point I start running low on memory. Over time the amount of memory this 32-bit service uses keeps increasing until Nagios complains and I have to restart the system. I'm doing a decent amount of SNMP calls for various things, using check_intel_raid, a perl script.

With the perl script I check:

[*] Virtual Disk state
[*] External enclosure Power Supply
[*] External enclosure fan status
[*] External enclosure temp status
[*] External enclosure any alarms

I also have Dell Command Monitor 9.1 installed on this server and monitoring thru SNMP:

[*] Chassis status
[*] Chassis Intrusion Status
[*] Cooling device status
[*] Overall System State
[*] Voltage Status
[*] Memory Status
[*] Processor Status.

Also have an SNMP_TRAP service setup but don't know if it does anything ever. See log entries about trapping ping failures which is odd as SNMP couldn't send those messages if the system is down.

Anywho, any idea what's causing the memory, I'll call it leaks, in my system? My guess is dell command monitor but don't want to be presumptuous until I can find some way to know what's causing this.

Thanks.

JR

EDIT: Note I have a 64-bit SNMP Service running using minimal memory and a 32-bit SNMP Service using loads of memory. The 64-bit one has an expandable branch showing it's running the SNMP Service, the 32-bit one doesn't expand but does point to a snmp.exe file in the C:\Windows\SysWOW64 folder and also calls itself the "SNMP Service"

EDIT2: I can say if I kill the 32-bit version of SNMP memory clears up but monitoring starts failing. 64-bit is still running and Command Monitor is 64-bit though looks like it's using a 32-bit SNMP version. I can say on my Windows 7 64-bit computer with the same setup, both snmp executables exist, but only the 64-bit one seems like it's showing in task manager.

[ssax]

I haven't heard of a memory leak in 2012 R2 essential's SNMP service.

The only thing that I can think of is that the perl script isn't exiting properly when it runs, if you open up task manager to you see an abundance of processes?

What about on the XI server?

Code: Select all

ps aux |grep check_intel_raid

Are you seeing anything in the event logs that may be related?

Is there any virus/spyware/intrusion prevention that may be interacting negatively with the checks? Make sure to check in those logs as well.


Thank you

[jriker1]

Sorry for the delay, been researching. From what I can see, It's probably the now called Avago Raid Web Console. There is talks about a DLL that has a memory leak and that's what some of the processes are accessing all the time. In theory the bug was supposed to be fixed in later releases which I'm on, but possibly not. Since the web console is a 32-bit process, and this memory leak is related to the 32-bit version of SNMP, thinking that may be a match.

JR

[tgriep]

Have you ran the Windows updates on that server?
Try updating the server and the application if possible and see if that helps.

[jriker1]

Yeah been there done that as they say. No more updates. I guess the next logical thing to do is stop the SNMP checks for my intel raid and see if it goes away. Making the assumption that if no calls are being made the memory leak will not grow????

Thanks.

JR

[tgriep]

Yes, stopping the checks should help out in debugging this to see if the software leaks on it own.

[jriker1]

Think it's still leaking however I assume a program running on Nagios that makes a call to the server requesting the results of a particular MIB indentifier can't itself be the memory leak because the calling app is on the nagios server, not the server being monitored with the leak? Is that accurate?

Thanks.

JR

[tgriep]

Yes, that sound right.

[jriker1]

After a lot of troubleshooting it comes down to LSI or Avago's storlib.dll file. I have removed all other SNMP based tools and this is the only one when I remove it the memory leak goes away. Plus the only one that activates the 32-bit SNMP environment on the server. All the others run under 64-bit and the 32-bit version doesn't show in the task list. When the LSI instance is there the 32-bit version shows and memory starts going from 3k to what is now 1.3GB and will eventually go into the 2GB range. Of course now that I know this not sure there is a fix.

JR

[tgriep]

Thanks for posting back your findings. Only thing left for you to do is see if you can get a 64bit version from the manufacturer.