Re: [Nagios-devel] [Fwd: Memory leak in Nagios head]
Posted: Tue Nov 30, 2004 3:43 pm
Ethan Galstad wrote:
> Looks like the real culprit is the setenv() system command.
> Apparently, memory "freed" by the unsetenv() command is not actually
> freed, causing the leak. I added this feature within the last few
> weeks to allow macro variables to be avaiable as env vars in all
> commands. Too bad this causes problems, as it would have been a nice
> feature.
>
It could possibly be worked around by passing envp[] from execle, or
doing some pointer magic with **environ, but I've never tried that, and
I'm not sure it would be worth the changes (not to mention it might
break something else in the process).
> For the time being, I have simply made the
> set_macro_environment_var() function in utils.c return without doing
> anything. This seems to fix the problem,
I'll get to testing right away.
> although I am sure there are a few small leaks elsewhere.
>
I've found a few here and there, so I've started working on a memory
tracker (it's planned to grow into a garbage collector someday). It's
very crude right now but should catch most of the in-house leaks. It's
currently conditional to MEMLEAK_DEBUG being defined and isn't quite
ready for testing just yet.
>
> On 29 Nov 2004 at 21:34, Andreas Ericsson wrote:
>
>
>>Matthew Kent wrote:
>>
>>>Forwarding this on in case anyone else has seen this behaviour and
>>>has some suggestions. I'll give it a run through valgrind and see if
>>>I can spot anything this evening.
>>>
>>
>>Thanks, Matt.
>>
>>A small update;
>>
>>After having run the daemon about 10 hours at a test system, memory
>>consumption has escalated from roughly 1MB to around 24MB. Not very
>>nice figures. It seems that sending a HUP makes memory consumption
>>make a small jump (usually around 20K).
>>
>>This small script should crash Nagios after a while. It might have to
>>be run more than once. I haven't tried it very much. It works
>>regardless of whether or not the eventbroker is compiled in.
>>
>>-----
>>#!/bin/sh
>>
>>pid=`cat /usr/local/nagios/var/nagios.lock`
>>for i in `seq 1 100`; do
>> kill -HUP $pid
>> sleep 1
>> ps -up $pid || break
>>done
>>
>>[ $i -ne 100 ] && echo "Nagios successfully crashed"
>>
>>-----
>>
>>With --enable-DEBUGALL, this output is shown by tailing the last 20 or
>>so lines of the output.
>>
>>find_host() start (repeats about 40 or so times)
>>write_to_all_logs() start
>>write_to_syslog() start
>>write_to_log() start
>>write_to_all_logs() end
>>Caught SIGSEGV, shutting down...
>>close_command_file() start
>>close_command_file() end
>>write_svc_message() start (repeats 4 times)
>>
>>Another weird thing is that the SIGSEGV crash causes Nagios to remove
>>the lockfile, but not the objects.cache-file, which causes the
>>webinterface to happily chug on as if nothing's happened. That should
>>be fairly easily fixed, but I'd rather get to the bottom of the memory
>>leak.
>>
>>
>>
>>>--------------------------------------------------------------------
>>>----
>>>
>>>Subject:
>>>Memory leak in Nagios head
>>>From:
>>>Andreas Ericsson
>>>Date:
>>>Mon, 29 Nov 2004 12:47:27 +0100
>>>To:
>>>[email protected]
>>>
>>>To:
>>>[email protected]
>>>CC:
>>>[email protected]
>>>
>>>
>>>Ahoy Ethan.
>>>
>>>There's a memory leak in Nagios HEAD, causing it to crash when (it
>>>seems) enough checks has been executed. I would have emailed the
>>>nagios-devel list but I believe I'm still banned from it pending a
>>>solution to the sourceforge list server fuckup? Could one of you
>>>forward this to that list, please?
>>>
>>>It's compiled without embedded perl, and without perl caching, like
>>>so; ./configure --prefix=/opt/monitor
>>>
>>>I'll start digging today, so I might have a patch ready for you
>>>soon.
>>>
>>>I Cc'ed Matthew Kent on this as well, seeing as he's been very
>>>helpful with the finding and fixing earlier.
>>>
>>
>>--
>>Andreas Ericsson [email protected]
>>OP5 AB www.op5.se
>>Lead Developer
>>
>>
>>-------------------------------------------------------
>>SF email is sponsored by - The IT Product Guide
>>Read hones
...[email truncated]...
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]
> Looks like the real culprit is the setenv() system command.
> Apparently, memory "freed" by the unsetenv() command is not actually
> freed, causing the leak. I added this feature within the last few
> weeks to allow macro variables to be avaiable as env vars in all
> commands. Too bad this causes problems, as it would have been a nice
> feature.
>
It could possibly be worked around by passing envp[] from execle, or
doing some pointer magic with **environ, but I've never tried that, and
I'm not sure it would be worth the changes (not to mention it might
break something else in the process).
> For the time being, I have simply made the
> set_macro_environment_var() function in utils.c return without doing
> anything. This seems to fix the problem,
I'll get to testing right away.
> although I am sure there are a few small leaks elsewhere.
>
I've found a few here and there, so I've started working on a memory
tracker (it's planned to grow into a garbage collector someday). It's
very crude right now but should catch most of the in-house leaks. It's
currently conditional to MEMLEAK_DEBUG being defined and isn't quite
ready for testing just yet.
>
> On 29 Nov 2004 at 21:34, Andreas Ericsson wrote:
>
>
>>Matthew Kent wrote:
>>
>>>Forwarding this on in case anyone else has seen this behaviour and
>>>has some suggestions. I'll give it a run through valgrind and see if
>>>I can spot anything this evening.
>>>
>>
>>Thanks, Matt.
>>
>>A small update;
>>
>>After having run the daemon about 10 hours at a test system, memory
>>consumption has escalated from roughly 1MB to around 24MB. Not very
>>nice figures. It seems that sending a HUP makes memory consumption
>>make a small jump (usually around 20K).
>>
>>This small script should crash Nagios after a while. It might have to
>>be run more than once. I haven't tried it very much. It works
>>regardless of whether or not the eventbroker is compiled in.
>>
>>-----
>>#!/bin/sh
>>
>>pid=`cat /usr/local/nagios/var/nagios.lock`
>>for i in `seq 1 100`; do
>> kill -HUP $pid
>> sleep 1
>> ps -up $pid || break
>>done
>>
>>[ $i -ne 100 ] && echo "Nagios successfully crashed"
>>
>>-----
>>
>>With --enable-DEBUGALL, this output is shown by tailing the last 20 or
>>so lines of the output.
>>
>>find_host() start (repeats about 40 or so times)
>>write_to_all_logs() start
>>write_to_syslog() start
>>write_to_log() start
>>write_to_all_logs() end
>>Caught SIGSEGV, shutting down...
>>close_command_file() start
>>close_command_file() end
>>write_svc_message() start (repeats 4 times)
>>
>>Another weird thing is that the SIGSEGV crash causes Nagios to remove
>>the lockfile, but not the objects.cache-file, which causes the
>>webinterface to happily chug on as if nothing's happened. That should
>>be fairly easily fixed, but I'd rather get to the bottom of the memory
>>leak.
>>
>>
>>
>>>--------------------------------------------------------------------
>>>----
>>>
>>>Subject:
>>>Memory leak in Nagios head
>>>From:
>>>Andreas Ericsson
>>>Date:
>>>Mon, 29 Nov 2004 12:47:27 +0100
>>>To:
>>>[email protected]
>>>
>>>To:
>>>[email protected]
>>>CC:
>>>[email protected]
>>>
>>>
>>>Ahoy Ethan.
>>>
>>>There's a memory leak in Nagios HEAD, causing it to crash when (it
>>>seems) enough checks has been executed. I would have emailed the
>>>nagios-devel list but I believe I'm still banned from it pending a
>>>solution to the sourceforge list server fuckup? Could one of you
>>>forward this to that list, please?
>>>
>>>It's compiled without embedded perl, and without perl caching, like
>>>so; ./configure --prefix=/opt/monitor
>>>
>>>I'll start digging today, so I might have a patch ready for you
>>>soon.
>>>
>>>I Cc'ed Matthew Kent on this as well, seeing as he's been very
>>>helpful with the finding and fixing earlier.
>>>
>>
>>--
>>Andreas Ericsson [email protected]
>>OP5 AB www.op5.se
>>Lead Developer
>>
>>
>>-------------------------------------------------------
>>SF email is sponsored by - The IT Product Guide
>>Read hones
...[email truncated]...
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]