Re: [Nagios-devel] More Threads Please?
Posted: Wed Aug 19, 2009 2:58 pm
Steven D. Morrey wrote:
> Hello Everyone,
>
> I've been profiling Nagios and found that at least on my installation with 30,000+ checks that 60% of the time is spent in reaping events and 3% is spent in saving status and another 1% is spent in writing out retention files.
> This means that less than 30% is actually spent executing checks, to compound the problem the latency is really, really high and it looks like it's due to high priority events blocking execution.
> It seems to me that all of these "high priority" events are high priority because they need to run right now, but I don't see why they need to block execution of everything else just to perform housekeeping tasks.
>
> Assuming I were to put a mutex on the relevant objects, can anyone foresee a problem with running these high priority events in their own thread or even threads?
>
> I have 2 designs in mind.
> One would be a simple refactor that calls handle_timed_event with pthread create, on the high priority events.
> This would create a new thread for each high priority event, but would be literally be a change to only a couple lines of code.
>
> Another design would call for a breaking the main event loop into a high priority thread and a low priority thread.
> In this case we would only have 2 threads, we loose a bit of the flexibility (and danger) and of having per event threads for high priority events, we also loose most of the overhead of thread creation.
>
> I'm not sure high priority events are frequent enough that thread creation overhead is significant.
>
> Anyways, as always your thoughts are very much appreciated.
>
I'd be keen to see the difference it makes running high priority events
in separate threads, I don't know nagios well enough to spot anything
that might break as a result, but it sounds interesting to try out.
if option one is easier to implement, I wouldn't mind trying to help
with this, or testing. I don't have much of a test environment at the
moment though.
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]
> Hello Everyone,
>
> I've been profiling Nagios and found that at least on my installation with 30,000+ checks that 60% of the time is spent in reaping events and 3% is spent in saving status and another 1% is spent in writing out retention files.
> This means that less than 30% is actually spent executing checks, to compound the problem the latency is really, really high and it looks like it's due to high priority events blocking execution.
> It seems to me that all of these "high priority" events are high priority because they need to run right now, but I don't see why they need to block execution of everything else just to perform housekeeping tasks.
>
> Assuming I were to put a mutex on the relevant objects, can anyone foresee a problem with running these high priority events in their own thread or even threads?
>
> I have 2 designs in mind.
> One would be a simple refactor that calls handle_timed_event with pthread create, on the high priority events.
> This would create a new thread for each high priority event, but would be literally be a change to only a couple lines of code.
>
> Another design would call for a breaking the main event loop into a high priority thread and a low priority thread.
> In this case we would only have 2 threads, we loose a bit of the flexibility (and danger) and of having per event threads for high priority events, we also loose most of the overhead of thread creation.
>
> I'm not sure high priority events are frequent enough that thread creation overhead is significant.
>
> Anyways, as always your thoughts are very much appreciated.
>
I'd be keen to see the difference it makes running high priority events
in separate threads, I don't know nagios well enough to spot anything
that might break as a result, but it sounds interesting to try out.
if option one is easier to implement, I wouldn't mind trying to help
with this, or testing. I don't have much of a test environment at the
moment though.
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]