Re: [Nagios-devel] A NDOUtils replacement : CentreonBroker
Posted: Wed Jul 01, 2009 9:17 pm
Matthieu Kermagoret wrote:
> On Wed, Jul 1, 2009 at 5:17 PM, Andreas Ericsson wrote:
>> Matthieu Kermagoret wrote:
>> This means there are four major contenders for the spot of being the
>> default database broker for Nagios; NDOUtils, Merlin, IDOUtils and
>> CentreonBroker. This is great, as it means the best technical solution
>> will win. At the same time, it's not so great since it means at least
>> four developers are working toward similar goals, but not working
>> together towards that goal.
>>
>
> I agree with you that there will certainly be some efforts wasted but
> it seems to me that each broker has its own peculariarity (replication
> for Merlin, performance for CentreonBroker, ...).
>
Well, Merlin handles 80k hosts and 500k services just fine, no matter
which host the checks are being executed on, so performance is hardly
something only CentreonBroker does well
>>> - multiple input ports
>>> - multiple database output
>>> - configurable logs
>> These sound neat though, although I fail to see how it can be useful to
>> have a single daemon output to more than one database since all databases
>> worth their salt already have replicating capabilities.
>>
>>> - TLS connection
>> This doesn't parse, or TLS doesn't mean thread-local storage to you.
>>
>
> I thought that now TLS is more known as the new name of SSL than of
> the name of a low-level feature.
>
I'm the stupid one here. Sorry for the noise. This sounds fairly
interesting though. I'll have to see how you've implemented this and
see if I can scrounge it for Merlin some day.
>>> - optimized DB schema
>> While this is good to hear, as the DB schema of the original NDOutils
>> was largely what prevented it from scaling even close to linearly, it
>> also makes me quite sad since there's now 3 separate projects that try
>> to achieve the same thing, and addon authors (like NagVis, Nagios-BP,
>> PNP4Nagios et al) have to add support for all of them if end-users are
>> to retain freedom of choice in the backend.
>>
>> At a first glance, your schema looks remarkably like that of the merlin
>> database (which in turn is derived from the monitor_gui database, which
>> in turn is derived from the nacoma database), with the major difference
>> that you use pluralistic names for your object tables ("hosts" instead
>> of "host") and some different names for some of your columns. Since
>> NagVis, Nagios-BP and PNP4Nagios already supports the Merlin database,
>> perhaps you should have a look and see what would be simple to change
>> to conform to that at least.
>>
>
> We discovered your project too recently. But that's almost certainly
> what we will try to do in the near future.
>
Sounds like a good plan. I see your db schema dates back only to June 15
(at least in SVN), so I guess you're not too fixed on it yet
>>> - multithreaded (use modern processors)
>> This doesn't buy one so much for an event-daemon running outside Nagios
>> as one may think, since Nagios is more than capable of saturating one
>> core by itself. Why was this determined necessary?
>>
>
> I don't really see what you mean (sorry for my poor english) but I'll
> try to explain why our daemon is multithreaded.
>
> Merethis is starting to work with some big client installation with
> many pollers. We just want to be able to collect all the event streams
> these pollers are sending at a single point. But there's so much data
> that the single-threaded ndo2db cannot process all events. So we'd
> like to have a broker daemon that can scale on multi-processor
> multi-core touchy machines.
>
It's perfectly possible to handle several hundred poller systems with
a single multiplexing thread though (and never having to worry about
locking is definitely nice). I found that when I started thinking
about implementing Merlin as a multithreaded daemon it just got messy
for the same reasons I couldn't use a networked message-queue to
implement the cross-host event transport mechanism. Either I'd have
multiple threads listening to the remote hosts and then writing and
sending from a
...[email truncated]...
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]
> On Wed, Jul 1, 2009 at 5:17 PM, Andreas Ericsson wrote:
>> Matthieu Kermagoret wrote:
>> This means there are four major contenders for the spot of being the
>> default database broker for Nagios; NDOUtils, Merlin, IDOUtils and
>> CentreonBroker. This is great, as it means the best technical solution
>> will win. At the same time, it's not so great since it means at least
>> four developers are working toward similar goals, but not working
>> together towards that goal.
>>
>
> I agree with you that there will certainly be some efforts wasted but
> it seems to me that each broker has its own peculariarity (replication
> for Merlin, performance for CentreonBroker, ...).
>
Well, Merlin handles 80k hosts and 500k services just fine, no matter
which host the checks are being executed on, so performance is hardly
something only CentreonBroker does well
>>> - multiple input ports
>>> - multiple database output
>>> - configurable logs
>> These sound neat though, although I fail to see how it can be useful to
>> have a single daemon output to more than one database since all databases
>> worth their salt already have replicating capabilities.
>>
>>> - TLS connection
>> This doesn't parse, or TLS doesn't mean thread-local storage to you.
>>
>
> I thought that now TLS is more known as the new name of SSL than of
> the name of a low-level feature.
>
I'm the stupid one here. Sorry for the noise. This sounds fairly
interesting though. I'll have to see how you've implemented this and
see if I can scrounge it for Merlin some day.
>>> - optimized DB schema
>> While this is good to hear, as the DB schema of the original NDOutils
>> was largely what prevented it from scaling even close to linearly, it
>> also makes me quite sad since there's now 3 separate projects that try
>> to achieve the same thing, and addon authors (like NagVis, Nagios-BP,
>> PNP4Nagios et al) have to add support for all of them if end-users are
>> to retain freedom of choice in the backend.
>>
>> At a first glance, your schema looks remarkably like that of the merlin
>> database (which in turn is derived from the monitor_gui database, which
>> in turn is derived from the nacoma database), with the major difference
>> that you use pluralistic names for your object tables ("hosts" instead
>> of "host") and some different names for some of your columns. Since
>> NagVis, Nagios-BP and PNP4Nagios already supports the Merlin database,
>> perhaps you should have a look and see what would be simple to change
>> to conform to that at least.
>>
>
> We discovered your project too recently. But that's almost certainly
> what we will try to do in the near future.
>
Sounds like a good plan. I see your db schema dates back only to June 15
(at least in SVN), so I guess you're not too fixed on it yet
>>> - multithreaded (use modern processors)
>> This doesn't buy one so much for an event-daemon running outside Nagios
>> as one may think, since Nagios is more than capable of saturating one
>> core by itself. Why was this determined necessary?
>>
>
> I don't really see what you mean (sorry for my poor english) but I'll
> try to explain why our daemon is multithreaded.
>
> Merethis is starting to work with some big client installation with
> many pollers. We just want to be able to collect all the event streams
> these pollers are sending at a single point. But there's so much data
> that the single-threaded ndo2db cannot process all events. So we'd
> like to have a broker daemon that can scale on multi-processor
> multi-core touchy machines.
>
It's perfectly possible to handle several hundred poller systems with
a single multiplexing thread though (and never having to worry about
locking is definitely nice). I found that when I started thinking
about implementing Merlin as a multithreaded daemon it just got messy
for the same reasons I couldn't use a networked message-queue to
implement the cross-host event transport mechanism. Either I'd have
multiple threads listening to the remote hosts and then writing and
sending from a
...[email truncated]...
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]