Re: [Nagios-devel] RFC embedded Perl Nagios changes: usability and performance.
Posted: Sun Jan 04, 2004 5:11 pm
Hi Stanley -
Your suggestions certainly sound worth pursuing. I try to stay as
far away as possible from the epn code, but if you submit patches
against CVS HEAD (2.x), I'll include them in 2.0. The one change I
did make in 2.0 is add a config file option to unload/reload the epn
after a certain number of uses to possibly help cut down on memory
leaks (not sure if it will help though).
On 1 Jan 2004 at 13:54, Stanley Hopcroft wrote:
> Dear Ladies and Gentlemen,
>
> I am writing to request your comments on proposed changes to the
> embedded Perl interpreter support in Nagios (ePN).
>
> This letter has two sections: part A is general discussion and the
> proposals; B concerns testing.
>
> Part A Discussion and Proposals.
>
> The ePN support contributed by Mr Stephen Davies embeds a Perl
> interpreter into the Nagios binary and (with much cleverness) allows
> Perl plugins to be compiled only once before they are run (as well as
> letting the Plugin call the Perl exit without trashing the
> interpreter; saving the output the plugin writes to STDOUT and other
> good things)
>
> ePN provides these benefits to Perl plugins and Nagios
>
> 1 Perl plugins are not subject to the OS forking the Nagios process;
> Perl plugins are called as Nagios functions
>
> While Nagios forks to execute _each_ plugin, an ePN Nagios does not
> request another fork (in the popen() system call) to run the plugin.
>
> 2 Perl plugins are compiled only once, saving both an exec of the Perl
> interpreter and the Perl compilation phase (to the Perl op-code parse
> tree) each time a Perl plugin is run. Since the compilation phase may
> include loading Perl modules required by the plugin (some of which are
> huge), there is quite a saving or work if not execution time (since
> Perl is pretty fast) in single compilation.
>
> There are also ePN tradeoffs such as markedly increased memory
> consumption (the Perl parse trees remain in memory).
>
> The ePN implementation that has performed well in both Netsaint
> 0.0.[4-7] and Nagios 1.[0-1] and it seems to be remains unchanged in
> the HEAD CVS branch (the Perl calls in the head checks.c seem to be
> the same as those in 1.x, and p1.pl appears unchanged), could be
> enhanced in these areas
>
> 1 Performance
>
> Three observations concerned with performance are that
>
> 1.1 The plugin output is returned to Nagios through the file system
> (rather than as an extra element of a list returned by
> Embed::Persistent::run_plugin).
>
> Instead of STDOUT being tied to the file system it could be tied to an
> in memory data structure (probably scalar) and the value the plugin
> 'writes' as output returned as an extra element on the Perl stack.
>
> This I think is a straight forward enhancement that saves Nagios
> system calls to generate a temporary file name, open the file, read
> the line of plugin output and unlink the file.
>
> Unfortunately, I don't know why STDOUT was tied to a file: it doesn't
> seem to have any debugging advantages because the contents are either
> logged by Nag (and the file unlinked) or there are _no_ contents.
>
> 1.2 The ePN author comments specifically on the separation of the
> parse and execution phases
>
> '
> # Only major changes are to separate the compiling and cacheing from
> # the execution so that the cache can be kept in "non-volatile" parent
> # process while the execution is done from "volatile" child processes
> '
>
> Unfortunately, I dnn't understand this (presumably the 'processes' are
> the eval_file and run_package subroutines in the Embed::Persistent
> package. However, while eval_file is only concerned whether to parse
> the plugin, the data structure that it uses to avoid reparsing
> unchanged plugins that have already been compiled (%Cache) is an entry
> in the package symbol table: it is visible to both subroutines).
>
> It _may_ be better to replace the two Nagios calls to Perl (one for
> eval_file and the other for run_package) by one call to a new Perl
> subroutine that optionally compiles and runs the plugin.
>
> This change is extensive; I have no plans to
...[email truncated]...
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]
Your suggestions certainly sound worth pursuing. I try to stay as
far away as possible from the epn code, but if you submit patches
against CVS HEAD (2.x), I'll include them in 2.0. The one change I
did make in 2.0 is add a config file option to unload/reload the epn
after a certain number of uses to possibly help cut down on memory
leaks (not sure if it will help though).
On 1 Jan 2004 at 13:54, Stanley Hopcroft wrote:
> Dear Ladies and Gentlemen,
>
> I am writing to request your comments on proposed changes to the
> embedded Perl interpreter support in Nagios (ePN).
>
> This letter has two sections: part A is general discussion and the
> proposals; B concerns testing.
>
> Part A Discussion and Proposals.
>
> The ePN support contributed by Mr Stephen Davies embeds a Perl
> interpreter into the Nagios binary and (with much cleverness) allows
> Perl plugins to be compiled only once before they are run (as well as
> letting the Plugin call the Perl exit without trashing the
> interpreter; saving the output the plugin writes to STDOUT and other
> good things)
>
> ePN provides these benefits to Perl plugins and Nagios
>
> 1 Perl plugins are not subject to the OS forking the Nagios process;
> Perl plugins are called as Nagios functions
>
> While Nagios forks to execute _each_ plugin, an ePN Nagios does not
> request another fork (in the popen() system call) to run the plugin.
>
> 2 Perl plugins are compiled only once, saving both an exec of the Perl
> interpreter and the Perl compilation phase (to the Perl op-code parse
> tree) each time a Perl plugin is run. Since the compilation phase may
> include loading Perl modules required by the plugin (some of which are
> huge), there is quite a saving or work if not execution time (since
> Perl is pretty fast) in single compilation.
>
> There are also ePN tradeoffs such as markedly increased memory
> consumption (the Perl parse trees remain in memory).
>
> The ePN implementation that has performed well in both Netsaint
> 0.0.[4-7] and Nagios 1.[0-1] and it seems to be remains unchanged in
> the HEAD CVS branch (the Perl calls in the head checks.c seem to be
> the same as those in 1.x, and p1.pl appears unchanged), could be
> enhanced in these areas
>
> 1 Performance
>
> Three observations concerned with performance are that
>
> 1.1 The plugin output is returned to Nagios through the file system
> (rather than as an extra element of a list returned by
> Embed::Persistent::run_plugin).
>
> Instead of STDOUT being tied to the file system it could be tied to an
> in memory data structure (probably scalar) and the value the plugin
> 'writes' as output returned as an extra element on the Perl stack.
>
> This I think is a straight forward enhancement that saves Nagios
> system calls to generate a temporary file name, open the file, read
> the line of plugin output and unlink the file.
>
> Unfortunately, I don't know why STDOUT was tied to a file: it doesn't
> seem to have any debugging advantages because the contents are either
> logged by Nag (and the file unlinked) or there are _no_ contents.
>
> 1.2 The ePN author comments specifically on the separation of the
> parse and execution phases
>
> '
> # Only major changes are to separate the compiling and cacheing from
> # the execution so that the cache can be kept in "non-volatile" parent
> # process while the execution is done from "volatile" child processes
> '
>
> Unfortunately, I dnn't understand this (presumably the 'processes' are
> the eval_file and run_package subroutines in the Embed::Persistent
> package. However, while eval_file is only concerned whether to parse
> the plugin, the data structure that it uses to avoid reparsing
> unchanged plugins that have already been compiled (%Cache) is an entry
> in the package symbol table: it is visible to both subroutines).
>
> It _may_ be better to replace the two Nagios calls to Perl (one for
> eval_file and the other for run_package) by one call to a new Perl
> subroutine that optionally compiles and runs the plugin.
>
> This change is extensive; I have no plans to
...[email truncated]...
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]