Brian,
The concept works good. If you run individual nfdump commands. Instead of seeing one nfdump command in 'top' using 100% of one CPU, I see many nfdump commands on many CPU using 50% or less CPU. So I am encouraged.
I suspect it would be better if the native python code that runs NNA did what the wrapper is trying to do. All I am doing is taking one BIG job, and making it into many small jobs. Then I use GNU Parallel to fire off all the small jobs to all the CPU. I think perl can spawn off separate jobs with their own PID, and rejoin the initial perl program when they finish. So GNU Parallel is not mandatory, just convenient.
This is all still make believe, in development, but look for me SteveBeauchemin on Github and you can clone the perl file. If you read the code, it should give you an idea of what I am doing. There are just some things I need to work around. Like Top Talkers. I don't know how to make those queries into smaller pieces. And the Chord diagrams that we all love, if you run 4 at the same time like one of the basic screens does, well... I am having difficulties there. But I just started to tinker and it is all new. And fun.
I just got NNA repaired and back to normal today. Yes, I broke it, then fixed it. So I am starting to play with nfdump again to see which actual NNA items work properly with the wrapper. I think I need the newer version of nfdump too, as I did reload and got it reverted to 1.6.13, and am now seeing errors that I did not have when I ran nfdump v1.6.15.
You know, I'm not a real programmer, I just play one here on Earth...
We can go PM if that makes sense. I'd like to have feedback. I think what I did so far is trivial for a real programmer. Once the concept is understood, and the nfdump parameters are explored. If you grok what I am doing to make this work, and can get it built into the native NNA, I'll wait for the new version.
But, other than complain about how slow stuff is, I prefer to poke the beast and see what I can do to make things better. I think you may have noticed that huh.
Let me know what you think.
Steve B
I am including in code brackets below all the changes I have made to the NNA stuff so far. (not the Nagios XI component though)
It makes NNA usable for me. Without these changes, not much fun... this makes NNA useful without nfdump wrapper.
Some stuff will still take forever to run and that is why I had the wrapper idea.
Code: Select all
Disable some automatic run of reports.
When clicking a menu to get to the reports tab, the currently selected item
will automatically run before you have a chance to change the menu item to
the report you actually want to see.
This change will change the default such that you need to click the run button.
vi /var/www/html/nagiosna/application/views/generic/reports.php
Change in vicinity of Line 33
// var run_change = true;
// SLB 2017-07-06 stop page from running automatically.
var run_change = false;
When selecting a Report - do not run simply because I chose it. What if I
missed by one item, and wanted the next one down the list. Only run once
the Run Report button is clicked.
Change in vicinnity of line 157
// var run_change = true;
// SLB 2017-07-10 stop page from running automatically.
run_change = false;
Caveat: When you do click the Run Report button, there is no indication that
the tool is trying to populate new data. The previous screen stays in place
until replaced by new content, no spinner, no indicator of work in progress.
The header changes but the table data does not. It is confusing and makes you
think it has the new data present.
Changes to add more time interval choices
The smallest window by default is 24 hours. These changes below
add in 2, 4, 6, and 12 hour choices for queries. You can always
use a custom time frame, but these are now available, and the default
is changed from 24 hours to 2 hours. This default change helps performance.
Files being changed are located here:
/var/www/html/nagiosna/www/media/js/helpers.js
/var/www/html/nagiosna/application/language/english/main_lang.php
/var/www/html/nagiosna/application/views/reports/home.php
/var/www/html/nagiosna/application/views/generic/reports.php
/var/www/html/nagiosna/application/views/generic/summary.php
/var/www/html/nagiosna/application/views/generic/queries.php
/var/www/html/nagiosna/application/views/queries/home.php
=============================================
diff OriginalFiles/helpers.js NewFiles-2017-07-19/helpers.js
83a84
> // SLB 2017-07-19 Added smaller time ranges
86c87,96
< if (start == '-24 hours' && end == '-1 second') {
---
> // ================= changed from here - SLB 2017-07-19
> if (start == '-2 hours' && end == '-1 second') {
> return "last 2 hours";
> } else if (start == '-4 hours' && end == '-1 second') {
> return "last 4 hours";
> } else if (start == '-6 hours' && end == '-1 second') {
> return "last 6 hours";
> } else if (start == '-12 hours' && end == '-1 second') {
> return "last 12 hours";
> } else if (start == '-24 hours' && end == '-1 second') {
87a98
> // ================= to here - SLB 2017-07-19
100a112
>
132a145
> // SLB 2017-07-19 changed seconds field
136c149,151
< var new_time = front[2] + '/' + front[0] + '/' + front[1] + '.' + date[1] + ':00';
---
> // SLB 2017-07-19 changed 00 to 59
> // var new_time = front[2] + '/' + front[0] + '/' + front[1] + '.' + date[1] + ':00';
> var new_time = front[2] + '/' + front[0] + '/' + front[1] + '.' + date[1] + ':59';
147a163
> // SLB 2017-07-19 Added smaller time ranges
151c167,176
< if (start == '-24 hours' && end == '-1 second') {
---
> // ================= changed from here - SLB 2017-07-19
> if (start == '-2 hours' && end == '-1 second') {
> tr.val('2h');
> } else if (start == '-4 hours' && end == '-1 second') {
> tr.val('4h');
> } else if (start == '-6 hours' && end == '-1 second') {
> tr.val('6h');
> } else if (start == '-12 hours' && end == '-1 second') {
> tr.val('12h');
> } else if (start == '-24 hours' && end == '-1 second') {
152a178
> // ================= to here - SLB 2017-07-19
184a211
> // SLB 2017-07-19 Added smaller time ranges
189c216,225
< if (select == '24h') {
---
> // ================= changed from here - SLB 2017-07-19
> if (select == '2h') {
> dates.start = '-2 hours';
> } else if (select == '4h') {
> dates.start = '-4 hours';
> } else if (select == '6h') {
> dates.start = '-6 hours';
> } else if (select == '12h') {
> dates.start = '-12 hours';
> } else if (select == '24h') {
192c228,229
< dates.start = '-48 hours';
---
> dates.start = '-2 days';
> // ================= to here - SLB 2017-07-19
231c268
< }
\ No newline at end of file
---
> }
=============================================
diff OriginalFiles/main_lang.php NewFiles-2017-07-19/main_lang.php
190a191,194
> $lang['timerange_2hrs'] = 'Last 2 Hours';
> $lang['timerange_4hrs'] = 'Last 4 Hours';
> $lang['timerange_6hrs'] = 'Last 6 Hours';
> $lang['timerange_12hrs'] = 'Last 12 Hours';
1051c1055
< $lang['update_no_logs'] = 'No updates have been performed through the web UI.';
\ No newline at end of file
---
> $lang['update_no_logs'] = 'No updates have been performed through the web UI.';
=============================================
diff OriginalFiles/reports-home.php NewFiles-2017-07-19/reports-home.php
29c29
< $('#timerange').val('24h').trigger('change');
---
> $('#timerange').val('2h').trigger('change');
306a307,310
> <option value="2h"><?php echo lang('timerange_2hrs'); ?></option>
> <option value="4h"><?php echo lang('timerange_4hrs'); ?></option>
> <option value="6h"><?php echo lang('timerange_6hrs'); ?></option>
> <option value="12h"><?php echo lang('timerange_12hrs'); ?></option>
322c326
< <input type="text" value="24 hours" title="<?php echo lang('timerange_elsapsed_desc'); ?>" class="input-small tt_bind" id="elapsed_start">
---
> <input type="text" value="2 hours" title="<?php echo lang('timerange_elsapsed_desc'); ?>" class="input-small tt_bind" id="elapsed_start">
=============================================
diff OriginalFiles/reports.php NewFiles-2017-07-19/reports.php
495a496,499
> <option value="2h"><?php echo lang('timerange_2hrs'); ?></option>
> <option value="4h"><?php echo lang('timerange_4hrs'); ?></option>
> <option value="6h"><?php echo lang('timerange_6hrs'); ?></option>
> <option value="12h"><?php echo lang('timerange_12hrs'); ?></option>
509c513
< <input type="text" value="24 hours" title="<?php echo lang('timerange_elsapsed_desc'); ?>" class="input-small tt_bind" id="elapsed_start">
---
> <input type="text" value="2 hours" title="<?php echo lang('timerange_elsapsed_desc'); ?>" class="input-small tt_bind" id="elapsed_start">
=============================================
diff OriginalFiles/summary.php NewFiles-2017-07-19/summary.php
40a41
> // SLB 2017-07-19 Change refresh to larger time - some reports just finish loading just as refresh sharts
42c43,47
< setInterval(function() { reload_visuals(); }, 300000);
---
> // setInterval(function() { reload_visuals(); }, 300000);
> // Have the graph & top talkers load fresh every 10 minutes
> // setInterval(function() { reload_visuals(); }, 600000);
> // Have the graph & top talkers load fresh every 15 minutes
> setInterval(function() { reload_visuals(); }, 900000);
91a97,100
> <option value="-2 hours|-1 second"><?php echo lang('timerange_2hrs'); ?></option>
> <option value="-4 hours|-1 second"><?php echo lang('timerange_4hrs'); ?></option>
> <option value="-6 hours|-1 second"><?php echo lang('timerange_6hrs'); ?></option>
> <option value="-12 hours|-1 second"><?php echo lang('timerange_12hrs'); ?></option>
=============================================
diff OriginalFiles/queries.php NewFiles-2017-07-19/queries.php
416a417,420
> <option value="2h"><?php echo lang('timerange_2hrs'); ?></option>
> <option value="4h"><?php echo lang('timerange_4hrs'); ?></option>
> <option value="6h"><?php echo lang('timerange_6hrs'); ?></option>
> <option value="12h"><?php echo lang('timerange_12hrs'); ?></option>
430c434
< <input type="text" value="24 hours" title="<?php echo lang('timerange_elsapsed_desc'); ?>" class="input-small tt_bind custom-field" id="elapsed_start">
---
> <input type="text" value="2 hours" title="<?php echo lang('timerange_elsapsed_desc'); ?>" class="input-small tt_bind custom-field" id="elapsed_start">
612c616
< <?php if (empty($download)) { echo $footer; } ?>
\ No newline at end of file
---
> <?php if (empty($download)) { echo $footer; } ?>
=============================================
diff OriginalFiles/queries-home.php NewFiles-2017-07-19/queries-home.php
35c35,37
< $('#timerange').val('24h').trigger('change');
---
> // SLB 2017-07-19 Changed 24h to 2h
> // $('#timerange').val('24h').trigger('change');
> $('#timerange').val('2h').trigger('change');
287a290,293
> <option value="2h"><?php echo lang('timerange_2hrs'); ?></option>
> <option value="4h"><?php echo lang('timerange_4hrs'); ?></option>
> <option value="6h"><?php echo lang('timerange_6hrs'); ?></option>
> <option value="12h"><?php echo lang('timerange_12hrs'); ?></option>
301c307
< <input type="text" value="24 hours" title="<?php echo lang('timerange_elsapsed_desc'); ?>" class="input-small tt_bind" id="elapsed_start">
---
> <input type="text" value="2 hours" title="<?php echo lang('timerange_elsapsed_desc'); ?>" class="input-small tt_bind" id="elapsed_start">
352c358
< <?php echo $footer; ?>
\ No newline at end of file
---
> <?php echo $footer; ?>
=============================================
