NNA GUI extremely slow

[lucas.shelton]

We've installed NNA as a VM and are trying to configure the reporting, queries, etc but the GUI is extremely slow. We've checked resources allocated to the VM and they are all fine. As of now it's barely consuming resources. Has anybody else encountered this issue?

[ssax]

What version of NNA are you using? You can grab it from the bottom left hand corner of the web interface.

How much RAM and CPU are you allocating?

Under Administration > Global Settings do you have resolve hostnames enabled (Reports and Queries or Graphs)?

What does the load look like?

Code: Select all

top

[lucas.shelton]

We've allocated 8GB of RAM, which is currently about 70% utilization. A bit high but shouldn't be causing the types of delays I'm seeing. What I do notice is the swap usage is maxed at 100%. What exactly does the swap usage do, and is there a way to bring the utilization down to a reasonable level.

We're running 2R1.0.

Also the resolve hostnames box is not checked.

[jdalrymple]

Code: Select all

du -h /usr/local/nagiosna/var/flows

please

[lucas.shelton]

Top:

top - 15:54:45 up 13 days, 16:53, 1 user, load average: 0.00, 0.03, 0.13
Tasks: 163 total, 2 running, 161 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 33.3 sy, 0.0 ni, 66.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 7748236 total, 2586792 free, 348268 used, 4813176 buff/cache
KiB Swap: 2097148 total, 2097148 free, 0 used. 6896000 avail Mem


du -h /usr/local/nagiosna/var/XXXPA3050/flows:

2.7G /usr/local/nagiosna/var/XXXPA3050/flows/

[jdalrymple]

NNA is a disk AND CPU hungry beast - mostly disk. Queries for any source involving the system manipulating capture files for that sources entire lifecycle, even if only querying in a short timespan.

I'm going to guess you only have 1 source
I'm also going to guess that you have a pretty long lifetime on your data

Are either of those guesses incorrect?

Going further, what can you tell us about the hardware specs for your machine, in particular the disk backing it?

[lucas.shelton]

We are collecting from two Palo Alto firewalls, the lifetime is 24 hours. It's running in a VMWare environment, we have three Dell r730 holding the discs.

[jdalrymple]

So your disks are local to the hypervisor? I guess with that I would advise a few things.

1) You can isolate whether this is a disk issue at all by watching `top` while issuing a query or some other such frontend action that is causing you a delay. Wathc for high percentages in the wait category. If it's not a disk issue, great, we'll look towards other things. If it is however, then:

i - Make sure there aren't problems with your disk controller such as disabled write cache or battery issues. These types of problems cause the disks to be overly busy performing writes so that your data is secure and doesn't leave much time for the disks to do the necessary reading that NNA needs.

ii - Look in your VMware vSphere console for high io wait times for this VM.

As I mentioned, we have seen some slowdowns in massive environments courtesy of the outlined performance shortcomings. It sounds like your environment is pretty small though, and given the right hardware we've not seen where NNA wouldn't handle your size environment without slowdown.

[lucas.shelton]

No, this is a SAN setup. We think we've got it narrowed down to a CPU issue. When I run "top" and issue a query, the nfdump process jumps to 100%. However, the overall CPU load is only 25%. Since we have four CPU's dedicated to this VM (Xen not VMWare) this tells me that this process is only using one of the CPU's and not distributing the load across all four CPU's. I've attached a screenshot showing what I'm talking about.

[lucas.shelton]

BTW would it be beneficial to have this thread moved to the customer support forum? We have a support contract for Nagios.