Hanging queries

[aparfenov]

We are evaluating the demo VM (nagiosna-2.2.3-64.ova), the source is a Linux server 10.1.2.3 with fprobe installed. Data lifetime is 24 hours, disk usage – 34M.

Why are some queries taking forever to complete? For example:

"dst ip 10.1.2.3" aggregated by dstip,srcip works
"dst ip 10.1.2.3" aggregated by dstport,srcip hangs
"dst ip 10.1.2.3" aggregated by srcip works
"dst ip 10.1.2.3" aggregated by dstport works
"dst ip 10.1.2.3" aggregated by srcip,dstport hangs

The server does not show any CPU utilization, and a chord daigram is quicky shown, but query results are never returned.

Also what is the logic behind chord diagrams in queries? Reports have 4 different diagrams, but queries seem to always show only one.

Sincerely,
Anthony

[tgriep]

There are some setting that have to be changed to get the NNA GUI to process large Queries.
Login as root and edit the /etc/php.ini file
Add the following line to the bottom of the file

Code: Select all

max_input_vars=50000

Save it and edit this file

Code: Select all

/etc/httpd/conf/httpd.conf

Add the following line to the bottom of the file

Code: Select all

LimitRequestLine 100000

Save the file and restart Apache for the changes to take affect.

Code: Select all

service httpd restart

Depending on how much data is in your source, you may have to increase those numbers.

Try it out and let us know how it works for you.

[aparfenov]

Thank you for your suggestions, but there is no difference.
Data lifetime is 24 hours, disk usage – 34M. Is it considered a lot of flow data? I have also tried to set max_input_vars to 100000.
It can quickly aggregate on dstport and srcip, but not on both.

[tgriep]

Did you increase this option

Code: Select all

LimitRequestLine 100000

in the /etc/httpd/conf/httpd.conf file to a larger value as well?

Increase that value and do the following too.

Edit the /etc/php.ini file and change the following from

Code: Select all

max_execution_time = 30
max_input_time = 60
memory_limit = 128M

to

Code: Select all

max_execution_time = 60
max_input_time = 120
memory_limit = 512M

Save the file and restart Apache for the changes to take affect.

Code: Select all

service httpd restart

How many CPU's do you have allocated to the server if it is running in a virtual environment?
If you only have 2 allocated, try adding 2 more.

If the above changes do not work, run the following as root but replace <sourcename> with the actual name of the source you are trying to run the query against.

Code: Select all

nfdump  -R  /usr/local/nagiosna/var/<sourcename>/flows -A dstport,srcport,dstip,srcip 'dst ip 10.1.2.3' -O bytes -o csv |wc -l

Post the output, it should print out the number of lines the query generated.

[aparfenov]

Your nfdump command quickly returned 291529
"-A dstport,srcip" quickly returned 170481

I have increased max_input_vars to 300000 in addition to changing other values you have recommended. But the query in GUI still hangs, and the server does not show any CPU utilization, while it is running.

Are these numbers just too big to be used in GUI (too many pages in query output)?

[tgriep]

The numbers could be to large to be loaded in the GUI.
It could still take a while to the GUI to render all of the data so let it run for 10 to 20 minutes.

FYI, if you printed the query with the 170 thousand lines, it would be slightly less that 10000 pages.

Can you run this as root and post the output?

Code: Select all

grep LimitRequestLine  /etc/httpd/conf/httpd.conf

[aparfenov]

[root@nnademo ~]# grep LimitRequestLine /etc/httpd/conf/httpd.conf
LimitRequestLine 100000

[tgriep]

Edit this file

Code: Select all

/etc/httpd/conf/httpd.conf

Change this line from

Code: Select all

LimitRequestLine 100000

to

Code: Select all

LimitRequestLine 1000000

Save the file and restart apache by running

Code: Select all

service httpd restart

See if the query works after that change.

[aparfenov]

The query in GUI still hangs. I guess it is probably just too big.

[tgriep]

Almost 300 thousand lines is a lot of data to display so the web interface may never display it even with the increased values.