Query always not finished!!!

[eric.lin826]

Hi
When I Use the query function to query a week's data, the job never finished.
Any one can help me to figure out what is the problem?

My Server Infomation:
CentOS 6.7 x64
Nagios Network Analyzer 2.2.0

[bwallace]

Without troubleshooting, I can tell you right off the bat this is most likely related to nfdump and that it reads flow data single-threaded. Nfdump is a 3rd party component in NNA and other products which rely on nfdump experience the same issues:
https://www.mail-archive.com/nfsen-disc ... 02819.html

It's not ideal, but in other cases similar to yours, we've suggested the following

1. Filter the incoming netflow using views (narrow the scope by including only the ports/IPs you're interested in)
2. Use small timeframes when running queries
3. Get faster I/O

I am curious, could you post the output of du -h /usr/local/nagiosna/var/ so we know the size of the files NNA needs to parse at the moment?

[joeywu]

Hi,

I had try the command what you suggest , i had upload the result screenshot in the attachment,

in the screenshot you can see there is a file which named Internet , that is our target , it only record 3 weeks data

and the data lifetime is 60 days.

[tmcdonald]

For reference, these two posters appear to be colleagues in case there was any confusion as to who we are working with.

Can you run the query and then post the output of the top command? That will give us a better idea of what is happening when the query is running.

Also, what sort of disks do you have on these machines? Are they local or network-attached? SSD or disc-based?

[joeywu]

Hi ,
Actually , eric.lin826 and i are colleagues , and i am the owner of this issue.

I had upload the screenshot of TOP Command when i running query in attachment.

My server Spec as below

HP DL380 Gen8 (Physical Machine not VM)

CPU :24Core X5675
Memory : 128G
Disk : 500G (Local)
OS : CentOS 6.7 Min

[tgriep]

Can you run the following command as root on the NA server and then run the query and post the output of the tail command.

Code: Select all

tail -f /var/log/httpd/*_log

If your query is large, you may want to edit the /etc/php.ini file and increase the limits like the example below.

Code: Select all

 max_execution_time = 60     ; Maximum execution time of each script, in  seconds
 max_input_time = 60     ; Maximum amount of time each script may spend parsing request data
 memory_limit = 256M      ; Maximum amount of memory a script may consume 

If it still fails, increase them again and see if that helps.

[joeywu]

Hi,

I had modify my php.ini , and it seems work , my query job seems normal.

BTW , is there any documentations or reference for configure tuning , something like

Query 1G data , memory limit = 1024MB,
Query 100m data , memory limit = 128MB

Thanks.

[tgriep]

Glad that is it working for you now.
There isn't a hard and fast setting for the php.ini file. Every system has different needs so it is hard to define.
Shall we close and lock up this post as solved?

[joeywu]

Sure ,

Please help me to close this ticket , thanks for all your help!