Page 1 of 1
Query always not finished!!!
Posted: Thu May 05, 2016 9:08 am
by eric.lin826
Hi
When I Use the query function to query a week's data, the job never finished.
Any one can help me to figure out what is the problem?
My Server Infomation:
CentOS 6.7 x64
Nagios Network Analyzer 2.2.0
Re: Query always not finished!!!
Posted: Thu May 05, 2016 10:20 am
by bwallace
Without troubleshooting, I can tell you right off the bat this is most likely related to nfdump and that it reads flow data single-threaded. Nfdump is a 3rd party component in NNA and other products which rely on nfdump experience the same issues:
https://www.mail-archive.com/nfsen-disc ... 02819.html
It's not ideal, but in other cases similar to yours, we've suggested the following
1. Filter the incoming netflow using views (narrow the scope by including only the ports/IPs you're interested in)
2. Use small timeframes when running queries
3. Get faster I/O
I am curious, could you post the output of
du -h /usr/local/nagiosna/var/ so we know the size of the files NNA needs to parse at the moment?
Re: Query always not finished!!!
Posted: Thu May 12, 2016 1:10 am
by joeywu
Hi,
I had try the command what you suggest , i had upload the result screenshot in the attachment,
in the screenshot you can see there is a file which named Internet , that is our target , it only record 3 weeks data
and the data lifetime is 60 days.
Re: Query always not finished!!!
Posted: Thu May 12, 2016 12:57 pm
by tmcdonald
For reference, these two posters appear to be colleagues in case there was any confusion as to who we are working with.
Can you run the query and then post the output of the top command? That will give us a better idea of what is happening when the query is running.
Also, what sort of disks do you have on these machines? Are they local or network-attached? SSD or disc-based?
Re: Query always not finished!!!
Posted: Fri May 13, 2016 2:57 am
by joeywu
Hi ,
Actually , eric.lin826 and i are colleagues , and i am the owner of this issue.
I had upload the screenshot of TOP Command when i running query in attachment.
My server Spec as below
HP DL380 Gen8 (Physical Machine not VM)
CPU :24Core X5675
Memory : 128G
Disk : 500G (Local)
OS : CentOS 6.7 Min
Re: Query always not finished!!!
Posted: Fri May 13, 2016 12:36 pm
by tgriep
Can you run the following command as root on the NA server and then run the query and post the output of the tail command.
If your query is large, you may want to edit the /etc/php.ini file and increase the limits like the example below.
Code: Select all
max_execution_time = 60 ; Maximum execution time of each script, in seconds
max_input_time = 60 ; Maximum amount of time each script may spend parsing request data
memory_limit = 256M ; Maximum amount of memory a script may consume
If it still fails, increase them again and see if that helps.
Re: Query always not finished!!!
Posted: Wed May 18, 2016 10:13 am
by joeywu
Hi,
I had modify my php.ini , and it seems work , my query job seems normal.
BTW , is there any documentations or reference for configure tuning , something like
Query 1G data , memory limit = 1024MB,
Query 100m data , memory limit = 128MB
Thanks.
Re: Query always not finished!!!
Posted: Wed May 18, 2016 10:25 am
by tgriep
Glad that is it working for you now.
There isn't a hard and fast setting for the php.ini file. Every system has different needs so it is hard to define.
Shall we close and lock up this post as solved?
Re: Query always not finished!!!
Posted: Sun May 22, 2016 9:52 pm
by joeywu
Sure ,
Please help me to close this ticket , thanks for all your help!