Nagios Support Forum

npolovenko wrote:@CameronWP, Please open the elasticsearch.yml file:

Code: Select all

nano /usr/local/nagioslogserver/elasticsearch/config/elasticsearch.yml

And add this line to the end of the file:

Code: Select all

threadpool.search.queue_size: -1

And after that please restart the elasticsearch with:

Code: Select all

service elasticsearch restart

I also recommend increasing the ram and the CPU for this VM. Seems like a lot of these problems are the result of resource insufficiency, or the query is too long.

I will do this but you should be aware that this isn't a VM but an HP DL380G8 with 64GB of RAM with nothing else running on it except Log Server. I am trying to do a simple search with one query item and two filters and it is crashing things. It does seem like it may be a bug.

Here is a shot of current resource usage:

mcapra wrote: There's a very good reason the ElasticSearch search queue size is capped: It leaves room for write operations. Exhausting your thread pool with a large search seems like it could potentially introduce write failures.

OP, the above notwithstanding, can you change it to -1 temporarily to see if it reduces this one particular error? That will give the devs some guidance on what I assume will end up being a bug report.

Hello:

I made the change as requested and I am running into the same issues. The first export post-change only had 1228 rows. I re-ran the export with no changes (literally just clicked the link again) and that time I got 1923 records. I ran it a third time in the same fashion and got 1923 again but it seems pretty inconsistent.

Thanks!

@CameronWP, I talked to our dev team and they said it's unlikely a threadpool.search.queue_size issue. You can change it back to 1k.
Can you make sure that the dashboard that you're using for this query is saved? Sometimes when you run a query on an unsaved dashboard you can get this kind of behavior.

I have removed that line from the nano /usr/local/nagioslogserver/elasticsearch/config/elasticsearch.yml.
This is a saved query so I don't think that is causing my issue.

We can continue to troubleshoot this on the forum, but at this point it might make sense to open a ticket at https://support.nagios.com/tickets/

It will be up to the tech that takes the ticket as to when, but opening a ticket will allow us to eventually do a remote session.

Ok, sounds good! I have opened a ticket.

Thanks!

locking because ticket received