Page 1 of 2
Maintenance doesn't seem to run
Posted: Wed Feb 11, 2015 10:11 am
by ljorg
Hi,
Ive set maintenance to close any index older than seven days, but it doesn't do anything. I have to close indexes manually. It appears that with more than 130 million records (or so) the performance of Nagios Log Server crawls to a stop. Adding extra memory or cores doesn't seem to help, but closing old indexes helps tremendously.
My settings are shown in the attachment below
Clipboard01.jpg
Re: Maintenance doesn't seem to run
Posted: Wed Feb 11, 2015 10:15 am
by teirekos
Have you tried to configure the heap memory? Check the link below:
http://www.elasticsearch.org/guide/en/e ... izing.html
Re: Maintenance doesn't seem to run
Posted: Wed Feb 11, 2015 11:24 am
by scottwilkerson
I agree with @teirekos about the heap memory.
Also can you run the following command and return the output
Code: Select all
curl -XGET "http://localhost:9200/nagioslogserver/commands/_search?pretty&q=command:do_maintenance"
Re: Maintenance doesn't seem to run
Posted: Thu Feb 12, 2015 2:30 am
by ljorg
scottwilkerson wrote:I agree with @teirekos about the heap memory.
I'll get on top of that soon.
scottwilkerson wrote:Also can you run the following command and return the output
This is the output:
Code: Select all
{
"took" : 9,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 4.749504,
"hits" : [ {
"_index" : "nagioslogserver",
"_type" : "commands",
"_id" : "ZM65w-m1QFy9is8Ebkdw0w",
"_score" : 4.749504,
"_source":{"created":"2014-12-09 15:02:37","active":1,"status":"waiting","type":"system","node":"global","command":"do_maintenance","run_time":1423749939,"frequency":"86400","last_run_output":"Maintenance and Backup jobs are being executed","last_run_time":"2015-02-11 15:05:39","last_run_status":"SUCCESS"}
} ]
}
}
Re: Maintenance doesn't seem to run
Posted: Thu Feb 12, 2015 8:59 am
by scottwilkerson
It appears to be running, it was last executed on "2015-02-11 15:05:39"
One thing to note, on some systems we have noticed a bug releated to the addon that processes these. You can determine if you are affected by these by running the following from the command line.
this bug will be resolved in the next release due out soon.
Re: Maintenance doesn't seem to run
Posted: Thu Feb 12, 2015 10:49 am
by ljorg
The output of curator:
Code: Select all
Traceback (most recent call last):
File "/usr/bin/curator", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in <module>
working_set.require(__requires__)
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: elasticsearch>=1.0.0,<2.0.0
Does that mean I'm affected and is that why maintanence doesn't work although it runs?
Lars
Re: Maintenance doesn't seem to run
Posted: Thu Feb 12, 2015 6:00 pm
by tmcdonald
How did you install this server? One of our pre-built VMs, or did you use fullinstall on an existing server? It almost looks like python was already installed. What version is it?
Re: Maintenance doesn't seem to run
Posted: Fri Feb 13, 2015 4:57 am
by ljorg
It's installed on a RedHat Enterprise Linux 6. Python is probably installed by default. It's version 2.2.6.
Lars
Re: Maintenance doesn't seem to run
Posted: Fri Feb 13, 2015 10:27 am
by scottwilkerson
Nagios Log Server 1.3 came out yesterday which has a fix for this problem.
http://library.nagios.com/library/produ ... /downloads
Re: Maintenance doesn't seem to run
Posted: Mon Feb 16, 2015 9:49 am
by teirekos
I've installed NLS 1.3 I left it all the weekend and still no snapshots created.