Nagios Log Server - Backup and Maintenance

[scottwilkerson]

this could be a mis-matched version of pip or virtualenv

Lets run the following

Code: Select all

pip install --upgrade pip 
pip install --upgrade virtualenv

then

Code: Select all

curator -v

[roddergreg]

Code: Select all

pip install --upgrade pip
Requirement already up-to-date: pip in /usr/lib/python2.6/site-packages
Cleaning up...

Code: Select all

pip install --upgrade virtualenv
Downloading/unpacking virtualenv
  Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded
Installing collected packages: virtualenv
Successfully installed virtualenv
Cleaning up...

Code: Select all

curator -v 
Traceback (most recent call last):
  File "/usr/bin/curator", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: elasticsearch>=1.0.0,<2.0.0

[roddergreg]

Not running a CentOS install running RHEL 6.5 but i've googled this error. This page was found.

https://github.com/elasticsearch/curator/wiki/FAQ

addb947e_o.png

[tmcdonald]

Were you able to run the full path?

Code: Select all

python /usr/lib/python2.6/site-packages/curator/curator.py -v

[roddergreg]

Code: Select all

 python /usr/lib/python2.6/site-packages/curator/curator.py -v
curator.py 1.2.2

The curator command runs successfully when I specify the full path

Code: Select all

su nagios -c "python /usr/lib/python2.6/site-packages/curator/curator.py close --older-than 5"
2014-12-16 16:46:23,043 INFO      Job starting...
2014-12-16 16:46:23,115 INFO      Beginning CLOSE operations...
2014-12-16 16:46:23,155 INFO      Attempting to close index logstash-2014.12.07.
2014-12-16 16:46:23,164 INFO      Skipping index logstash-2014.12.07: Already closed.
2014-12-16 16:46:23,164 INFO      Attempting to close index logstash-2014.12.08.
2014-12-16 16:46:23,173 INFO      Skipping index logstash-2014.12.08: Already closed.
2014-12-16 16:46:23,173 INFO      Attempting to close index logstash-2014.12.09.
2014-12-16 16:46:23,181 INFO      Skipping index logstash-2014.12.09: Already closed.
2014-12-16 16:46:23,181 INFO      Attempting to close index logstash-2014.12.10.
2014-12-16 16:46:23,191 INFO      Skipping index logstash-2014.12.10: Already closed.
2014-12-16 16:46:23,192 INFO      Attempting to close index logstash-2014.12.11.
2014-12-16 16:46:23,965 INFO      logstash-2014.12.11: Successfully closed.
2014-12-16 16:46:23,966 INFO      logstash-2014.12.12 is within the threshold period (5 days).
2014-12-16 16:46:23,966 INFO      logstash-2014.12.13 is within the threshold period (5 days).
2014-12-16 16:46:23,966 INFO      logstash-2014.12.14 is within the threshold period (5 days).
2014-12-16 16:46:23,966 INFO      logstash-2014.12.15 is within the threshold period (5 days).
2014-12-16 16:46:23,966 INFO      logstash-2014.12.16 is within the threshold period (5 days).
2014-12-16 16:46:23,966 INFO      CLOSE index operations completed.
2014-12-16 16:46:23,966 INFO      Done in 0:00:00.933866.

[jomann]

Can you do a:

Code: Select all

pip show elasticsearch

We are looking into a fix for this for the next release so this doesn't happen. As for now, you can change the curator command in cmdsubsys_helper.php if you'd like in the /var/www/html/nagioslogserver/application/helpers I would like the line numbers but there are multiple places in the file. You could find/replace "curator" with the full "python /x/x/x/curator.py" for now.

[roddergreg]

Code: Select all

pip show elasticsearch
---
Name: elasticsearch
Version: 1.2.0
Location: /usr/lib/python2.6/site-packages
Requires: urllib3

I've replaced all the entries of curator in cmdsubsys_helper.php with /usr/lib/python2.6/site-packages/curator/curator.py

Thanks for the workaround.

[lgroschen]

So the backups are working fully now? Let us know if we can close this topic.

[roddergreg]

Code: Select all

curl -XGET 'http://localhost:9200/nagioslogserver/commands/_search?pretty&q=command:do_maintenance'
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 4.8642325,
    "hits" : [ {
      "_index" : "nagioslogserver",
      "_type" : "commands",
      "_id" : "7CefFIprSgmAv0HEM0dxPg",
      "_score" : 4.8642325,
      "_source":{"created":"2014-10-28 11:46:59","active":1,"status":"waiting","type":"system","node":"global","command":"do_maintenance","run_time":1418918191,"frequency":"86400","last_run_output":"Maintenance and Backup jobs are being executed","last_run_time":"2014-12-17 10:56:31","last_run_status":"SUCCESS"}
    } ]
  }
}

It appears the maintenance ran successfully. The indexes were closed older than 5 days and indexes older than 10 were deleted. However the the backup location is empty. So it appears the maintenance is running correctly but the backups are failing nothing is created.

[lgroschen]

You have them going to /tmp right? Anything change since you posted last?

Post a screenshot of your Backup & Maintenance page