Search found 22 matches

by mark.payne
Tue Dec 01, 2015 7:28 pm
Forum: Nagios Log Server
Topic: Backups failing | OutOfMemoryError[Java heap space]
Replies: 15
Views: 3397

Re: Backups failing | OutOfMemoryError[Java heap space]

We are on the latest 1.3.0
This is the version we installed, not an upgrade from a earlier version.
by mark.payne
Tue Dec 01, 2015 5:33 pm
Forum: Nagios Log Server
Topic: Backups failing | OutOfMemoryError[Java heap space]
Replies: 15
Views: 3397

Re: Backups failing | OutOfMemoryError[Java heap space]

I reran the following commands at 11:30 local time on each server: python /usr/lib/python2.7/site-packages/curator/curator.py show --show-snapshots --repository 'backup' python /usr/lib/python2.7/site-packages/curator/curator.py snapshot --older-than 1 --repository 'backup' Then I ran on each server...
by mark.payne
Mon Nov 30, 2015 7:55 pm
Forum: Nagios Log Server
Topic: Backups failing | OutOfMemoryError[Java heap space]
Replies: 15
Views: 3397

Re: Backups failing | OutOfMemoryError[Java heap space]

We are on version 1.3 python /usr/lib/python2.7/site-packages/curator/curator.py show --show-snapshots --repository 'backup' This out puts the following: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/curator/curator.py", line 736, in <module> main() File "/...
by mark.payne
Wed Nov 25, 2015 3:42 pm
Forum: Nagios Log Server
Topic: Backups failing | OutOfMemoryError[Java heap space]
Replies: 15
Views: 3397

Re: Backups failing | OutOfMemoryError[Java heap space]

I have reverted the HEAP changes back to default. Information requested below: md5sum /usr/lib/python2.7/site-packages/curator/curator.py 9d19626b8486f05156c77a0dacc93343 /usr/lib/python2.7/site-packages/curator/curator.py curl localhost:9200/_snapshot/_all {"backup":{"type":&quo...
by mark.payne
Tue Nov 24, 2015 9:30 pm
Forum: Nagios Log Server
Topic: Backups failing | OutOfMemoryError[Java heap space]
Replies: 15
Views: 3397

Re: Backups failing | OutOfMemoryError[Java heap space]

I increased each server to 32GB. Still getting the same error: 2015-11-25 15:29:29,804 INFO Beginning SNAPSHOT operations... Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/curator/curator.py", line 736, in <module> main() File "/usr/lib/python2.7/site-packag...
by mark.payne
Tue Nov 24, 2015 8:13 pm
Forum: Nagios Log Server
Topic: Backups failing | OutOfMemoryError[Java heap space]
Replies: 15
Views: 3397

Re: Backups failing | OutOfMemoryError[Java heap space]

I changed the heap size to 8g and this was ok but still received the same issue.
by mark.payne
Tue Nov 24, 2015 7:36 pm
Forum: Nagios Log Server
Topic: Backups failing | OutOfMemoryError[Java heap space]
Replies: 15
Views: 3397

Re: Backups failing | OutOfMemoryError[Java heap space]

Currently it is set to ES_HEAP_SIZE=$(expr $(free -m|awk '/^Mem:/{print $2}') / 2 )m If I set this to ES_HEAP_SIZE=4g I believe this would give ES less memory than is already being allocated. I tried it as 4g and now there is 65% free memory out of 16GB when it was 36% free so ES doesn't have as muc...
by mark.payne
Tue Nov 24, 2015 6:38 pm
Forum: Nagios Log Server
Topic: Backups failing | OutOfMemoryError[Java heap space]
Replies: 15
Views: 3397

Backups failing | OutOfMemoryError[Java heap space]

I have two servers in a cluster each with a backup share mounted locally. It managed to backup the first days backup then stopped working. No "backup snapshots" are present in Backup and Maintenance. Both servers can access the share. I have reset all jobs and rerun but still doesn't work....
by mark.payne
Thu Oct 29, 2015 11:54 pm
Forum: Nagios Log Server
Topic: Cannot add new install to existing cluster
Replies: 9
Views: 1946

Re: Cannot add new install to existing cluster

Great thanks for that, sometimes we need to double check things just to be sure. After I installed I added another drive and moved the directory and data to the new drive. Was this on the new or existing server? Did you follow any documented procedure for this? What is the permissions of: ll /mnt/s...
by mark.payne
Thu Oct 29, 2015 11:52 pm
Forum: Nagios Log Server
Topic: Cannot add new install to existing cluster
Replies: 9
Views: 1946

Re: Cannot add new install to existing cluster

Reverted the move of the data directory and can now add to the cluster. Moved the data back to the new drive and all is well. I guess shouldn't be changing data around until the cluster is setup... How this would work when you have large amounts of data already to replicate and the drive you initial...