Cap the amount of memory logstash and elasticsearch can use?
Posted: Thu Nov 07, 2019 2:31 pm
Hello,
Is it possible to place a hard cap on the amount of memory that logstash and elasticsearch are allowed to utilize? Perhaps by percentage rather than a specific GB amount?
Each node in my Log Server instances has 64GBs of memory, but whenever the logstash or elasticsearch services need to restart there isn't enough available memory for them to do it and they end up failing to stop properly. For example, I updated my nodes to version 2.1.1 this morning and when one of them tried to stop elasticsearch it threw the following error:
It happens every time. I think it's because elasticsearch is using more than 50% of the total memory.
Adding more memory to the system wouldn't help would it? I was thinking we could cap the memory usage for elasticsearch to like 49% or something like that? Not sure if that's even possible.
Thank you.
Is it possible to place a hard cap on the amount of memory that logstash and elasticsearch are allowed to utilize? Perhaps by percentage rather than a specific GB amount?
Each node in my Log Server instances has 64GBs of memory, but whenever the logstash or elasticsearch services need to restart there isn't enough available memory for them to do it and they end up failing to stop properly. For example, I updated my nodes to version 2.1.1 this morning and when one of them tried to stop elasticsearch it threw the following error:
Code: Select all
root@nagioslscc2:/root> systemctl status elasticsearch
● elasticsearch.service - LSB: This service manages the elasticsearch daemon
Loaded: loaded (/etc/rc.d/init.d/elasticsearch; bad; vendor preset: disabled)
Active: active (running) since Thu 2019-11-07 11:10:08 PST; 11min ago
Docs: man:systemd-sysv-generator(8)
Process: 15631 ExecStop=/etc/rc.d/init.d/elasticsearch stop (code=exited, status=1/FAILURE)
Process: 15857 ExecStart=/etc/rc.d/init.d/elasticsearch start (code=exited, status=0/SUCCESS)
Tasks: 100
CGroup: /system.slice/elasticsearch.service
└─15883 /bin/java -Xms32133m -Xmx32133m -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDump...
Nov 07 11:10:08 nagioslscc2 systemd[1]: Starting LSB: This service manages the elasticsearch daemon...
Nov 07 11:10:08 nagioslscc2 runuser[15874]: pam_unix(runuser:session): session opened for user nagios by (uid=0)
Nov 07 11:10:08 nagioslscc2 runuser[15874]: pam_unix(runuser:session): session closed for user nagios
Nov 07 11:10:08 nagioslscc2 elasticsearch[15857]: Starting elasticsearch: [ OK ]
Nov 07 11:10:08 nagioslscc2 systemd[1]: Started LSB: This service manages the elasticsearch daemon.
Adding more memory to the system wouldn't help would it? I was thinking we could cap the memory usage for elasticsearch to like 49% or something like that? Not sure if that's even possible.
Thank you.