Page 1 of 2
Linux Cached Memory
Posted: Tue May 26, 2020 12:40 pm
by saber
Hi,
We are using the latest "custom_check_mem" and even the "-n" param but no success. It's still show Critical even if all the RAM is in cache.
CentOS 7.8 x86_64 latest version of everything.
Code: Select all
# cat /usr/local/nagios/etc/nrpe/common.cfg
(...)
command[check_mem]=/usr/local/nagios/libexec/custom_check_mem -n $ARG1$
(...)
# /usr/local/nagios/libexec/custom_check_mem -n -w 20 -c 10
CRITICAL - 12432 / 128675 MB (9%) Free Memory, Used: 37987 MB, Shared: 218 MB, Buffers + Cached: 80799 MB | total=128675MB free=12432MB used=37987MB shared=218MB buffers_and_cached=80799MB
Any idea?
Thanks,
Saber
Re: Linux Cached Memory
Posted: Tue May 26, 2020 2:17 pm
by lmiltchev
Can you run the following commands on the remote machine, and show the output?
Code: Select all
free -m
/usr/local/nagios/libexec/custom_check_mem -w 20 -c 10
/usr/local/nagios/libexec/custom_check_mem -n -w 20 -c 10
Also, upload the entire /usr/local/nagios/libexec/custom_check_mem script that you are using on the forum.
Re: Linux Cached Memory
Posted: Tue May 26, 2020 8:25 pm
by saber
Hi,
I appreciate the answer!
Code: Select all
# free -m
total used free shared buff/cache available
Mem: 128675 37092 10767 218 80816 90590
Swap: 8191 0 8191
# /usr/local/nagios/libexec/custom_check_mem -w 20 -c 10
CRITICAL - 10751 / 128675 MB (8%) Free Memory, Used: 37107 MB, Shared: 218 MB, Buffers + Cached: 80815 MB | total=128675MB free=10751MB used=37107MB shared=218MB buffers_and_cached=80815MB
# /usr/local/nagios/libexec/custom_check_mem -n -w 20 -c 10
CRITICAL - 13263 / 128675 MB (10%) Free Memory, Used: 37153 MB, Shared: 218 MB, Buffers + Cached: 80815 MB | total=128675MB free=13263MB used=37153MB shared=218MB buffers_and_cached=80815MB
Re: Linux Cached Memory
Posted: Wed May 27, 2020 10:02 am
by lmiltchev
Hmm, the free memory in your last command is definitely wrong... I wonder if you have some of the programs that the plugin uses installed in a "non-default" location.
Please run the commands below, and show the output.
Code: Select all
which gawk
which dc
which sed
memoutput=`free -m -w | head -2 | tail -1`
free=`echo $memoutput | gawk '{print $4+$7}'`
echo $memoutput
echo $free
Re: Linux Cached Memory
Posted: Wed May 27, 2020 2:31 pm
by saber
Hi,
Thanks for the reply.
Code: Select all
# which gawk
/usr/bin/gawk
# which dc
/usr/bin/dc
# which sed
/usr/bin/sed
# memoutput=`free -m -w | head -2 | tail -1`
# free=`echo $memoutput | gawk '{print $4+$7}'`
# echo $memoutput
Mem: 128675 38105 9736 226 78276 2557 89568
# echo $free
12293
Re: Linux Cached Memory
Posted: Wed May 27, 2020 3:10 pm
by lmiltchev
The "free" memory seems to be correct in the latest output. When you run:
the "-w" flag (wide output) shows buffers and cache memory separated (in different columns). So, when you use the "custom_check_mem" plugin with "-n", only the "cache" memory is added to the "free" memory.
In your example:
# memoutput=`free -m -w | head -2 | tail -1`
# free=`echo $memoutput | gawk '{print $4+$7}'`
# echo $memoutput
Mem: 128675 38105 9736 226 78276 2557 89568
free=`echo $memoutput | gawk '{print $4+$7}'`
free = free + cache = 9736 + 2557 =
12,293
# echo $free
12293
When you run your check:
Code: Select all
/usr/local/nagios/libexec/custom_check_mem -n -w 20 -c 10
the output would be CRITICAL as:
free mem % = (free / total) x 100% = (12293 / 128675) x 100% = 9% (some rounding)
which is less than your critical threshold.
Can you try checking the "wide output" and running the check, verifying the output?
One more thing... the script has a hard coded path to sed (/bin/sed), but your path is /usr/bin/sed. I am not sure if this is causing the issue, but you could modify the plugin by entering the correct path to sed. Let us know if this helped.
Re: Linux Cached Memory
Posted: Wed May 27, 2020 5:18 pm
by saber
Hi,
There is nothing weird on the system and we have this issue on many servers too... Most of them have more than enough memory and this particular one has over 89GB in cached memory.
Sed binary seems fine too:
Code: Select all
# which sed
/usr/bin/sed
# sed --version
sed (GNU sed) 4.2.2
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Jay Fenlason, Tom Lord, Ken Pizzini,
and Paolo Bonzini.
GNU sed home page: <http://www.gnu.org/software/sed/>.
General help using GNU software: <http://www.gnu.org/gethelp/>.
E-mail bug reports to: <[email protected]>.
Be sure to include the word ``sed'' somewhere in the ``Subject:'' field.
# /bin/sed --version
/bin/sed (GNU sed) 4.2.2
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Jay Fenlason, Tom Lord, Ken Pizzini,
and Paolo Bonzini.
GNU sed home page: <http://www.gnu.org/software/sed/>.
General help using GNU software: <http://www.gnu.org/gethelp/>.
E-mail bug reports to: <[email protected]>.
Be sure to include the word ``sed'' somewhere in the ``Subject:'' field.
Re: Linux Cached Memory
Posted: Thu May 28, 2020 9:04 am
by lmiltchev
OK, let's try one more time to view the resources, and run the check immediately after this. Please show the output of the command below.
Code: Select all
free -m -w;echo "";/usr/local/nagios/libexec/custom_check_mem -n -w 20 -c 10
If numbers don't make sense, we would have to schedule a remote session to further troubleshoot the issue.
Re: Linux Cached Memory
Posted: Thu May 28, 2020 1:10 pm
by saber
Hi,
Here is the output:
Code: Select all
# free -m -w;echo "";/usr/local/nagios/libexec/custom_check_mem -n -w 20 -c 10
total used free shared buffers cache available
Mem: 128675 37392 9998 234 78707 2577 90273
Swap: 8191 0 8191
CRITICAL - 12573 / 128675 MB (9%) Free Memory, Used: 37394 MB, Shared: 234 MB, Buffers + Cached: 81284 MB | total=128675MB free=12573MB used=37394MB shared=234MB buffers_and_cached=81284MB
Re: Linux Cached Memory
Posted: Thu May 28, 2020 1:26 pm
by lmiltchev
This looks correct.
example-01.jpg
Your free memory + cache memory is 12573 (or pretty close to that).
free + cache = 9998 + 2577 =
12575
(free / total) x 100% = (12575 / 128675) x 100% =
9% (rounded)
9% < 10% (your critical threshold), that's why your status is CRITICAL.
You could adjust your warning and critical thresholds in order to eliminate unnecessary notifications.