Nagios Support Forum

Support for Nagios products and services
https://support.nagios.com/forum/

constant increase in memory usage due to PHP-FPM usage

https://support.nagios.com/forum/viewtopic.php?t=62249

Page 1 of 1

constant increase in memory usage due to PHP-FPM usage

Posted: Mon Apr 12, 2021 6:38 am
by jweijters
Hi,
I Split this from https://support.nagios.com/forum/viewto ... 10#p328684


what we see is that at a certain moment, there are like 3 or 4 php-fpm processes using > 15% memory this is approx 2.5GB.
When we restart the apache and php-fpm, they will just come back, even when we just reboot the server, these memory consuming processes come back within secconds.
Only when we remove all the session files which are in /var/lib/php/sessions, and reboot the server these large consuming processes will not come back, at least for a while.
In these session files we can see which user is using which file.
however when we look in debug the php-fpm processes, we see that each process is using multiple session files

can we from the session file find out what happens so it is filling up the memory in the php-fpm processes?

for instance:

Code: Select all

 $ ps aux |grep -i php-fpm
    root       1390  0.0  0.0 525328  1296 ?        Ss   Apr01   0:50 php-fpm: master process (/etc/php-fpm.conf)
    apache     2153  1.0  0.3 560824 51356 ?        S    09:03   0:19 php-fpm: pool www
    apache     2347  3.2 15.7 3065216 2555376 ?     S    09:03   1:01 php-fpm: pool www
    apache     4404  1.0  0.3 646744 55392 ?        S    09:05   0:19 php-fpm: pool www
    apache     8348  1.1  0.3 558328 49660 ?        S    09:06   0:20 php-fpm: pool www
    apache     8350  1.2  0.3 558020 49040 ?        S    09:06   0:22 php-fpm: pool www
    apache     8352  1.2  0.2 556964 46960 ?        S    09:06   0:21 php-fpm: pool www
    apache     8358  1.0  0.3 559276 49640 ?        S    09:06   0:17 php-fpm: pool www
    apache    13924  3.4 15.7 3070032 2560148 ?     S    09:10   0:51 php-fpm: pool www
    apache    15127  0.9  0.2 551312 42604 ?        S    09:11   0:14 php-fpm: pool www
    apache    23926  1.0  0.3 566868 56384 ?        S    09:18   0:10 php-fpm: pool www
    apache    23930  0.9  0.3 559108 49276 ?        S    09:18   0:09 php-fpm: pool www
    root      44581  0.0  0.0 112816   984 pts/1    S+   09:34   0:00 grep --color=auto -i php-fpm
    apache   244490  1.3  0.3 562116 52760 ?        S    09:01   0:28 php-fpm: pool www
    apache   244540  1.1  0.3 566596 58064 ?        S    09:01   0:23 php-fpm: pool www
    apache   245616  3.0 15.7 3072524 2562648 ?     S    09:02   0:59 php-fpm: pool www
    /var/lib/php/session]$ ls -l
    total 6652
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_079foaqrupam4u593p32nnhruc
    -rw------- 1 apache apache   1139 Apr  7 12:12 sess_1uevltbl5u4ofu6q872d8489e1
    -rw------- 1 apache apache     21 Apr  7 12:04 sess_298avq2du4se1pj77s7b6kd4gu
    -rw------- 1 apache apache 287684 Apr  7 12:12 sess_3bpi4fpt4gc3mdnt5eurcji6eo
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_3l4k7kevtv6n2ld6uhl383dpv1
    -rw------- 1 apache apache     21 Apr  7 12:03 sess_3lvfpam5bom12ne1ro660jsqpp
    -rw------- 1 apache apache 167849 Apr  7 12:12 sess_5291l38gcuosluj3blbhtbpe7c
    -rw------- 1 apache apache 144904 Apr  7 12:11 sess_5d5fs7k9p7rtt4psh86bnp1lj0
    -rw------- 1 apache apache    677 Apr  7 12:12 sess_5q5n2ujv56hesbe0hef16i2ids
    -rw------- 1 apache apache     21 Apr  7 10:47 sess_714sgqudkm6ui1qivjuk46ofuq
    -rw------- 1 apache apache  47802 Apr  7 12:12 sess_7he7b57iavqtk58gv73fbk9f9o
    -rw------- 1 apache apache  47802 Apr  7 12:12 sess_85hfogs646htgrhp630nokj2d6
    -rw------- 1 apache apache   2707 Apr  7 12:12 sess_8osjssj3nvsre9uqq4r1hbknhm
    -rw------- 1 apache apache    859 Apr  7 11:27 sess_96vbhlis5bls6vu8r72qrj3qb0
    -rw------- 1 apache apache    860 Apr  7 11:10 sess_99p24ego5rekeoqpdi8i910723
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_ahsp3cht0vuidd1lp7i0o0ehbb
    -rw------- 1 apache apache 475346 Apr  7 12:12 sess_bftialsdql0nr8ikoicm5afb4c
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_bt7q337djv7vviqtlfa55vaj4n
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_c3rpqpm5t2stjnsrm4uj3e6vu5
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_c8sl67dcbcvfvvcd3u3vtihfiv
    -rw------- 1 apache apache 551462 Apr  7 12:12 sess_cucvuipojbeiei2s125k8sooko
    -rw------- 1 apache apache 214684 Apr  7 12:12 sess_elqgsk710rab2jt1lv1v8cefre
    -rw------- 1 apache apache 466851 Apr  7 12:09 sess_erk1di0l3b219jofabslm54i13
    -rw------- 1 apache apache 430204 Apr  7 12:11 sess_eutt1u2huh0l6vc9hcpiksj5cp
    -rw------- 1 apache apache     21 Apr  7 10:49 sess_hssur9tln1ng2lsmcfpbtqsr41
    -rw------- 1 apache apache 466886 Apr  7 12:12 sess_inco5fj4ous25p9gcst48doeve
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_j5tolfcnpj9t0uppk16247cpf0
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_klnd0ghfa6l9ekna2v0d1plbit
    -rw------- 1 apache apache    860 Apr  7 11:26 sess_ko9ls4u5d0811iptkuot5rq1ep
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_mbcq2hs32gb5ul8ive2jdf5eua
    -rw------- 1 apache apache 316398 Apr  7 11:35 sess_mg7cootti307c7fk4spjm5all6
    -rw------- 1 apache apache 120262 Apr  7 12:12 sess_n30hc7bhfpice8ao28cq43rn5v
    -rw------- 1 apache apache 466800 Apr  7 10:45 sess_n83bfhp631b40g0u233tth57hj
    -rw------- 1 apache apache     21 Apr  7 10:50 sess_n9aa2jjr5qe711ntop0539sppb
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_n9q04297tgbohf57mq59in6l65
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_ncf18f7v70e0se87mbkebkqli3
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_nh4g3vco171ll5nssscorapcbs
    -rw------- 1 apache apache    862 Apr  7 12:12 sess_nii4tlebj79n0mgensmv1ib3v1
    -rw------- 1 apache apache 167520 Apr  7 11:47 sess_nte4if26d2pqk4aglbcunjgkda
    -rw------- 1 apache apache    856 Apr  7 12:12 sess_o1cbj4edjicf6rt4hnjeur4q2s
    -rw------- 1 apache apache 427699 Apr  7 12:12 sess_pades6dn567ein1a24dsnsufj3
    -rw------- 1 apache apache 174563 Apr  7 12:03 sess_pk58csi5n4home20k6h6v294b7
    -rw------- 1 apache apache     21 Apr  7 10:43 sess_q5mlpuu9oajitrs013os40hd6g
    -rw------- 1 apache apache    893 Apr  7 12:12 sess_q8qs7q5c6mib0iiv77uo501jot
    -rw------- 1 apache apache 519213 Apr  7 12:12 sess_r5d81fq4pjlfvrch6601ulbug2
    -rw------- 1 apache apache 242287 Apr  7 12:12 sess_u02q0s67kqsn64t29tbirulk00
    -rw------- 1 apache apache    903 Apr  7 12:12 sess_ucr0m6bv7aeh7s66349sin127i
    -rw------- 1 apache apache 702086 Apr  7 12:12 sess_ujv9d51u12qn2pktragpgbf7hg
    -rw------- 1 apache apache     21 Apr  7 12:12 sess_vplj08esmjm9tg8se474v5da5j
    -rw------- 1 apache apache 213008 Apr  7 12:12 sess_vumsnb62rk8v9serjgbqohl2ma

    $ less sess_n83bfhp631b40g0u233tth57hj
    language|s:5:"en_US";user_id|s:3:"270";username|s:3:"tvn";session_id|i:1358425...... < AND MUCH more data >

Re: constant increase in memory usage due to PHP-FPM usage

Posted: Mon Apr 12, 2021 5:16 pm
by ssax
You can try editing your /etc/php-fpm.d/www.conf and try changing this:

Code: Select all

;pm.max_requests = 500
To this:

Code: Select all

pm.max_requests = 50
Then restart httpd/php-fpm:

Code: Select all

systemctl restart httpd php-fpm
Then monitor it and see if that alleviates it.

If that doesn't resolve it, you can try doing this to see if it helps:

https://www.tecmint.com/improve-php-fpm-performance/

Re: constant increase in memory usage due to PHP-FPM usage

Posted: Wed Apr 21, 2021 1:16 am
by jweijters
Hi,

I reduced the amount of pm.max_requests to 50, however this is not successful.

apache 43079 31.5 15.7 3078492 2551220 ? S 08:13 0:29 php-fpm: pool www

what nagios query/page/module is using so much memory?

kind regards,

Joris Weijters

Re: constant increase in memory usage due to PHP-FPM usage

Posted: Wed Apr 21, 2021 9:36 am
by dchurch
PHP-fpm is designed to spawn a few PHP processes that live a while, handle multiple requests, and then are reaped (or aren't) when a certain amount of idle time has passed or a certain number of requests has been processed.

Basically this means any one request isn't responsible for causing the memory ballooning. It could be anything, really. I'd only really be concerned if it's still happening.

Further recommendations

PHP is ballooning out to ~250MB per process, and your PHP-FPM configuration is either set to spawn too many of them, or they're not being reaped. My idea is to change how the children are created, and reap them more effectively.

Edit under the [www] heading in /etc/php-fpm.d/www.conf (replace and/or uncomment values rather than adding them):

Code: Select all

[www]
# Now directives under this apply to the "www" pool, same as your screenshot
# ...
pm = ondemand
pm.max_children = 5
pm.process_idle_timeout = 10s
pm.max_requests = 50
Then restart httpd/php-fpm:

Code: Select all

systemctl restart httpd php-fpm
All times are UTC-05:00
Page 1 of 1

Powered by phpBB® Forum Software © phpBB Limited