PHP-FPM Process Managers
Introduction
When looking into issues or configuration of php-fpm, its best to first understand the 3 process managers the php-fpm can use. A lot of the time a customer will be using a pm that is not the right one to use for their current setup. Often you’ll see a server that is running 1 pool using the default dynamic, what may prove problematic as explained below.
Dynamic
Dynamic PM is PHP-FPM’s default setting. Similar to Apache’s prefork MPM. It attempts to spin up and down child processes as needed. Tuning dynamic pm can prove difficult, because web traffic fluctuates with spikes and dips in traffic. If a site is not very busy, resources are wasted by the minimum servers the pm.dynamic creates as default, but if traffic is heavy, php-fpm has to spin up more causing unnecessary overhead doing so.
Using pm you may have see errors similar to: dynamic
WARNING: [pool xxxx] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers),
spawning 32 children, there are 4 idle, and 59 total children
Adjusting the settings below may solve this, but its very likely that it would still result in errors as above due to network traffic ebb and flows.
Configuring pm.dynamic memory usage:
These rules are updated on each individual pool configuration. This is an example of a default pool configuration, notice the pm directives on the middle:
[www]
user = php-fpm
group = php-fpm
listen = 127.0.0.1:9000
listen.allowed_clients = 127.0.0.1
pm = dynamic
pm.max_children = 50
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 35
slowlog = /var/log/php-fpm/www-slow.log
php_admin_value[error_log] = /var/log/php-fpm/www-error.log
php_admin_flag[log_errors] = on
php_value[session.save_handler] = files
php_value[session.save_path] = /var/lib/php/fpm/session
php_value[soap.wsdl_cache_dir] = /var/lib/php/fpm/wsdlcache
Static
The PM static setting spawns php-fpm processes that stay on consistently. If customer has a busy server with a lot of RAM, if might be a good option, as keeping server’s at the ready for connections reduces the overhead in having to spin up and down servers during high traffic events. Another good option for web servers with a several sites hosted would be to have the busiest site set to its own static pool and other smaller pools set to ondemand so the they are available but the resources are allocated for the main high traffic site.
The setting for pm.static will depend heavily on the amount free memory available to the server. You don’t want to spawn too many servers causing the server to get into a out of memory (OOM) situation. As mentioned before, static is really only for servers that have a lot of memory available to it. C onfiguring pm.static’s memory usage
Conceptually, tuning static is pretty easy. We need to figure out how many processes our server can handle at once, taking RAM and CPU usage into consideration. Much like apache’s max children setting, there are a few tools to determine the pm.static setting, but also like apache its more of an art than a science, as the amount of memory needed for php is a moving target. Its best to look at the data from tools like and make a conservative determination on what the max_children setting should be.
Ondemand
Ondemand will only spin up a process when there’s a request for the site. This is good for sites that don’t get a lot of traffic and have little overhead. By splitting up pools to each site, we can use ondemand in conjunction with the other pms, for example, having the busiest site on the server run static, but the other smaller sites running ondemand so we can better use the resources on the server.
Configuring pm.static’s memory usage:
One word of warning is that ondemand is not always the cure for pm.dynamic errors. The common advice is to use pm ondemand, as is the advice in this same support thread . However, that’s even worse, because ondemand will shutdown idle processes right down to 0 when there’s little to no
traffic and then you’ll end up with just as much overhead issues as traffic fluctuates. Unless, of course you set the idle timeout extremely high. In which case you should just be using pm.static + a high pm.max_requests.
Tools for tuning
PHP-FPM Pal
PHP-FPM Pal is a lot like apache2buddy, in that its an automated way of getting a recommendation based on the current environment of php-fpm. htlo ok users can run this script with:
- htlook -p
Or to run manually:
- curl -sL https://raw.githubusercontent.com/pksteyn/php-fpmpal/master/php-fpmpal.sh | bash
Php-fpm pal’s output is pretty straight forward. Its important to remember that the memory usage is a moving target, it will fluxuate with traffic, what part of the application is being access, etc. Take the recommendations and try to be conservitive with what you set the max_children to. For example, If the tuner says max_children should be 200, aim for something like 150 or 175 and evaluate performance.
[root@heel-2020 ~]# curl -sL https://raw.githubusercontent.com/pksteyn/php-fpmpal/master/php-fpmpal.sh |
bash
( ) ( ( ( *
)\ ) ( /( )\ ) )\ ) )\ ) ( ` (
(()/( )\())(()/( (()/( (()/( )\))( ) )\
/(_))((_)\ /(_)) /(_)) /(_))((_)()\ ` ) ( /( ((_)
(_)) _((_)(_)) (_))_|(_)) (_()((_) /(/( )(_)) _
| _ \ | || || _ \ ___ | |_ | _ \ | \/ |((_)_\ ((_)_ | |
| _/ | __ || _/|___|| __| | _/ | |\/| || ‘_ \)/ _` || |
|_| |_||_||_| |_| |_| |_| |_|| .__/ \__,_||_|
========================================= |_| =============
===== List of PHP-FPM pools =====— www —
Configuration file: /etc/php-fpm.d/www.conf
List of processes: 5587 5588 5589 5590 5591
Number of processes: 5
Current max_children value: 50
Total memory usage for pool in KB: 24380
Average memory usage per process in KB: 4876
Total potential memory usage for pool (based on average process) (KB): 243800
Largest process in this pool is (KB): 4876
Total potential memory usage for pool (based on largest process) (KB): 243800
===== Server memory usage statistics =====
Total server memory in KB: 1876292
=Total Apache memory usage in KB: 0
=Total nginx memory usage in KB: 0
=Total Varnish memory usage in KB: 0
=Total MySQL memory usage in KB: 114448.2
=Total PHP-FPM memory usage in KB: 24380
Memory available to assign to PHP-FPM pools in KB: 1107156 (total free memory + PHP-FPM’s current memory
usage)
Total potential PHP-FPM memory usage based on largest processes (KB): 243800 (22.02%) …GOOD
Total potential PHP-FPM memory usage based on average process size (KB): 243800 (22.02%) …GOOD
===== Recommendations per pool =====– www — currently uses 24380 KB memory (100.00% of all PHP-FPM memory usage). It should be allowed to use
about 1107156 KB of all available memory. Its average process size is 4876 KB so this means max_children
should be set to ~227. It is currently set to 50 (this can be changed in /etc/php-fpm.d/www.conf).
===== Other considerations to take into account =====
No other recommendations at this stage.
Note: It is not ideal to run PHP-FPMpal shortly after restarting PHP-FPM or your webservices. This is
because PHP-FPMpal makes recommendations based on the average pool process size, and if PHP-FPM was
restarted a short while ago then the likelihood is high that there won’t have been many requests made to the
sites since the restart, and metrics will be skewed and not show a normalised average.
It is also worth noting that if you’ve recently restarted any services that normally use up a large amount
of memory then you probably want to wait a while before running PHP-FPMpal (e.g. if MySQL normally uses 50%
of memory, but you’ve just restarted it then it may only use 10% of memory right now, thus the
recommendations will be very skewed).
============================================================
Top
Good ol’ top is a decent way of determining the max_children. Since we are concerned with RAM usage we can sort by ‘M’, and we can isolate php fpm with ‘o’ and typing COMMAND=php-fpm
top – 09:43:54 up 309 days, 21:23, 1 user, load average: 0.01, 0.02, 0.05
Tasks: 115 total, 1 running, 114 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.8 us, 0.2 sy, 0.0 ni, 96.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 1876292 total, 337692 free, 488572 used, 1050028 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 1088296 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
5586 root 20 0 259032 3868 756 S 0.0 0.2 13:34.72 php
fpm
5587 php-fpm 20 0 259032 3472 360 S 0.0 0.2 0:00.00 php
fpm
5588 php-fpm 20 0 259032 3472 360 S 0.0 0.2 0:00.00 php
fpm
5589 php-fpm 20 0 259032 3476 360 S 0.0 0.2 0:00.00 php
fpm
5590 php-fpm 20 0 259032 3476 360 S 0.0 0.2 0:00.00 php
fpm
5591 php-fpm 20 0 259032 3480 364 S 0.0 0.2 0:00.00 php-fpm
From this output we can see that according to MEM , each process is taking up 0.2% of memory. This server is configured with a max_children of 50, %MEM so if the current ram usage holds we would use at most ~10% of total ram on this server (0.2 x 50 = 10). If you were to have different values for % , simply take an average (add each
%MEM and divide by the number of processes you added) then multiply that by the max_children.
Tips & Warnings
Related Links
https://haydenjames.io/php-fpm-tuning-using-pm-static-max-performance/