Nginx and PHP-FPM Configuration and Optimizing Tips and Tricks
I wrote before a guide Howto install Nginx/PHP-FPM on Fedora 18/17, CentOS/RHEL 6.3/5.9, but this guide is just installation guide and many cases Nginx and PHP-FPM basic configuration is good enough, but if you want to squeeze all the juice out of your VPS or web server / servers and do your maintenance work little bit easier, then this guide might be useful. These tips are based entirely on my own experience, so they may not be an absolute truth, and in some situations, a completely different configuration may work better. It’s also good to remember leave resources for another services also if you run example, MySQL, PostgreSQL, MongoDB, Mail server, Name server and/or SSH server on same machine.
And yes here we go…
Nginx Configuration and Optimizing Tips and Tricks
Nginx Tip 1. – Organize Nginx Configuration Files
Normally Nginx configuration files are located under /etc/nginx path.
One good way to organize configuration files is use Debian/Ubuntu Apache style setup:
## Main configuration file ## /etc/nginx/nginx.conf ## Virtualhost configuration files on ## /etc/nginx/sites-available/ /etc/nginx/sites-enabled/ ## Other config files on (if needed) ## /etc/nginx/conf.d/ |
Virtualhost files have 2 paths, because sites-available directory can contain any stuff, like test configs, just copied/created configs, old configs and so on. And sites-enabled contains only really enabled configurations, actually just only symbolic links to sites-available directory.
Remember add following includes at the end of your nginx.conf file:
## Load virtual host conf files. ## include /etc/nginx/sites-enabled/*; ## Load another configs from conf.d/ ## include /etc/nginx/conf.d/*; |
Nginx Tip 2. – Determine Nginx worker_processes and worker_connections
Default setup is okay for worker_processes and worker_connections, but these values could be little bit optimized:
max_clients = worker_processes * worker_connections
Just Nginx basic setup can handle hundreds of concurrent connection:
worker_processes 1; worker_connections 1024; |
Normally 1000 concurrent connection / per one server is good, but sometimes other parts like disks on server might be slow, and it causes that the Nginx is locked on I/O operations. To avoid locking use example following setup: one worker_precess / per processor core, like:
Worker Processes
worker_processes [number of processor cores]; |
To check how many processor cores do you have, run following command:
cat /proc/cpuinfo |grep processor processor : 0 processor : 1 processor : 2 processor : 3 |
So here is 4 cores and worker_processes final setup could be following:
worker_processes 4; |
Worker Connections
Personally I stick with 1024 worker connections, because I don’t have any reason to raise this value. But if example 4096 connections per second is not enough then it’s possible to try to double this and set 2048 connections per process.
worker_processes final setup could be following:
worker_connections 1024; |
I have seen some configurations where server admins are used too much Apache and think if I set Nginx worker_processes to 50 and worker_connections to 20000 then my server could handle all traffic once what we get monthly…but yes it’s not true. It’s just wasting of resources and might cause some serious problems…
Nginx Tip 3. – Hide Nginx Server Tokens / Hide Nginx version number
This is good for security reasons hide server tokens / hide Nginx version number, especially, if run some outdated version of Nginx. This is very easy to do just set server_tokens off under http/server/location section, like:
server_tokens off; |
Nginx Tip 4. – Nginx Request / Upload Max Body Size (client_max_body_size)
If you want to allow users upload something or upload personally something over the HTTP then you should maybe increase post size. It can be done with client_max_body_size value which goes under http/server/location section. On default it’s 1 Mb, but it can be set example to 20 Mb and also increase buffer size with following configuration:
client_max_body_size 20m; client_body_buffer_size 128k; |
If you get following error, then you know that client_max_body_size is too low:
“Request Entity Too Large” (413)
Nginx Tip 5. – Nginx Cache Control for Static Files (Browser Cache Control Directives)
Browser caching is import if you want save resources and bandwith. It’s easy setup with Nginx, following is very basic setup where logging (access log and not found log) is turned off and expires headers are set to 360 days.
location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml)$ { access_log off; log_not_found off; expires 360d; } |
If you want more complicated headers or some other expiration by filetypes then you could configure those separately.
Nginx Tip 6. – Nginx Pass PHP requests to PHP-FPM
Here you could use default tpc/ip stack or use directly Unix socket connection. You have to also setup PHP-FPM listen exactly same ip:port or unix socket (with Unix socket also socket permission have to be right). Default setup is use ip:port (127.0.0.1:9000) you could of course change ips and ports what PHP-FPM listens. Here is very basic configuration with Unix socket example commented out:
# Pass PHP scripts to PHP-FPM location ~* \.php$ { fastcgi_index index.php; fastcgi_pass 127.0.0.1:9000; #fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock; include fastcgi_params; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param SCRIPT_NAME $fastcgi_script_name; } |
It’s also possible to run PHP-FPM another server and Nginx another.
Nginx Tip 7. – Prevent (deny) Access to Hidden Files with Nginx
It’s very common that server root or other public directories have hidden files, which starts with dot (.) and normally those is not intended to site users. Public directories can contain version control files and directories, like .svn, some IDE properties files and .htaccess files. Following deny access and turn off logging for all hidden files.
location ~ /\. { access_log off; log_not_found off; deny all; } |
PHP-FPM Configuration Tips and Tricks
PHP-FPM Tip 1. – PHP-FPM Configuration files
Normally PHP-FPM configuration files are located on /etc/php-fpm.conf file and /etc/php-fpm.d path. This is normally excellent start and all pool configs goes to /etc/php-fpm.d directory. You need to add following include line on your php-fpm.conf file:
include=/etc/php-fpm.d/*.conf |
PHP-FPM Tip 2. – PHP-FPM Global Configuration Tweaks
Set up emergency_restart_threshold, emergency_restart_interval and process_control_timeout. Default values for these options are totally off, but I think it’s better use these options example like following:
emergency_restart_threshold 10
emergency_restart_interval 1m
process_control_timeout 10s |
What this mean? So if 10 PHP-FPM child processes exit with SIGSEGV or SIGBUS within 1 minute then PHP-FPM restart automatically. This configuration also sets 10 seconds time limit for child processes to wait for a reaction on signals from master.
PHP-FPM Tip 3. – PHP-FPM Pools Configuration
With PHP-FPM it’s possible to use different pools for different sites and allocate resources very accurately and even use different users and groups for every pool. Following is just example configuration files structure for PHP-FPM pools for three different sites (or actually three different part of same site):
/etc/php-fpm.d/site.conf /etc/php-fpm.d/blog.conf /etc/php-fpm.d/forums.conf
Just example configurations for every pool:
/etc/php-fpm.d/site.conf
[site] listen = 127.0.0.1:9000 user = site group = site request_slowlog_timeout = 5s slowlog = /var/log/php-fpm/slowlog-site.log listen.allowed_clients = 127.0.0.1 pm = dynamic pm.max_children = 5 pm.start_servers = 3 pm.min_spare_servers = 2 pm.max_spare_servers = 4 pm.max_requests = 200 listen.backlog = -1 pm.status_path = /status request_terminate_timeout = 120s rlimit_files = 131072 rlimit_core = unlimited catch_workers_output = yes env[HOSTNAME] = $HOSTNAME env[TMP] = /tmp env[TMPDIR] = /tmp env[TEMP] = /tmp |
/etc/php-fpm.d/blog.conf
[blog] listen = 127.0.0.1:9001 user = blog group = blog request_slowlog_timeout = 5s slowlog = /var/log/php-fpm/slowlog-blog.log listen.allowed_clients = 127.0.0.1 pm = dynamic pm.max_children = 4 pm.start_servers = 2 pm.min_spare_servers = 1 pm.max_spare_servers = 3 pm.max_requests = 200 listen.backlog = -1 pm.status_path = /status request_terminate_timeout = 120s rlimit_files = 131072 rlimit_core = unlimited catch_workers_output = yes env[HOSTNAME] = $HOSTNAME env[TMP] = /tmp env[TMPDIR] = /tmp env[TEMP] = /tmp |
/etc/php-fpm.d/forums.conf
[forums] listen = 127.0.0.1:9002 user = forums group = forums request_slowlog_timeout = 5s slowlog = /var/log/php-fpm/slowlog-forums.log listen.allowed_clients = 127.0.0.1 pm = dynamic pm.max_children = 10 pm.start_servers = 3 pm.min_spare_servers = 2 pm.max_spare_servers = 4 pm.max_requests = 400 listen.backlog = -1 pm.status_path = /status request_terminate_timeout = 120s rlimit_files = 131072 rlimit_core = unlimited catch_workers_output = yes env[HOSTNAME] = $HOSTNAME env[TMP] = /tmp env[TMPDIR] = /tmp env[TEMP] = /tmp |
So this is just example howto configure multiple different size pools.
PHP-FPM Tip 4. – PHP-FPM Pool Process Manager (pm) Configuration
Best way to use PHP-FPM process manager is use dynamic process management, so PHP-FPM processes are started only when needed. This is almost same style setup than Nginx worker_processes and worker_connections setup. So very high values does not mean necessarily anything good. Every process eat memory and of course if site have very high traffic and server lot’s of memory then higher values are right choise, but servers, like VPS (Virtual Private Servers) memory is normally limited to 256 Mb, 512 Mb, 1024 Mb. This low RAM is enough to handle even very high traffic (even dozens of requests per second), if it’s used wisely.
It’s good to test how many PHP-FPM processes a server could handle easily, first start Nginx and PHP-FPM and load some PHP pages, preferably all of the heaviest pages. Then check memory usage per PHP-FPM process example with Linux top or htop command. Let’s assume that the server has 512 Mb memory and 220 Mb could be used for PHP-FPM, every process use 24 Mb RAM (some huge content management system with plugins can easily use 20-40 Mb / per PHP page request or even more). Then simply calculate the server max_children value:
220 / 24 = 9.17
So good pm.max_children value is 9. This is based just quick average and later this could be something else when you see longer time memory usage / per process. After quick testing it’s much easier to setup pm.start_servers value, pm.min_spare_servers value and pm.max_spare_servers value.
Final example configuration could be following:
pm.max_children = 9 pm.start_servers = 3 pm.min_spare_servers = 2 pm.max_spare_servers = 4 pm.max_requests = 200 |
Max request per process is unlimited by default, but it’s good to set some low value, like 200 and avoid some memory issues. This style setup could handle large amount of requests, even if the numbers seems to be small.
Do you have problems or some nice Nginx and PHP-FPM tips and trick?
Please feel free to post your questions and tips and tricks here.
93 Comments
Leave a Comment
Trackbacks/Pingbacks
- Install Nginx/PHP-FPM on Fedora 16/15, CentOS/RHEL 6/5.7 - [...] This is just very simple basic configuration, but if you want configure and optimize Nginx and PHP-FPM then check ...
- Ideanub – Configuring Multiple Symfony Websites on Nginx with PHP-FPM - [...] hurt to make a few small changes in the config files to improve the performance. Thanks to JR’s post ...
I found my way here googling to get to your VirtualBox post (also well done). I’ve been using php-fpm and nginx for some years now and I do some of these things but some are new.
Very nicely done and well-presented. I just wanted to say thanks.
Thanks Chris!
I have also used Nginx and PHP-FPM (earlier FastCGI) setup long time on my testing environments, now I’m gradually transferring all production web servers to use Nginx and PHP-FPM instead Apache and PHP. Nowadays Nginx and PHP-FPM is excellent choice for everyone who wants handle even massive traffic with good and cheap Virtual Private Servers (VPS). Some cases configuration is even easier than Apache configuration.
Very nice to hear that you found even something new on this guide and later I will add more tips, when I have some good ideas. ;)
Do I need separate linux users (useradd …) for this part:
user = site
group = site
?
Great tutorial, thanks! I have already setup a couple of things according to your post.
Hi Nader and thanks!
This is just example to show how powerful PHP-FPM Pools are, so you could of course use existing users and groups, example nginx user and nginx group or apache user and apache group.
This example use totally separate Linux users and groups, because i.e. those pools users / groups access could be limited just their own directories. This is much more secure way if you have multiple totally separated sites. One site security issue does not affect whole system and other sites, because all sites are running by different users with very limited permissions.
But any user/group defined in a PHP-FPM Pool must exist as a user account in Linux?
Hi Thomas,
Yes, user and group must exist on Linux system.
Which files must then be owned by this specific user defined in the pool?
Only the pool-configuration-file?
Or all files of the relevant service in /var/www (or /usr/share/nginx/www), e.g. ownclowd?
In other words: which files will be accessed by the user account defined in the pool-configuration?
THX
Hi Thomas,
All files of the service (like /var/www/example1) must be owned by specific user. Configuration files are accessed/used only by PHP-FPM process.
Hi, great tutorial, I really love the setup here. I have one question tho, and it is also regarding users and security.
On this setup you have different users: site, forum, blog. These will be users with limited capabilities, with an eye on security.
Now my question is, how can you manage this in a real system. How would you go about on editing the files for the different users. If you have a user that you use to login, let’s says www-user, how can that user edit the files that are owned by users: site, forum, blog.
If you use a remote program that can not sudo such as winscp on windows then you are pretty much stuck I guess. Or is there a way around this?
I assume here that the users site, forum, blog do not have login permission.
Hi Saif Bechan nice to see you here :)
Here, these users can be, for example two types. Users who have access only to the own directories without login permission or users who have the login permission only to the “chrooted system”, like virtual hosting setups.
If you want modify all those sites with www user then you something like following setup (this, is just theory / example):
1. Create users blog, site, forums with nologin
2. Create www user with login permission
3. Add www to blog, site and forums groups
4. Directory structure could look following:
If I run multiple pools with different users, then I normally use root user to modify files. This is theoretical example, which I did on /tmp directory and permissions should work nicely… :)
I’ve been using a similar separate user per site setup for each of my python based sites and found your article whilst looking for a similar way to do this with php, thanks.
One nice trick is to use setgid on your site directories. If you use setgid on a directory all files and subdirectories inherit its group. So, my site specific directories are all owned by the www user with which I log in (meaning I can edit files without sudo) but setgid ensures all files have a group which is site specific.
mkdir site
# set owner to www & group to site
sudo chown www:site site
# make sure the group can write to the directory
sudo chmod g+w site
# setgid on the site directory
sudo chmod g+s site
It also helps to set your umask to 002 in your ~/.profile so that files you create group writable by default.
I’d be interested to hear your thoughts on this setup.
Great post! Don’t forget to enable APC and memcache if you really want to get top speeds. You can also set a reverse proxy in nginx for PHP requests (which means nginx caching PHP requests if possible):
Inside location:
fastcgi_cache nginxcache;
fastcgi_cache_valid any 5m;
Inside http
fastcgi_cache_path /tmp/nginx_cache levels=1:2 keys_zone=nginxcache:50m inactive=1d max_size=400m;
fastcgi_cache_key “$scheme$request_method$host$request_uri”;
And by the way, maybe you should look into honeypot technique for preventing spam submissions :)
Hi Ashraf,
Thanks for nice Nginx reverse proxy tip :) If you have time, you can write more detailed guide, with some full config on If !1 0 forums. I add link with description to your guide of course. :)
I am thinking of writing own instructions about APC, because it works of course also with other HTTP servers example with Apache. It’s not just Nginx and PHP-FPM related stuff. And I already have Memcache guide… :)
I actually have honeypot on forums section, but current technique is just totally zero spam technique, but yes I know, it might be irritating…maybe I should change it… :D
Hello,
I’m trying to run with nginxcp, how to create single account don’t want to use nginx
thanks
Hi Dedi,
My apologies, but I do not quite understand your question? Could you try to explain more specific what you are trying to do and what is the problem?
Great article, thanks for putting all these tips together!
One thing that’s not clear to me is how to direct a PHP application to actually use the PHP-FPM pools you’ve provided for it?
Hi DeltaHF,
If I understood your question correctly…
When you create PHP-FPM pools, let’s say you have 3 pools pool1, pool2, pool3 which listens 127.0.0.1:9000, 127.0.0.1:9001, 127.0.0.1:9002:
1. Create virtual host where you pass PHP requests to pool1 (127.0.0.1:9000)
2. Create virtual host where you pass PHP requests to pool2 (127.0.0.1:9001)
3. Create virtual host where you pass PHP requests to pool3 (127.0.0.1:9002)
4. Create virtual host where you pass PHP requests again to pool2 (127.0.0.1:9001)
5. …
Did this answer your question? :)
Thanks, JR! Yes, I was overlooking the address:port number defined by fastcgi_pass in the Nginx configuration file and I think that disconnect was confusing me.
I have a blog at / and a message board at /forum on my site. So, if I want the forum to run in its own PHP-FPM pool, would I define a unique fastcgi_pass in a location block like so (assuming the pool is already properly defined, of course)?
location /forum {
fastcgi_pass 127.0.0.1:9001;
..
}
Would this correctly override the “default” port defined in my general “location ~ \.php$ {…}” block as I would expect?
You are welcome! :)
Yes, you could use this almost that, but you can’t override any default ports, so you have use little bit different setup…example something like following.
Separated php-fastcgi.conf file:
This is just idea how it could works (not working example)…normally I use different PHP-FPM pools only with different virtual hosts.
Please let me know if you need more help, or did you get idea? :)
Hi JR,
I try to use fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
but when I uncomment it in the browser I get 502 Bad Gateway.
What should I do?
Thank you!
Elaidon
Hi Elaidon,
How is your PHP-FPM pool configured what you try to use?
Hi, Thanks for your blog post it is very informative.
I just want to ask one question. How can I set a timeout for the php processes? sometimes one of the php-fpm child is using too much cpu which makes my server unresponsive. Right now I am restarting php-fpm every hour using cronjobs. But is there a setting in nginx or php-fpm that can solve my problem?
Hi Gray,
You can use PHP-FPM pool option pm.max_requests to limit max request per process.
Default value is 0 (unlimited), but if you setup it example some value between 50-500 then process stops when it’s handled example 200 requests.
I was having some problems with my WHMCS on a Fedora NGINX/PHP-FPM installation (based off your instructions). Turns out the PHP sessions directory /var/lib/php/session was owned by “apache” group (assuming an apache install). I had to chgrp apache /var/lib/php/session to get stuff working properly.
Might be worth mentioning somewhere that the php-fpm package-related perms have to be updated.
Other than that, great articles and tips. Can’t wait for them to be updated for Fedora 17! ;)
Hi N,
Thanks for this info! :)
Yes I start test and updating all guides for Fedora 17 when Beta/Final is released… ;)
Thanks for your tutorial. Again its very useful especially nginx tip #5 and php-fpm tip #2
All my websites are using nginx + php-fpm + mysql so I can apply your tips on my servers.
:)
Hi, Thanks for your guidelines. It is helpful.
Can you explain about the PHP-FPM Pools in more details..
PHP-FPM Tip 3. – PHP-FPM Pools Configuration
I followed your guidelines to create my second virtual host.
I name my 2nd host as forums & using different PHP-FPM pools, name forums
my conf files should be look like this?
Create the files from the /etc/nginx/sites-available/forums and under the server { } I put in,
location ~ \.php$ {
include /etc/nginx/fastcgi_params;
fastcgi_pass 127.0.0.1:9002;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /var/www/forums/public_html$fastcgi_script_name;
}
AM i right? is this configuration able to use the 2nd PHP-FPM pools?
Hi woody,
Your configuration looks perfect! :D
Your PHP-FPM pool listen localhost port 9002 and then you pass requests to that port…yes you are right and it should use the 2nd PHP-FPM pools. :)
Thank you man. Successfully installed nginx. But php_cgi :) Does the difference?
Hi Ahmet,
You are welcome! :)
Hi JR, Thanks :)
I need your help to clarify the code below, kinda confuse about the code below,
location / {
index index.html index.htm index.php;
}
location ~ \.php$ {
include /etc/nginx/fastcgi_params;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /srv/www/testsite.local/public_html$fastcgi_script_name;
}
The location / and location ~ \.php$
What is that actually means?
Besides that, how do we know, the php-fpm is running on every request from php?
Is there any tools to test?
Need your advice. :) really appreciate it ~
You are welcome! :)
Actually Nginx Wiki explains this very clearly. ;)
Hi JR ,
Thanks for the tips , i am going to try and implement these settings now on our server .
With the above mentioned settings and below mentioned hardware configs : what kind of load can the server take : approximately how many visitors a minute or per day :
CentOS 5.7
NGINX
8 Gb RAM ( my full ram keeps running out of memory with just 1000 visitors)
We went live for an hour this morning and we kept crashing with the following error:
Too many connections to php-fpm. Server is reaching pm.max_children value.
Then our server admins did a hard reboot on the server , and our drive started getting mounted on read-only mode.
And then we were forced to run a FSCK ! We are down 8 hours and counting right now.
Wish we could hire you as consultant to setup our system and let our admins maintain it time to time. If you are up for it , please hit us up on the email mentioned above.
Hi Karan,
Do you run just Nginx and PHP-FPM on that server?
I think that 8 Gb RAM with good network connection and processor(s) should be enough to handle very huge amount of traffic.
Could you post output of following commands from your server (example to pastebin.com):
Let’s check first if I can help you just with seeing this info… ;)
Here’s the reply JR : http://pastebin.com/7z7KGZ8X
we could copy most of it except php-i
Please let us know what tweaks are needed on NGINX and PHP-fpm
Thanks Karan,
Your server looks very good. :)
Actually Nginx configuration looks good to me (4 worker processes should be enough, but 8 is also okay with your server) and PHP-FPM need some configuration.
Let’s do some calculation…if you have 8 Gb memory and let’s use now 4 Gb for PHP-FPM. This is a rough estimate, but let’s say that your PHP-FPM processes use 66 Mb RAM (is not so much really I think) and you have 4 Gb RAM to use.
4096 / 66 = 62.06
Based on this theory, it should be safe to set:
And also enable following settings for PHP-FPM:
emergency_restart_threshold 10 emergency_restart_interval 1m process_control_timeout 10sI recommend you to backup your config before doing anything, if something goes wrong then you can very easy switch back to your original config… :) After config change remember restart PHP-FPM… :)
You can monitor your php processes with top, htop or even with following command:
HI JR ,
We noticed that our PHP-FPM processes is burning through at 300Mb+ each : here are the screenshots :
http://grab.by/cUnc
http://grab.by/cUng
as u can see we are running out of 8 GB ram pretty quick…not sure whats causing this issue
please let us know what do here – we are hosting an e-commerce shopping cart based on Magento.
Let us know.
Hi again Karan,
Thank you for this information.
Actually VIRT is not right column to follow in this case. VIRT column represents how much memory the program is able to access at the present moment. All memory usage combined memory it has mapped into itself, files on disk that have been mapped into it and memory shared with other processes.
RES stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming. (and also %MEM column value is calculated directly from RES).
So RES is right / much better column to follow. Memory usage per process is therefore around 75-90 Mb (~1 percent of 8 Gb). Let’s calculate again:
4096 / 90 = 45.51
Based on this information I recommend to try following values then:
And
emergency_restart_threshold 10 emergency_restart_interval 1m process_control_timeout 10sYou should definitely adjust these values, when you see the real situation.
However, the problem is indeed Magento, rather than the configuration of the server. Magento is just too large and too heavy to any environment. Magento hosting is really tedious and unpleasant task.
Please let me know if these values work, or if you have problems?
Thanks JR ,
Seems some improvements.
Please see below :
Cpu(s): 11.6%us, 3.6%sy, 0.0%ni, 84.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 8163952k total, 3282776k used, 4881176k free, 375324k buffers
Swap: 4192956k total, 0k used, 4192956k free, 1316412k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27992 apache 16 0 340m 80m 41m R 40.3 1.0 0:07.54 php-fpm
27985 apache 15 0 334m 73m 40m S 16.0 0.9 0:14.53 php-fpm
27984 apache 16 0 338m 77m 40m S 2.7 1.0 0:04.81 php-fpm
5114 mysql 15 0 1534m 708m 5996 S 2.0 8.9 226:07.99 mysqld
27991 root 15 0 12760 1176 828 R 0.3 0.0 0:00.32 top
1 root 15 0 10320 688 580 S 0.0 0.0 0:00.68 init
2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/0
Few days back my total memory went right upto 7.5 GB and there was only a free memory of 500-700 MB
Then i typed this command to clear cache
sudo sync
sudo echo 3 | sudo tee /proc/sys/vm/drop_caches and i had 6 GB free. Do you know how do i limit this cache. It seems to be eating up all the memory on the server, if i dont regularly flush it.
Do you think this heavy caching could be caused by varnish cache / APC cache etc?? is there anyway i can limit this.
You are welcome!
Actually ~3.2 Gb used from 8 Gb looks very low, normally Linux use ~90 % all available memory. So that 7.5 Gb / 8 Gb is much more normal and better situation.
Do you have Varnish running and PHP APC installed on this server?
Hi JR,
Varnish Cache was running, we disabled it after the server crashed thinking we needed to save some memory or the cache was being put on the memory and thus crashing the server. APC is enabled on this server.
Few mins back we were again not able to access the server. I guess my server guys were running a stress test on the server. They gave me back with these details.
Total transferred: 2976410 bytes
HTML transferred: 240762 bytes
Requests per second: 96.36 [#/sec] (mean)
Time per request: 10377.283 [ms] (mean)
Time per request: 10.377 [ms] (mean, across all concurrent requests)
Transfer rate: 28.01 [Kbytes/sec] received
Kindly suggest what do you advice.
Thanks again for all the help provided.
Thank you so Much JR ,
we are going to take a look and get back to you.
Hello JR,
I would greatly appreciate if you could take a look at my Nginx-FPM config http://pastebin.com/93FQXLq0 The server is similar to Karan’s (Xeon 3210 QUAD with 8GB RAM) and has heavy traffic (around a thousand concurrent connections average), so any suggestion to improve performance would be greatly appreciated.
Regards
Hello Jim,
I’m very sorry, I forgot to answer your question…do you have some problems with current configuration?
Since you are in a localhost environment, instead of using tcp 127.0.0.1 to connect Nginx with PHP-FPM, just use socket connections? Faster!
Thank you! This is a great post.
Really good article – thanks for the effort, it really helped me a lot! :)
Hello JR,
Can you help me configurating an optimal nginx pm? I am really confused with the formula.
I have a VPS of 1GB RAM and 10k daily visitors. Please help
Thanks :)
Hey :)
i was wondering if the listen in the php-fpm.conf can take an ipv6 address plus the port instead of the 127.0.0.1
because i can start the php-fpm but after that i can see that the @ where its listening is really strange and is totaly different from the ipv6 that i gave to listen …
Thank you for your answers !!!
Hi Nao,
Very good question :) Unfortunately I think that the php-fpm can’t listen/handle ipv6 requests, if we believe PHP (php-fpm) manual:
listen string
The address on which to accept FastCGI requests. Valid syntaxes are: ‘ip.add.re.ss:port’, ‘port’, ‘/path/to/unix/socket’. This option is mandatory for each pool.
So it looks like only the ipv4 address + port, port or unix socket are supported…
Just wanted to mention that on Ubuntu, after installing FPM with aptitude, its config file is in /etc/php5/fpm (the blog post mentioned some different paths).
Thanks for an informative guide!
Hi JR, great explanation! I’m using nginx since a few years ago, but your post helped me migrate to php 5.3!
Keep up the good work!!
Hi!
Can I use multiple Unix socket connection, means for every site defined by /etc/nginx/sites-available/.conf I use a deditcated Unix socket connection?
Does this make sense in the context of security?
THX
Hi Thomas,
It shouldn’t be a problem, if you configure also one Unix socket per PHP-FPM pool.
In security context this doesn’t make big difference. If you run this setup just one server then this setup is just as secure as your server with one Unix socket, on the other hand you can restrict permissions of PHP-FPM pool users and groups to access only selected files.
So, in this context of security it makes sense to have a single Unix sockets for all connections, but specific user accounts with restricted permissions for every site.
THX
alright atm i have nginx installed in a few vm’s so of course port 80 will be in use how can i change nginx.conf so i can access saidsite.com instead of saidsite.com:port
Hi derp,
Use listen 80; following inside you server block:
Hi, thanks for the post. Please I have a question.
I’m running nginx on one of 3 jails on a Freebsd installation server.
Now one of the applications is giving this error:
increase post_max_size and upload_max_filesize to 10M
Upload
I don’t how how to increase this.
It’s simple enough in Centos where I do this in php.ini.
In Freebsd, I don’t know where to change it. I’ve found several copies of php.ini here and there and increased the value there. I restarted nginx, but no luck.
Kindly help, thanks.
Hi ginger,
Actually you should restart php-fpm instead nginx.
I can’t say directly what php.ini your system uses, but it might be simply /etc/php.ini or /usr/local/etc/php.ini.
I found the php.ini in /usr/local/etc/php.ini
I increased the limit as instructed, “increase post_max_size and upload_max_filesize to 10M”
and then went to /usr/local/sbin and did service php-fpm restart
It restarted but did not solve the problem
I updated two copies of php.ini – one in the jail housing nginx, and the other in the main root of the server.
Any ideas please? Thanks.
Hi again ginger,
One thing came to my mind. So some application shows:
increase post_max_size and upload_max_filesize to 10M
The problem might also be nginx, not PHP, so you can try to set client_max_body_size in your nginx config.
Like:
You obviously have no idea what you’re talking about — stop misinforming people. Some examples…
location ~* \.php$ {
try_files $uri /index.php;
Hi Derek,
And thanks, I updated first.
Second is based entirely on my own experience (as I tell at the beginning of this page) and if you read PHP-FPM manual then you know why this pm.max_requests even exists. From PHP-FPM configuration manual:
Do you think that PHPs manual will lead people astray?
And you (with arrogant attitude) obviously think that PHPs 3rd party libraries does never have any memory leaks?
Or you maybe could tell some better method avoid PHP and PHPs 3rd party libraries memory leaks?
Hi there!
I’ve the following PHP file:
I’m using nginx/1.2.4 + PHP 5.3.18-1~dotdeb.0 (fpm-fcgi) with specified pools for each virtualhost by using http://www.sebdangerfield.me.uk/2012/05/nginx-and-php-fpm-bash-script-for-creating-new-vhosts-under-separate-fpm-pools/
My problem is, if I run that script nginx is loading it a bunch of time and finally stop. When I stop it running by pressing ESC it says something like an error occured. The picture isn’t that big so it should just load few seconds but it doesn’t… Any ideas?
Thank you in advance.
Regards
Jonas
Hi Jonas,
I fixed your comment little bit. Right/working syntax is <pre lang=”php”>some code</pre>.
And then your problem.
Is all other php scripts working normally?
Do you see any errors on nginx log files and/or php-fpm log files?
Hi,
I am shifting from Apache to Nginx. I am trying to find an alternative for suPHP. Configuring three different pools (site.conf, blog.conf & forum.conf ) as shown in this blog might solve my problem. Right ?
And then in my virtualhost (server tag) I can specify 127.0.0.1:9000, 127.0.0.1:9001, 127.0.0.1:9002 etc in (location – fastcgi_pass). Right ?
Hi Jigar,
Yes, you have fully understood the idea, PHP-FPM works as a replacement of suPHP and yes you can specify 127.0.0.1:9000, 127.0.0.1:9001, 127.0.0.1:9002 in fastcgi_pass.
Hi Guys,
I am getting this issue :
“2012/11/16 12:01:56 [error] 19393#0: *144134 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 123.456.789.2, server: domain.playlab.com.my, request: “GET /download.php?date_start=2012-10-15&date_end=2012-11-16 HTTP/1.1″, upstream: “fastcgi://127.0.0.1:9000″, host: “domain.com.my”, referrer: “http://domain.playlab.com.my/report.php?page=15&date_start=2012-10-15&date_end=2012-11-16″
This issue was generated when getting a report. I was told that, I need to increase the processing time “script” Is there anyone who can advise me on this issue? If the above mention information is not enough please do let me know. Hope to hear from you guys soon. Appreciate it.
Thank you
Hi Nathaneal,
You can check more info from php-fpm logs (under /var/log/php-fpm/).
If problem is PHP max_execution_time time and you want to change it, then just open /etc/php.ini and change:
and then you can also check PHP memory_limit, if it’s too low.
This is little bit off topic, but next you should maybe look into that script why it take so long to execute, because later you will see more problems (maybe even more fatal problems) if you have more users who generate this report same time. Maybe same thing could be done faster and more efficiently, for example with a background processes, database optimization or query optimization, etc. :)
Hi Jr,
Thanks for the advice. I have just amended….will monitor to see the result.
Thanks again. :D
Best Regards
Nathaneal
Hi Nathaneal,
You are welcome and let me know, if you have more problems. :)
Hi Jr!
The tips and the comments where very clarifying for me.
I came to thank you for the time you spend posting and answering people.
I just made a few ajustments in my php-fpm and nginx based on what you tiped here.
Hi i have this php script that runs under this other web host fine but i setup nginx/php5-fpm and when i go to run it it runs fine but doesn’t finish will eventually get 502 bad gateway now it does make a lot of http requests any idea what i should change i’m using default configs for the most part on a vps with 256mem
Hi x,
Is some simple php script working normally?
This is an excellent article. . .I just spun up a VM for a client, and plan to use this article as a starting point to optimize his nginx installation. ;-)
Hi Eric,
Nice to hear that you like this article, thanks. :)
Yeah it works for the most accounts but if i select 40-50 accounts and click “skill” when it goes to equipt stuff by making massive amounts of http request it will eventully show the nginx error but if i select just 20-30 it works fine :/ but on my apache host if i select everything it also works fine and doesn’t error which i find odd
Hi again x,
You can try to increase max_execution_time and memory_limit on your /etc/php.ini file. Then remember restart your php-fpm.
Example:
max_execution_time = 300 memory_limit = 128MNice article, thanks for the tips!
Hi i compiled php5-fpm from source and there’s no php.ini but there is conf.d folders and i’m needing to enable curl/short open tags this is what phpinfo shows any idea’s been stumped with this for a few days then found this site.
Configuration File (php.ini) Path /etc/php5/cgi
Loaded Configuration File (none)
Scan this dir for additional .ini files /etc/php5/cgi/conf.d
Hi derp,
Could you do following (as root):
updatedbAnd post output of following command:
locate php.iniAnd also output of following commands:
We have an issue where RES memory being used by each php-fpm process increased in size as time went by. After a fresh restart of php-fpm the RES for each was 30-70Mb. After letting it go for 8 hours, each php-fpm process was at 1.1GB and still growing. A restart of php-fpm brings it down. However, it was causing serious perfomance issues. At this point we still have not been able to pinpoint the issue. We did lower the pm.max_requests to 1,000 from 10,000 to recycle php. It didn’t stop the RES Memory for each process from growing however.
Hi Andy,
Could you post output, if you run following php script on your server:
Could you tell more specific what you are running on this server? I just wonder, if it is something massive or small?
we have 2 servers, web and DB. APC 3.1.9, nginx/1.0.14, php 5.3.3 memcached. 24GB ram each and good dual hex core cpus. I believe the issue lies with APC and possibly magento. possible memory leak but not ruling out APC configuration. One interesting note. There is an APC php page that you can use to monitor APC. I can get the server to function properly when the value ‘Cached Files’ stops caching files.
right now, this value is stuck at 847. When the server breaks, this value will continually grow to thousands, which it should do. so, something amiss with APC most likely.
Array ( [0] => Core [1] => date [2] => ereg [3] => libxml [4] => openssl [5] => pcre [6] => sqlite3 [7] => zlib [8] => bz2 [9] => calendar [10] => ctype [11] => hash [12] => filter [13] => ftp [14] => gettext [15] => gmp [16] => session [17] => iconv [18] => Reflection [19] => standard [20] => shmop [21] => SPL [22] => SimpleXML [23] => sockets [24] => exif [25] => tokenizer [26] => xml [27] => cgi-fcgi [28] => apc [29] => curl [30] => dom [31] => fileinfo [32] => gd [33] => json [34] => mcrypt [35] => memcache [36] => mysql [37] => mysqli [38] => PDO [39] => pdo_mysql [40] => pdo_sqlite [41] => soap [42] => wddx [43] => xmlreader [44] => xmlrpc [45] => xmlwriter [46] => xsl [47] => zip [48] => ionCube Loader )
Hi Andy,
Thanks and sorry for long delay, I actually remember that I answered your comment once.
But yes I have seen very bad performance problems with all Magento setups. Magento use RAM ~100 Mb per page load and if you run heavy traffic site even with proper caching and optimizations you probably have performance problems.
I have also seen some problems with APC when there is “too much” files to cache. Do you use APC also as user cache? Or just for PHP file cache?
If you have pretty much static Magento (PHP files) then you can set following to APC config:
apc.stat=0If apc.stat is off then APC will not check everytime are cached files changed or not. You get much better performance, but you have to remember restart web server or clear cache manually if you change some files.
I turned off APC and am using a filesystem cache. in Magento you define if you want to use APC as the caching module in the local.xml file. I removed all reference to APC, however the issue still persisted. With APC on, each php-fpm process that spawned increased in RES memory size. Even with no reference to APC. It was not until I completely turned of APC in apc.ini that this issue finally stopped. If Magento removes all references to APC, is it still possible that Magento and APC are still talking and don’t like each other? OR does this just sound like a pure APC problem?
Nice to hear that turning APC completely off solved your problem.
You can use APC as user cache, which you configure on Magento config files (local.xml) and PHP file cache for all .php files, which is used as default when you have APC running with php-fpm.
When you have turned APC off from apc.ini and removed all APC references from Magento config, then you should have php-fpm and Magento running totally without APC.
If you have this style problems with APC and still want some better caching than filesystem cache, then you can try xcache for .php file cache and Memcached as user cache. You have very powerful servers and lot of RAM, so that’s why it might wise to setup some caching using RAM.
thanks for this informational tutorial . it helped me to conrol my load around 1
Hi JR,
i am interested with your article, i have some problem with php5-fpm. U can see our top and config capture on the URL picture below :
-top :
http://grab.by/l0wo
-config php5-fpm :
pm.max_children = 16
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 8
Why php5-fpm service using high CPU load?is there any issue with my configuration? need your suggestion, please :)…
Many thanks.
Hi adm_junior,
Could you also post information which distro you use and what is your PHP version?
And then also some info what kind of site you are running, do you use some caching?
I have a 5 servers using load balancing.
One of them has 64 core cpu, 256GB memory. What’s your recommendation for this specification?
Thanks.
Hi Kevin,
Sounds awesome hardware. What specs you are looking for? Just nginx or also php-fpm on same machines?
Have to ask, that do you have some current setup and do you have some problems with it?
Sometimes it’s hard to say exactly best configuration, because it also depends on site(s) what you are running and of course traffic.
Your tips on tuning PHP-FPM really helped me out. I had my host recently migrated my service to a newer hardware and borked it. I had to rebuild a new LEMP stack once i got it all back together you really helped me resolve a zooming memory issue. Good write up. Hope you don;t mind a trackback. ;D