Request a Quote

Nginx Optimization for high traffic websites

Nginx Optimization for high traffic websites

December 31, 2018 at 10:27 am By Rishil Shaji
Nginx is fast, lightweight and alternative for Apache web server, Like any other web server, we need to tune nginx to get the full performance.

Why NGINX over Apache?

Apache and Nginx are the two most common open source web servers in the world. Together, they are responsible for serving over 50% of the traffic on the internet. NGINX was written specifically to address the performance limitations of Apache web servers. NGINX can be deployed as a standalone web server, and as a frontend proxy for Apache and other web servers. The performance and scalability of NGINX arise from its event‑driven architecture. It differs significantly from Apache's process‑or‑thread‑per‑connection approach – in NGINX, each worker process can handle thousands of HTTP connections simultaneously. This results in a highly regarded implementation that is lightweight, scalable, and high performance Following tweaks can be done to Nginx configuration for full performance.
    • Switch from TCP to UNIX domain sockets
    • Adjust Worker Processes
    • Setup upstream load balancing
    • Disable access log files
    • Enable GZip
    • Cache information about frequently accessed files
    • /etc/sysctl.conf tuning
    • Monitoring the no of connections

1. Switch from TCP to Unix domain sockets

When communicating to processes on the same machine UNIX sockets have better performance the over TCP because there's copying of data, fewer context switches. UNIX domain sockets know that they're executing on the same system, so they can avoid some checks and operations (like TCP negotiation and routing etc..).
Eg:
For TCP domain sockets we use IP sockets server 127.0.0.1:8080; , for changing the sockets to UNIX domain sockets we replace the IP with the port with the server socket file unix:/var/run/fastcgi.sock;

NB: If you are using more than 1000+ connections per server use TCP server because they scale better.

2. Adjust Worker Processes

The worker process is responsible for processing the request. It is recommended to set the no of worker processes with respect to the no of CPU cores. ie: adjusting the worker_processes in your nginx.conf file to the number of cores your machine has and increase the number of worker_connections. You can use below command for determining the no of  processors your machine:
cat /proc/cpuinfo | grep processor

3. Adjust Worker Connections

The maximum number of simultaneous connections that can be opened by a worker process. This no. include all connections. By default, this value is set as 512. Eg: If you are running only 1 process with 512 connections, you will only be able to serve 512 clients. If 2 processes with 512 connections each, you will be able to handle 2x512=1024 clients.

4. Setup Upstream Load Balancing

Multiple upstream backends on the same machine produce higher throughput than a single backend. For example, if we have 1000 max children, we can divide the no across multiple backend files according to the weight that is specified. Eg:
upstream backend {
	server 10.1.0.101 weight=4; 
	server 10.1.0.102 weight=2;
	server 10.1.0.103;
	}
Here the first server is selected twice as often as the second, which again gets twice the requests compared to the third.

5. Disable access log files

Access logging of high traffic website involves high I/O and disk space. Instead of writing the access log to the disk immediately we can buffer the log to the memory ie: add the buffer=size parameter to the access_log directive to write log entries to disk when the memory buffer fills up. If you don't want the access log you can use these to disable the access logging
access_log off; 
log_not_found off; 
error_log /var/log/nginx-error.log warn;
If you want access logging, but don't want to write to the disk immediately you can buffer the log to the memory ie: write log entries to disk when the memory buffer fills up.
access_log /var/log/nginx/access.log main buffer=16k;

5. Enable Gzip

Compression is huge performance accelerator. Compressing responses often significantly reduce the size of transmitted data. We can specify the compression level in gzip_comp_level 6; , but increasing the gzip_comp_level too high as the server will waste CPU cycle. You can specify the compression type, length and compress proxied request.

6. Cache information about frequently accessed files

Caching improves web server to deliver the contents on the clients request in a much faster way. In Nginx we can specify open_file_cache to the static web content, open_file_cache will cache metadata about files only. In this we can specify the validity, duration, caching of error pages. Three technique to cache content generated by the web server
      • Moving content closer to users
      • Moving content to faster machines
      • Moving content off of overused machines

7. PHP-FPM pool tuning

High traffic website uses php-fpm uses a lot of processor power. The problem is that when using the default configuration for php-fpm pool ends up in using high server resource (RAM and CPU). We can see the process that is been created by monitoring TOP, HTOP commands in linux. There are 4 values related to child processes that we should adjust to increase php-fpm performance:
      • pm.max_children: the maximum number of children that can be alive
      • pm.start_servers: the number of children created on startup.
      • pm.min_spare_servers: the minimum number of children idle.
      • pm.max_spare_servers: the maximum number of children in idle.
PHP-FPM can be tuned according to static and dynamic process. In static process we can set pm.max_children & pm.max_requests, Eg:
pm = static
pm.max_children = 5
By setting pm.max_children 5 (fixed value) we are setting maximum number of children that can be alive on the server. Similarly we can kill and respawns processes after handling a certain number of requests by setting pm.max_requests In Dynamic process we are setting pm.max_children, pm.start_servers, pm.min_spare_servers, pm.max_spare_servers, pm.process_idle_timeout. In dynamic process management we use:
pm = dynamic
We can calculate the Max process by the formula below:
Total Max Processes = (Total Ram - (Used Ram + Buffer)) / (Memory per php process)
Max request per process is unlimited by default, but it’s good to set some low value, like 200 and avoid some memory issues.

8. /etc/sysctl.conf tuning

Cloud instances that are not fully optimized for the full availability.e: the default systems may fail under high load. So we need to perform sysctl tweaking for maximum concurrency. Here is the list of tweaking that can be done to sysctl configuration file.
      • Increase system IP port limits to allow for more connections
      • Number of packets to keep in backlog before the kernel starts dropping them
      • Increase socket listen backlog
      • Increase TCP buffer sizes
      • Disable Ping-flood attacks
      • Do less swapping and use RAM

9. Monitoring the number of connections

Monitor the logs, connection, resource usage and no of waiting threads. Set alerts to get notified if the threshold value exceeds. For monitoring website and server monitoring and performance tracking, we can use cloudwatch metrics in aws or we can use monitoring tools such as Monitis etc.

Conclusion

For getting the full potential of any web server we need to tune it according to the server resource usage and gain its full potential. At the end of the day we need to keep on track of the server by monitoring the server resource utilization, tweaking according to needs making it more efficient. Making all the tweaking and optimization we can make nginx lightweight, scalable, and high-performance web server.