375 Million pageviews for 15$ a month (on an Amazon EC2-Micro instance.)
This is part of our “Hackerspace Trials and Tribulations” series in which we blog about certain problems we’ve encountered while getting our HubCityLabs off the ground, and the solutions we found worked best for us. You can read the first post of the series here: Hackerspace Trials and Tribulations: Payment Processing
Back in late August, one of our articles, DIY PCB Etching at HubCityLabs was covered on Hack-A-Day. Almost instantly, our website kicked the bucket. At the time, we were hosting the website on a small vps provided by one of our members.
We quickly transferred everything over to a temporary Amazon EC2 instance, and were able to get back up in short order. Since then, we’ve moved the site into its current location, on a Debian EC2 instance, and we were able to optimize things so that we’d be able to sustain a rate of 375 million page views in a month (500000 pageviews/hour) on a WordPress-based site. This is the story of how we got there.
Amazon lists the following hardware equivalency for their EC2 Micro instances:
- 613 MB memory
- Up to 2 EC2 Compute Units (for short periodic bursts)
- EBS storage only
- 32-bit or 64-bit platform
- I/O Performance: Low
- EBS-Optimized Available: No
I decided to go with a Debian image (debian-6.0-squeeze-base-x86_64) since Debian is my favorite server distro. Once that was up and running, it was decided that we would be running on an Nginx/php5-fpm/php5-apc stack due to the limited available resources.
We ran a blitz, and quickly found our first problem. In nginx’s error log, we were getting a lot of “too many open files” errors. To fix, we appeded “fs.file-max = 1000000″ to /etc/sysctl.conf, and also set our values in /etc/security/limits.conf to “nginx soft nofile 10000″ and “nginx hard nofile 100000″. That seemed to resolve the issue. At that point we were able to get about 1500 pageviews/hour.
For WordPress, we did 2 important things. First, we disabled WordPress’ internal cron by adding this in wp-config.php “define(‘DISABLE_WP_CRON’, true);”. Next, we setup a Debian Cron task to run the cron manually, on a schedule, instead of on each page load.
Next, we installed the WP-Super-cache plugin (there are other good ones too, like W3 Total Cache), and configured it for aggressive caching. At this point, Blitz.io reported 60000 pageviews/hour.
We installed Varnish as a font-end cache to reduce the load on the web server even more. After a little tweaking, Blitz.io reported us being able to sustain 500000 pageviews/hour without errors or timeouts. At that rate, the system is still snappy on the shell, and page load times are more than reasonnable.
Hopefully this will help other hackerspaces, and people on a limited budget, to bulletproof their web hosting on the cheap. There are probably more tweaking that could have been done, however we do not forsee a need, as half million pageviews/hour is much much more than we can possibly imagine having.
Installing NGINX and PHP-FPM running on UNIX File Sockets
10 Million Hits a Day on WordPress/Amazon EC2
Linux/NGINX Too many open files
WordPress and WP-CRON