Preparring for a High traffic surge (Twitter/Digg/Slashdott style)

Permalink
Hey guys,

One of the websites I manage is going to be linked to from a very high profile Twitterer later this week. The twitterer was keen to make us aware his previous link posts have sent websites crumbling (50k visitors in an hour).

Over the next couple of days I am scaling up the server to handle this load by increasing our VPS memory limits and changing from apache to an optimised webserver such as lighttpd or nginx.

The key to preventing the server overloading will be caching.

So firstly I am going to replace the homepage with a static html page using a apache redirect, to not effect the way concrete uses index.php to load all pages. Depending on how much time I have I may convert other high traffic pages to static.

I know concrete caches content by default which is great. Reducing db calls is obviously key. I have been looking into caching uses software like memcached but think this may be too time consuming to implement. Is there anything else I can quickly do to optimise concrete5?

Has anyone had any experience with high traffic concrete sites?

I will report back after the event to let you guys know how everything went. Maybe this will be of interest.

Many thanks,

 
Remo replied on at Permalink Reply
Remo
What I've been playing around with:

1 Page cache. C5 currently caches objects, in case a page is almost static you could also cache the complete page. I added a page attribute for this which I'm checking before the rendering process (ugly dispatcher hack) to display a simple html file. I never published it because you have to know where you can activate it.. I would get a lot of silly questions about it for sure

2. Reverse proxy. A lot of sites don't need a states. Meaning you could set up 5 web server which don't have a connection. Mirror the site from a master server to the slaves every hours.. Ugly but allows you to scale beyond everything. In combination with nginx as a reverse proxy, this should work fine..

3. memcachedb. I tried to replace eAccelerator using memcache but unfortunatley I stopped because we realized that we don't have a lot of content in that project where I needed a bit more power. We're working with CodeIgniter which is fine if you don't have content to manage..

4. IO. Usually you'll run into IO bottlenecks... VPS is not the best solution if you have lots of I/Os...
katz515 replied on at Permalink Reply
katz515
My concrete5 Japan site...

1. Installed eAccelarator

2. httpd.conf

My server setting

StartServers 20
MinSpareServers 2
MaxSpareServers 5
ServerLimit 200
MaxClients 200
MaxRequestsPerChild  4000
KeepAlive On
MaxKeepAliveRequests 4000
KeepAliveTimeout 1


Key point is KeepAlive parameter.
KeepAliveTimeout should be "1" so that it will die immediately, but at the same time, it can handle large number of users.

3.my.conf

is really depends on how big your MySQL database is... so cannot really tell you... I suggest to keep looking at your phpMyAdmin statistic to adjust each parameter.

This handled 10-20 simultaneous users/sec access

Good luck.
wltr replied on at Permalink Reply
wltr
If you have a .htaccess file you should put its content into the virtual host settings of apache and remove the file. Because .htaccess gets loaded on every request.
andrew replied on at Permalink Reply
andrew
1. Definitely eAccelerator.
2. If you increase your server's limits, make sure you enable more connections in your database, as well as increasing MySQL's memory limit.
3. Further caching is ideal, including full page caching. Unfortunately concrete5 doesn't do this yet, but we have hacked solutions in the past. Check out PEAR CacheLite.

For something a little bit more robust and a site that is less dynamic, a reverse proxy is another good solution.
andrew replied on at Permalink Reply
andrew
We'd love to hear how it goes after the linking!