Optimising Zope

We (http://www.ekit.com/) have a popular website, and it's hit fairly hard (around a 6 hits * a second). I've therefore been working on ways to get Zope to go as fast as possible. Here's some hints for others who are running (or are expecting to run) high-volume Zope sites:

1. CallProfiler up the wazoo. Perform lots of CallProfiler analysis and fixing of hotspots. We managed halve our average page load time just doing this.

2. Use RAM caches in Zope - find the HTML chunks that don't get changed for every visitor (but rather large groups or all visitors). Use the CallProfiler to isolate calls that change very little between users. RAM cache those. Again, we have achieved halving, or more, through caching just a couple of common components.

3. Make static content (images, stylesheets and some pages) cacheable. Put stylesheets and images in an area of your Zope that isn't authenticated by cookies. Most browsers won't cache images or a stylesheet if a cookie was sent along with the request. Similarly, don't request stuff by HTTPS when you don't need to - not only does the server have to do more work, but the client won't cache anything served by HTTPS. There's a neat tool out there that does cacheability testing:

   http://www.mnot.net/cacheability/

   Getting caching right in Zope can be quite a challenge - you may wish to just skip this and use a method like we did in 6b below.

4. ZEO clients - we have about 9 of them, spread over a number of CPUs. We'll be putting in some more real soon. If your Zope servers are CPU-bound, then extra ZEO clients will only help.

5. Don't change the ZODB unless it's absolutely necessary. Changes to the ZODB things down considerably. Use of a backend high-performance database like Oracle is a very good idea.

6. Use a front-end cache for static content. There's two ways to do this:

   1. Throw in squid and stuff will be served much much faster. This assumes that the content is cacheable - see point 3.
   2. We use apache frontends using FastCGI † to the ZEO clients. They have an on-disk cache which serves up images (using rewrite rules to trap image URLs before they are sent to Zope) meaning Zope can do more important stuff than serve up static binary files. This halves the average page load time - our pages have between 50 and 120 images (yes, 120 images is a lot and falls under the general rules for optimising HTML pages).

7. Python won't make use of multiple processors ‡ so if you have them available, you are strongly advised to set up at least one ZEO client per processor (on fast machines, having more than one can sometimes be another performance boost).