how to accelerate virtual host zope with squid

Created by . Last modified on 2003/08/05.

This howto has two parts - one is for single domain zope's (and will work
with older squids), the second is virtual hosting and needs squid 2.4 or
better. I recommend going for the second one even if you're not currently
doing multple domain hosting, because it will be easy to add them later
without anything coming offline.

~~~~~~~~~~~~~~~~~~~~~~
2/27/2004 -- I just noticed I got a complaint about this HOWTO in Jan...
Well, if you're frustrated with this how-to you can go check this guy's HOWTO out:
http://www.zope.org/Members/JCLawrence/LocalhostSquidHOWTO/index_html
~~~~~~~~~~~~~~~~~~~~~~

------------------------------------------------------------------------
HOW TO RUN ZOPE (ZSERVER) BEHIND SQUID AND XINETD, single site
(www.mysite.com) version.


Redhat 7.1, Squid2.3.STABLE4, zope 2.3.3, webmin0.91 (for administration)

Actually, XINETD isn't used, but part of the instructions below are to
disable it. But, we'll show you how to set it up so that you can, for
debugging purposes, run the XINETD service instead of SQUID, without having
to change your Zope setup.

1. create a new user zope with group nobody. The way I like to do it is
with a home directory of /usr/local/zope, and allowing logins (makes
upgrading your zope via FTP easy - no permission issues later). Install
your zope here.
- using Webmin, create a new startup action in 'bootup and shutdown'.
The main thing is that zope listens only to the local IP address
127.0.0.1. Here's an example:

/usr/local/zope/Zope-2.3.3/start -u zope -p - -W 8081 \
-a 127.0.0.1 -w 80 -t 20 \
&> /usr/local/zope/Zope-2.3.3/var/log/startup.log &

this also sets us up for 20 threads, WEBDAV port of 8081, and redirects
startup messages to a log file. Note, you MUST create the folder
var/log in your zope setup first. There are other ways to do that,
it's just something easy that I do so I can check startup messages if
need be. The stop script would be

/usr/local/zope/Zope-2.3.3/stop

2. in XINETD set up the http network service (change from default):
- bind to address: the external IP address, port 80.
- redirect to IP address 127.0.0.1, port 0.
- Run as user zope
- keep this service disabled while running squid, but it'll be handy
when first setting up zope.
- you must 'apply changes' to start or stop the http service.

3. get and install siteaccessenhanced product (http://www.zope.org/Members/sfm/SiteAccessEnhanced).
Put a siteroot in your root, with BASE of http://www.yoursite.com and PATH of /

4. in SQUID, change the following from default:
- Ports and Networking. Put in port 80, and the hostnames and IP
addresses you want to respond to. E.g. the external IP addresses (not
127.0.0.1) and names (www.yoursite.com)
- Other Caches. Put 'directly fetch URLs containing: /manage
- Miscellaneous Options. Set HTTP Accel Host to 127.0.0.1, and HTTP
Accel Port to 80.
- Access Control. Make a new ACL called zserver of type Web Server
Address, set the address to 127.0.0.1 netmask 255.255.255.255. Add proxy
restriction, 'Allow', Match ACL 'zserver'.
- Logging. Use HTTPD logging format.
- Cache Options. Maximum Request Body Size will limit the size of the
form-based uploads to your site. Default is about 1MB.

5. in 'bootup and shutdown' make sure xinetd, squid, and zope are all started on
boot. When running Squid, make sure the http service is disabled in
xinetd.

6. make sure in your Network Configuration that your external IP address
is associated with www.yoursite.com

7. Webalizer command:
/usr/local/bin/webalizer -p -N 10 -D dns_cache.db -o /usr/local/zope/Zope-2.3.3/var/cachestats/ /var/log/squid/access.log
You can set this up as a cron command to generate logs, then use the
localFS product to make a folder that's pointing to
/usr/local/zope/Zope-2.3.3/var/cachestats/ for viewing TTW. Note that
you must create the directory var/cachestats first.

------------------------------------------------------------------------
HOW TO RUN ZOPE (ZSERVER) BEHIND SQUID AND XINETD, multiple virtual site version.


Redhat 7.1+, Squid2.4.STABLE1, zope 2.5, webmin0.92 (for administration)

NOTE that squid2.4 comes with RH7.2. I had to force upgrade openssl-0.9.6b-8.i386.rpm
in order to install Squid 2.4. You NEED the newer squid in order to do
this without writing your own redirect scripts. (NOTE: forcing that install broke my ssh login ability so I wouldn't recommend it. I ended up having to upgrade to 7.2 and then run up2date to get everything back in sync. Not too painful!)

Actually, XINETD isn't used, but part of the instructions below are to
disable it. But, we'll show you how to set it up so that you can, for
debugging purposes, run the XINETD service instead of SQUID, without having
to change your Zope setup.

1. create a new user zope with group nobody. The way I like to do it is
with a home directory of /usr/local/zope, and allowing logins (makes
upgrading your zope via FTP easy - no permission issues later). Install
your zope here.
- using Webmin, create a new startup action in 'bootup and shutdown'.
The main thing is that zope listens only to the local IP address
127.0.0.1. Here's an example:

/usr/local/zope/Zope-2.5/start -u zope -p - -W 8081 \
-a 127.0.0.1 -w 80 -t 20 \
&> /usr/local/zope/Zope-2.5/var/log/startup.log &

this also sets us up for 20 threads, WEBDAV port of 8081, and redirects
startup messages to a log file. Note, you MUST create the folder
var/log in your zope setup first. There are other ways to do that,
it's just something easy that I do so I can check startup messages if
need be. The stop script would be

/usr/local/zope/Zope-2.5/stop

2. in XINETD set up the http network service (change from default):
- bind to address: the external IP address, port 80.
- redirect to IP address 127.0.0.1, port 0.
- Run as user zope
- keep this service disabled while running squid, but it'll be handy
when first setting up zope.
- you must 'apply changes' to start or stop the http service.

3. get and install siteaccessenhanced product (http://www.zope.org/Members/sfm/SiteAccessEnhanced).
Put a Virtual Host Monster in your root. Name it anything. Then go
back into it and start adding your virtual mappings of domains to
folders.

4. in SQUID, change the following from default:
- Ports and Networking. Put in port 80, and the hostnames and IP
addresses you want to respond to. E.g. the external IP addresses (not
127.0.0.1) and base domain names (yoursite.com)
- Other Caches. Put 'directly fetch URLs containing: /manage
- Miscellaneous Options. Set HTTP Accel Host to 127.0.0.1, and HTTP
Accel Port to 80.
- Access Control. Squid comes with localhost setup already, which is
good. We need to make ACLs for all our virtual domains.
Make a new ACL called ext-ip-addr of type Web Server
Address, set the address to your external IP address and netmask 255.255.255.255.
Add proxy restriction, 'Allow', Match ACL 'ext-ip-add'. Put it just
above 'deny all' in the access list. Next, make an ACL of type Web
Server Hostname, and put all your hostnames in there. Note you can do
catchalls like *.mysite.com to take care of all subdomains. Again add
this to the access list just above 'deny all'.
- Logging. Use HTTPD logging format so we can use webalizer. Otherwise
do whatever ya want!
- Cache Options. Maximum Request Body Size will limit the size of the
form-based uploads to your site. Default is about 1MB. My site has lots
of mp3 audio files, which can range up to 5MB. Note, this is a cheap
and dirty way to impose a maximum restriction on all uploads (since
currently zope doesn't have such a setting).
- currently Webmin doesn't support some new features of 2.4 which we
need, so you need to manually edit your /etc/squid.conf file and add the
following (either in the appropriate place or just at the bottom):

httpd_accel_single_host on #this forces all virtual hosts to http_accel_host

ie_refresh on #hack around IE5 refresh bug. IE will always need hacks :)


5. in 'bootup and shutdown' make sure xinetd, squid, and zope are all started on
boot. When running Squid, make sure the http service is disabled in
xinetd.

6. make sure in your Network Configuration that your external IP address
is associated with all your virtual domains. Obviously your DNS servers
have to be setup as well.

7. Webalizer command:
/usr/local/bin/webalizer -p -N 10 -D dns_cache.db -o /usr/local/zope/Zope-2.5/var/cachestats/ /var/log/squid/access.log
You can set this up as a cron command to generate logs, then use the
localFS product to make a folder that's pointing to
/usr/local/zope/Zope-2.5/var/cachestats/ for viewing TTW.

You're done! You should be able to browse zope now, and enjoy the enormous
speed improvemant of squid. Of course, there's a whole world out there of
tweeking your squid performance -- have fun!

[end]