When Cookies won't do it

Created by . Last modified on 2003/08/05.

On occasion we would like to store temporary data relating to an individual HTTP session. One way is to use cookies but if the length of the stored data becomes large then a cookie based solution becomes inefficient, as the cookie needs to be transfered to the server with each request. Similarly storing the data in ZODB is also inefficient because new object versions are created with each transaction.

A simple solution is to marshal the data in a file, which is fast and supports all basic python data structures. If we are careful to prevent the creation of a large number of files in a single directory, then file access times will be fast. A simple implementation follows:

    import os,string,time
    import marshal
    import App.FindHomes

    tmpStorageDir=os.path.join(INSTANCE_HOME,'var','tmpStorage')

    def tmpStore(self,data,RESPONSE,ex=os.path.exists,jn=os.path.join):
        '''Marshals and stores data on a regular file given by unique id'''

        # Create UID
        tm=time
        ip=self.REQUEST['REMOTE_ADDR']
        t=tm.gmtime(tm.time())
        uid='%s%s'%(tm.strftime('%j%I%M%S',t),ip[-5:])
        day=jn(tmpStorageDir,uid[:3])
        hour=jn(day,uid[3:5])
        minute=jn(hour,uid[5:7])

        if not ex(day):
            os.mkdir(day)
        if not ex(hour):
            os.mkdir(hour)
        if not ex(minute):
            os.mkdir(minute)

        f=open(jn(minute,uid),'w')
        marshal.dump(data,f)
        f.close()
        RESPONSE.setCookie('tmpStorageUID',uid)
        return uid

def tmpRetrieve(self,tmpStorageUID=None,jn=os.path.join):

'''Retrieves data stored in temporary file. '''

if tmpStorageUID==None: if self.REQUEST.has_key(tmpStorageUID): tmpStorageUID=self.REQUEST['tmpStorageUID'] else: return None uid=tmpStorageUID day=jn(tmpStorageDir,uid[:3]) hour=jn(day,uid[3:5]) minute=jn(hour,uid[5:7])

f=open(jn(minute,uid),r) data=marshal.load(f) f.close() return data

Install the module by downloading or copying the above module into your Extensions directory in the Zope distribution. Make sure to create a directory tmpStorage under Zope/var with appropriate permissions. Create two external methods somewhere in your Zope tree (possibly at the root level). For illustration let's call them tmpStore and tmpRetrieve.

To use them you could simply call them in DTML like:

    <dtml-var "tmpStore(objectIds(),RESPONSE)">

The above DTML snippet will temporary store the objectIds list of the current folder on the disk. It will also set a cookie on the client which enables automatic retrieval of the stored data at a later time. If cookies are not available the actual call to tmpStore returns the UID of the stored data which you can pass around either in the URL or as a Form variable. For example:

    <dtml-in tmpRetrieve>
      <dtml-var sequence-item>
    </dtml-in>

will retrieve the data and iterate over them displaying each id. Needless to say tmpStorage is not very useful for storing objectIds, but expensive methods that return large lists of data can be a good application for it.

If you decide to pass the UID in the URL make sure you use the variable name tmpStorageUID, like:

    http://www.zope.org?tmpStorageUID=<the uid returned from the tmpStore call>