ZenFromZopeCoders

A guide to being a Zope Coder

*Steve Alexander started on 29 September 2001*

First, a note that this is just one of many ways of doing things. It may not even be the best, but here it is anyway.

I haven't credited everyone who said everything. This is sort of deliberate, and sort of because I am lazy. I'm hoping that this will transform itself into a number of documents with correct answers and procedures in them, and thus it won't matter exactly who said what.

The information here is compiled from posts to the Zope-Coders list, and represents a How-To guide to being a Zope Coder -- a person with check-in rights to the Zope CVS.

Definitions

Here are some definitions of terms used when talking about working with CVS and Zope.

Sandbox

A directory on your own computer where you check out a branch from CVS and do work on it. This work can be experimental, as it is in your own sandbox.

Sandbox checkouts

These are checkouts on a branch (for example chrisw-zeowait-branch) for testing a prototype of a new feature.

You might want to test out a new feature by making a sandbox on a branch, making a small (1/2 page) fishbowl project, and seeing if it generates dissent, and lobbying the zope core team.

Current branch

The current release branch. At the time of writing, it is Zope-2_4-branch.

Trunk

The trunk should never be in a state where we would be uncomfortable making an alpha or beta from it on 2 days notice. The trunk represents the latest stable, pretty well tested, ready to run Zope.

The head

Same as the Trunk

Places to look for more information

This page (http://dev.zope.org/CVS/ZopeCVSFAQ) walks you through checking out Zope, cvs branches, how to change work in branches, and commit that work back into releases.

http://dev.zope.org/CVS/CommitterGuidelines

You can see who owns the large-scale projects related to Zope here (http://dev.zope.org/CVS/RepositoryProjects)

http://lists.zope.org/pipermail/zope-checkins/2001-September/date.html

Recommended practices

Document functional-style code with a short explanatory comment. This includes map, reduce, filter, list comprehensions.

Code checked in must work with the currently officially supported Python version, and it should work with later versions when possible and practical. The current officially supported Python at the time of writing is 2.1.

Do not compare by type:

      if type(foo) == type(''):

Instead, use 'instanceof':

      import types
      if instanceof(foo, types.StringType):

Don't use this common idiom:

      if type(foo) in (StringType, UnicodeType):

as it will break after the type/class unification starting in Python 2.2.

Brian Lloyd on supporting unicode:

    The official policy statement is: we don't want to be writing
    new code that (intentionally) would prevent unicode from being 
    used in a given situation.

    Getting Zope unicode-friendly will take time, and will need to 
    be tackled in a number of projects. Some will tackle actually 
    making use of unicode (for I18N), and others will be concerned 
    with fixing old code that is not prepared to see unicode.

    Most of the hard issues will be dealt with in the proposals and 
    other artifacts of those projects. When we run into (presumably 
    smaller) unicode-related issues in other work, the right answer 
    is to either make the code unicode-friendly (for minor things) 
    or to escalate the issue to a proposal (when we run into an 
    unexpected can of worms).

  Email addresses for project owners

    [email protected] for CMF

    [email protected] for Zope core.

    Chris McDonough has worked on ZCatalog, and can approve checkins.

    Andreas Jung is the current ZCatalog owner.

    Brian Lloyd is the owner of STX.

    The boundaries of ownership of packages in Zope are pretty flexible.

CHANGES.txt

The path to the file is doc/CHANGES.txt. It lists the changes that have been made to Zope since the last released version.

If you're fixing a bug, you only need to add to CHANGES.txt in the branch. If you're adding a feature, you only need to add to CHANGES.txt in the trunk. Brian Lloyd later stitches them together.

Bugs fixed and Features added

There should be a section "Features added" and "Bugs fixed" for every release or version inside CHANGES.txt. We usually append a comment to the end of the corresponding section.

You append your change to the end of the "Features added" or "Bugs fixed" section. The section to use is that for the current release, which appears at the top of the CHANGES.txt file.

The changes for each release are in reverse-chronological order by release. if you're changing the release branch's CHANGES.txt and there's no entry for the release you're working on (let say, for instance, it hasn't been released yet! ;-), just make a section named "Unreleased" and put the comments under it.

You should usually put a note in CHANGES.txt for each change you make. You should identify the note with your initials or some other identifier, so that it is easy to see who was responsible for what if there are questions later.

Note that it's usually a good idea to reference the Collector issue number in the CHANGES.txt.

Q: Should all changes get a note in CHANGES.txt?

A: Use your own judgement, but it is generally good citizenship to make a note of things you've done (if for no other reason, other people can tell that you've already taken care of a problem without wading through CVS logs). Excruciating detail is not necessary. A note of "The docstrings in Magic.py were way too terse, so I rewrote many of them to be more clear" is fine.

An example: the no-brainer bugfix

This is a special case where there is an obvious typo that obviously doesn't affect anything else.

Such a bugfix can be checked directly into bother the current branch, and the trunk. A note should be added to the CHANGES.txt on the current branch.

On bugs, bugfixes and features

Tim Peters says:

A bug is actually (at least) two bugs: the obvious bug, but also a bug in the test suite that failed to catch it (since a test suite's purpose is to catch bugs, a bug it misses is by defn. a failure of the test suite to do its job).

The difference between bugfixes and features

Bugfixes

No documentation obligations, except CHANGES.txt (unless it is a documentation bugfix)

Should update unit tests like this

Either correct a "faulty" test that let the bug through, or add a new test that would have caught the bug, and passes the fix. Don't forget the rationale.

Don't worry about test suites getting too bloaty. It is easier to remove / rationalize them if they are there in the first place!

Reference the collector id in the CHANGES.txt and checkin log

Should be checked into both Trunk and Stable releases.

Features

Usually have documentation, UI, and other dependencies.

Normally need a fishbowl project, although, can be a small 1/2 page fishbowl document.

Never check a feature into a stable release!

Be careful with adding small features; they often touch a lot of things. Best to ask about it on zope-dev or zope-coders. Remember, features can touch protocols, UI, documentation, ...

Tests

Q: Is the process for running the Zope tests the same or should I use testrunner.py?

A: This is a good question. For now, I'd pick an existing test module and ape its structure.

Also, Evan Simpson and Jeremy Hylton have some ideas about how best to code unit tests.

Jeremy said

"""I think we reached some conclusions about how to keep the testing structure simple and portable, meaning that it works the same way for different projects like Zope and StandaloneZODB?.

Evan-- Were you going to post anything about this? Or should I? We really ought to write down what we talked about before we both forget it all.

There's a further step to testing, which is to require that before you check something in you should run the tests and make sure you haven't broken anything. I assume that the testrunner script is the right way for Zope. If you make changes in ZODB, BTrees?, ZEO, etc., you should also run the StandaloneZODB? test suite.

There are a bunch of tests that fail when you run the Zope testrunner from the trunk. We should fix those ASAP, because external developers shouldn't be committing changes while our own code is in an unknown state.

"""

Q: What sort of tests should be aimed for with community checkins?

A: Tests that exercise the public interface of classes defined in the new or repaired modules.

Things to be done

Set up a page that lists the owners of packages at a fine-grained level.

Set up page with the following:

• what the next planned bug fix release is
• what the current bug-fix release branch is (where all bug fixes, but not new features, should be checked in)
• what the next planned feature release is
• what the correct branch is to check in both bug fixes and new features
• who to ask if its not clear what to do

Branches

Question Another point I'd like confirmation on: changes should be checked into the current main branch and the trunk. Are there ever any other branches we would need to be aware of, or is it the responsability of the branch owners periodically to merge changes from the branch off which they have branched? (This paragraph was brought to you by the word branch.)

Answer If you are collaborating with others on some non-trivial work, you might all choose to work on a particular branch that you've made, and someone on that team would be responsible for merging the branch to the appropriate places when you are all done.

There are currently no other "official" branches that you need to be aware of other than the trunk and current release branch.

How Evan keeps his own personal patched Zope up to date

My personal pattern for making non-bugfix changes to my Zope 2.3 server is to make the change in my sandbox, test it, then use cvs diff to make a patch file in a Patches directory, like this:

      cvs diff -u lib/python/OFS/ObjectManager.py >../Patches/omchange

If I want to start a new sandbox, I just:

      patch -p0 ../Patches/*

in the root of the new sandbox.

Maintenance of old branches

An old branch is one for a release that is not the current release.

For example, the current release brance is Zope-2_4-branch.

An old branch is zope-2_3-branch.

The official Zope Corp. position on checkins to old release branches (such as zope-2_3-branch) is as follows:

1. Zope Corp. has no intention of cutting any new releases of 2.3.x, or in general any non-current release branch (NoCuRB?).
2. Zope Corp. has no intention of officially supporting the fixing of bugs in any NoCuRB?, except via hotfixes.
3. Anyone with CVS write privileges should feel free to fix bugs in NoCuRBs?. Unless it doesn't make sense to do so, you should be sure that a bugfix is in the current release branch and trunk as well. Please document changes as you would in a current release branch.
4. Adding new features to NoCuRBs? is verboten! Don't do it! New features go in the trunk.

If somebody wants to take on the burden of creating and supporting a release derived from a NoCuRB?, feel free. If anybody is upset about bug fixes/changes that didn't make it into your release, they can whine at you. ZC doesn't have the resources right now to do this right and support it.

Who should or should not merge into the trunk

Q: Ah, okay, so community contributors should never merge to the head? Who should do that?

A: I don't think that's true... for features, there should be discussion before the changes are merged into the trunk and the current release branch. But then you should probably do it after getting permission from the "owner" of the package to which you're making changes.

I think basically what it boils down to is this for Zope core issues (as opposed to CMF issues): If a patch or feature is big and you even suspect that it will have documentation and/or interface impact, you need to get permission from the module owner before checking it in to the trunk or the release branch -- always do your development on a development branch; OTOH, if its a bugfix situation (something that has no interface or documentation impact) where you're not sure that your patch is "right", email the Zope-Coders list to get "direction"; OTOH, if it's a bugfix situation where its just a nobrainer, just check the fix in to both the trunk and the current release branch.

This is basically the way we operate internally at ZC (person and list names changed to protect the innocent).

How to get going on ideas for new features

Here is Tres Seaver's take on the process

1. Write a proposal for the change in the "approved" location (the dogbowl for CMF, dev.zope.org for Zope). Solicit feedback on the change from the appropriate mailing list. Small changes (trivial bugfixes) don't require proposals, but should have corresponding tracker / collector issues.
2. Update your sandbox to use a branch, which should be named to indicate the purpose the branch serves:

           $ cd projects/CMF
           $ cvs tag chrisw-FSSQLMethods-base  # remember our start point
           $ cvs tag -b chrisw-FSSQLMethods-branch \
             -r chrisw-FSSQLMethods-base       # make the branch from it
           $ cvs up -r chrisw-FSSQLMethods-branch # update the sandbox
   

3. Add the file to your branched sandbox:

           $ cd CMFCore
           $ cp /home/chrisw/FSSQLMethod.py .
           $ cp /home/chrisw/test_FSSQLMethod.py tests
           $ gvim tests/test_all.py            # add the test
   

4. Ensure that all unit tests run against the sandbox:: $ cd ../Instance $ ./zctl.py test Products/CMFCore?/test_all.py $ ./zctl.py test Products/CMFDefault?/test_all.py ...etc
5. Commit on the branch:: $ cvs add CMFCore?/FSSQLMethod?.py CMFCore?/tests/test_FSSQLMethod.py $ cvs commit -m " - Add FSSQL methods."
6. Lobby for inclusion in the head (bugfixes and new features)

Lobbying for new features

Lobbying mails should be sent to the "area owner" ([email protected] in the case of the CMF), and should include:

• URLs? to the checked-in versions, using http://cvs.zope.org.
• URLs? of the rationale for the changes.

Branched changes are most likely to be merged if they meet the following criteria:

• Rationale policy was followed.
• The patch is minimal for the purpose (no gratuitous diffs!)
• The patch/feature is understandable.
• The patch/feature doesn't break any existing tests.

Policies may vary slightly based on the area of the site.

From Brian Lloyd

""" Q: I'm not clear whether the recommended way of working in this case is to create a new branch in CVS especially for this fix, or if I should be checking directly into the trunk and branch.

For something this small, it is a matter of personal preference. For people who have gotten branch Zen, it can actually be less typing, less work and less error-prone to use a branch.

Some would rather just make the change in both appropriate places. As long as the fix get into both places and is correct, it's really up to you.

FWIW - I find the easiest way for me to work is to maintain two CVS checkouts: one of the trunk and one of the current release branch. When I fix a tiny bug (say in Folder.py), I fix the bug and update CHANGES.txt in the branch sandbox, do a cvs commit, then cd to the trunk sandbox and merge the change to Folder.py right from the current release branch:

cvs up -kk -j Zope-2_4-branch Folder.py

That merges in the fix I just made (so I don't have to type it again and maybe make a mistake), so then I just commit the Folder.py on the trunk and I'm done.

"""

Tres on CMF and tags """ In the future, I would expect to create tags / branches more in line with Brian's model:

• During pre-beta development (beta represents "feature freeze"), minor bugfixes are checked in on the head. Major bugfixes and new features are developed in activity branches (e.g., tseaver-ttwactions-branch), and merged when ready (all tests must pass on the merged sandbox before checking in the merge).
• At "beta", we create a new base tag and branch tag for the coming release (e.g., CMF-1_2-base and CMF-1_2-branch). No new features may be added on this branch. Releases (beta, final, bugfix) will be made from this branch.
• "Beta" creates a feature freeze for that branch; other feature development may continue in activity branches which will not be merged to the release branch.

"""

Backing out a change

Here's what Brian Lloyd says about backing out a change committed to CVS:

    It's not easy or fun - a good CVS book is probably the best resource 
    there :) You basically would need to manually revert all of the files
    you changed to the version before the change.

    For big changes, it can be a good idea to tag the head or branch that
    you are merging _into_ before attempting the merge (brian-pre_oops_tag).

    That way in the worst case, you can check out a sandbox based on your 
    oops tag and diff files with the screwed files (in another sandbox).

    Luckily, we've been able to coordinate a fairly large number of people 
    in various parts of the world for some time now without having had to 
    do a major painful backout. :)

beacon (Oct 17, 2001 11:49 am; Comment #2)

A little correction for the glossary - the head is really the tip of the trunk, I believe.

beacon (Oct 17, 2001 11:50 am; Comment #3)

A little correction for the glossary - the head is really the tip of the trunk, I believe.