BarryWarsaw

Just some quick notes to capture some thoughts.

Background: I'm currently leading the development for Mailman a mailing list manager written Python. Mailman has a bundled archiver called Pipermail, which is really nice in that it mostly works, is written in Python, and gives users a half-decent archiver out of the box.

Pipermail however has lots of problems: its code is very difficult to maintain, it doesn't handle more complex email types (i.e multiparts or attachments-- but MM2.1 will be better!), it's not very efficient, and it doesn't support searching, or more advanced forms of collaboration.

My short term goals would be to design a new archiver based on Zope, centrally modeled around an email message type.

A quick summary of features I want:

I want to throw all the messages into a big bucket and construct views on the archived messages on the fly. I don't like Pipermail's artificial separation of indexes into time-based buckets. Too many threads span multiple months.
Everything's done through the catalog. E.g. want to find all the postings by me in the last 3 months? Want to follow the entire foobar thread, even if it started 6 months ago? We should have canned catalog searches that mimic current behavior, but we can be much more flexible too.
Full-text searching. Need I say more?
weblog. I'm just starting to learn a bit more about weblogs and blogging, but I already know that I want followups and content to be addable from many differnt channels. They should crosspost, much like Mailman's current news<-->mail gateway.
post via web. Maybe this is weblog? I want a button for anybody reading the archive via the web to be able to post a followup or reply to the current message. I also want them to be able to start a new thread via the web.
forward via web. See an article you like? Or one your friend likes? Fill in your email address (or not, if you're already logged in) and hit the forward button. Note: this introduces an abuse vector, i.e. forward bomb people you don't like. Possible mitigating factors include requiring a logon, message/rfc822 encapsulation daily limit on forwards to an email address.
Note: you want to be able to forward a digest of interesting messages in one swoop, but see above.
Dang it would be cool to provide an nntp front end to the archived messages. Some people like reading discussions via email and/or digests, some like the web some like news. We could (and should) cater to all.
attachment archiving. All kinds of funky stuff comes in email. we should strip out attachments of known types, compare them to various filters (e.g. application/octet-streams get chucked), maybe pass them through virus checkers. This needs a back door to the delivery process (what I call the "m-and-m", i.e. moderate-and-munge) so that we could e.g. strip out large image attachments from messages exploded to the membership, replacing them with a url to the archived attachment.
url-based digests. Mailman has two types of digests currently, RFC 1153 and RFC 934. Both contain the full messages in their digests (with perhaps some header trimming). An alternative offering should be a summary digest which just contains urls to the articles in the archive.

Okay, so I admit I don't have a very lofty vision, but I still think this stuff would rock!

Side note: I have other areas of interest for integration, or leaching onto Zope technology. I want to be able to integrate membership databases with Zope (one membership in Zope, and you manage your list memberships through it). This would actually be possible with Mailman 2.1. I also want to explore the use of page templates to replace Mailman's current extremely ad-hoc web gui (some hardcoded scripts, some homegrown templating).