Motivating Organization Objects

I haven't done a great job clarifying the point of organization objects. I think what brings them home for me is the actual example that inspired them in the first place. In case it would help clarify the point for anyone else (and anyone else is interested), i'm going to sketch that out.

The crux for me was in the parent/child lineage relationships i added to ZWiki. I imlemented it by putting metadata about a page's parents in the page. I did it this way, instead of adding child info in the parents, because the "ancestry" info is shown on every view of a wiki page, and this makes it easiest to compute.

Either way, there's a lot of overhead in computing a wiki's comprehensive table-of-contents view(*). Not only is this topological sort complicated to recompute, but every page in the wiki has to be visited in order to determine it. Even determining a page's ancestry info, done on every visit to the page, entails examining up all the page's ancestors to determine their ancestry. This can mean a lot of unnecessary waking of ZODB ghosts.

Most of the repeated overhead could be avoided if this lineage map of the wiki were incrementally updated. Simplest would be to update the collective children, parents, and roots info (step 1 in the (*) footnote) as the nodes are changed. This would make the most commonly used information trivially available, and both that and the topological sort could be done at any moment without examining other pages.

This incrementally updated third-party object is a major start towards organization objects.

From there, consider that other wiki organizational aspects would be well served by similar arrangements. For instance, backlinks are currently assessed in ZWiki (and WikiForNow? and CMFWiki?) by going to every other page in the wiki and looking for wikilinks that refer to the target page. Yuck! Instead, some backlinks registry could be updated when each wiki page is changed, and referred to when a page's backlinks are requested - without examining other pages. RecentChanges is another such organizational relationship that would profit from this kind of treatment.

The clincher is that ZWiki search and jump-to mechanisms are done with similar, brute-force searches. These are things that should be done using the catalog, but doing that was problematic (due to implementation details and bugs).

So, the stage was set:

There's a number of types of page associations that were begging to be tracked in third-party registries, to be updated when the associations in one page and/or another changed.
One of those associations is very catalog-specific - search through the pages.
It would be handy to be able to unify the results of searches within the organizations across associations - for instance, the most recently changed pages within a certain portion of the wiki's lineage nesting.
As indicated in the OrganizationObjects? primary spiel, it would also be quite handy to be able to federate such queries. Eg, show me the RecentChanges ordering across a bunch of wikis - or even better, RecentChanges across a bunch of different content types, in a bunch of different places!

This set of features conspires to suggest my proposal:

third-party objects which maintain different kinds of content-organizational information
revised incrementally when one of the subjects changes,
and plugged into an opened-up catalog architecture, in order to be able to unify the result sets with other catalog queries.
An optional second query interface is needed for specialized results orderings, eg the lineage topological sort.

(* To compute the table-of-content view:

All the pages in the wiki are visited to collect a mapping from children to parents, and from parents to children, and to identify all the root nodes - those with no parents of their own. 2. Then the children are systematically descended from the roots to determine the exhaustive nesting relationships, with revisit-checks to prevent infinite regress on nesting loops.

I think this can be referred to as a topological sort.)