SummarizedVersion

Inherit ZPattern classes when creating ZClass

roche: well maybe i can now ask my first question to the real pje, why can't one inherit from ZPattern classes when you create a ZClass? Is that still a remnant of a zope bug?

pje: Which ZPattern classes are you talking about?

roche: well PlugIn for a start or PlugInContainer

pje: Some aren't registered as ZClass bases because they wouldn't do you much good at present. PlugIn is actually a base as of 0.4.0a5, but doesn't work yet.

PlugIn Registry

pje: The plugin registry is seperate from the regular Zope class registry, so plugins have to be registered from Python

Storing data in different databases

JeffH: I know one of the goals of ZPatterns (well, Racks, specifically) was to allow data to be stored in RDBMS or ODBMS, as appropriate for each application. Is ZPatterns "there", now, or is there more work to be done before that goal is a reality?

tsarna: I think it's a reality now. People are doing SQL with Generic Attribute Providers and Generic Triggers now, and I've been doing LDAP with it the last few weeks. LDAP is a bit more of a pain because the ZLDAP products don't work quite the way they need to. They don't have the power and flexibility of SQL Methods

PlugIns and PlugInContainers

roche: Should a PlugIn always live within a PluginContainer

pje: it doesn't have to. You can have a PlugIn that behaves as a PlugIn when in a container, but as an ordinary zope object when someplace else.

Book pje: The key to ZPatterns is O-O design patterns. Specifically, the work of Peter Coad in the "Object Models" book, and the Business object books by Eeles and Sims.

pje: The what, Chris? ChrisW: containment vs context

PlugIns in the core system

pje: Yeah, especially if I ever get ZClass sheets for PlugInGroups and all that, PlugIns would be a really cool thing to have in the core system.

tsarna: (by "more obviously useful", I mean that they're more likely to be recognized as useful... more people understand the problem they're trying to solve)

Sheet Providers

pje: No time estimate, but implementing a sheet provider is pretty straightforward if you want to do one by hand... In that all you need to do is call on another Specialist to give you an object you can return as a sheet.

But as soon as I get to work on SkinScript, there'll be an easier way. You'll be able to specify sheet attributes as well as object attributes.

Making ZPatterns approachable to the great unwashed masses

pje:

• The first audience is my own development team...
• The second audience is developers in my company...
• The third audience is Zope framework builders
• And the fourth is Zope developers in general

What are predicates

Predicates: objects that represent a logical condition to be used as a filter against some set of objects.

Example:

Attribute(x,LessThan?(3))

tsarna: predicates and the area of searching is kind of on the back burner for now. we have ideas about what we could implement, but we don't yet know enough about what would actually be useful.

Example:

Attribute(x,LessThan?(3)) & (Attribute(y,GreaterThan?(7)) | Attribute(z,Equal(9)))

stevea: how about: anything you'd put in a "where" caluse in an SQL statement?

pje: Possibly more restrictive than that, Steve.

roche: They almost seem like gatemasters?

pje: SQL allows comparison of attributes, but in the simplified predicate models we have done thus far we've focused on constant AVA's (attribute value assertions)

JeffH: predicate: something that is affirmed or denied of the subject in a proposition in logic "m-w.com"

stevea: Will the predicate language be pluggable? If I wanted to do one based on Z Schemas, would I be able to plug it in without rooting around in the code?

pje: That question's a bit premature.

Parrallel matching research

pje: We've been doing some research into what we call parallel matching - a connectionist technique for doing reverse searching... The idea is that when somebody posts an item to a portal, the system can quickly apply hundreds of users' stored search criteria against the new item and then carry out notifications, etc.

pje: We only have a theoretical model, not an implementation design.

ChrisW: Surely the loading on that might not be sensible? What if you have lots of dormant users? You carry out their searches even though they'll never use the results :(

pje: Actually, Chris, the algorithm we've come up with is actually linear relative to the number of attributes which the object to be matched has.

tsarna: Another reason they're on the back burner is that people find the current set of ideas and terms confusing enough, so introducing more now seems bad :)

pje: The only dependency on the other end is how many rules end up firing (i.e., how many e-mails to send or whatever)

pje: And those can be queued, to be sent out for example by our metadaemon system.

Using fromexpr and attrsxprs fields of GenericAttributeProvider

stevea: Can you explain a bit about ways of using the fromexpr and attrsexprs fields of a GenericAttributeProvider?

pje: The idea of having two sets of expressions is based on the idea that attributes often come in sets. For example, an SQL result row or an LDAP entry. So you will often want to retrieve that whole row or entry, whenever any one of its fields is asked for.

pje: The fromexpr is the expression which is called to retrieve that record, whenever any of the attrsexprs is asked for. And then the result of all the attrsexprs are loaded into the DataSkin?'s attribute cache. Unless the result of the fromexpr is NOT_FOUND. In which case, the asked-for attribute is cached as NOT_FOUND, but asking for any of the other attributes will cause the fromexpr to be called again.

pje: (That latter behavior may be a bug or a feature. :) )

tsarna: And asking for any of the attrs maskes them all available. The idea is that the fromattr is probably expensive to computer (means hitting SQL, LDAP, etc), while the attrsexprs are cheap to compute (putt an individual item from the results)

tsarna: so if you have to do the expensive part anyway, might as well make all the attributes availaible since there's a good chance you'll need them later anyway, and you'll save re-running the expensive part

stevea: Thanks. It's sometimes difficult to reverse-engineer the rationale from the code :-)

tsarna: It's a bit clearer in the skinscript version. I'm not sure exactly what syntax we settled on, but it's something like:

FROM expression COMPUTE attr1=..., attr2=...

pje: Two bits at best. :)

tsarna: (hence the name "fromexpr")

what is skinscript?

pje: SkinScript is a declarative scripting language which will let you define attributes, triggers, and propertysheets for a DataSkin. It will be a "Data Plug-in".

pje: You basically add a SkinScript object to your Rack, Specialist, Customizer, or whatever, then fill in what you want it to do.

ChrisW: P&T, do you ever worry that with the sprawling complexity of ZPatterns, people might just not get it and ignore it/ get frustrated by it?

pje: No.

JeffH: I am frustrated. But, I need ZPatterns. I can't afford to give up.

tsarna: Instead of adding a bunch of GAP's, GT's etc, you can add one thing with a nice big textarea, and have several statements in it. Each statement is equivalent to a single GAP, GT, etc.

pje: As with all Open Source projects, this is first and foremost a "scratch your own itch" effort.

Generic Triggers, the fields Execute, Keeping and Set Attrs

Generic Triggers

tsarna: Basically, GT's and GAP's were designed to test out the implementation of those before we have a skinscript parser done. You can think of SkinScript as removing complexity instead of adding it, really :). The idea with the generic trigger is that you want to do something when a condition occurs...

But, in order to know if the condition has occurred, you may need a previous value of some attribute or expression. So, the "keeping" field is a list of names or expressions to snapshot before a change takes place.

Each line is either a "name" (which is interpreted the way names are interpreted in DTML). Or an "expression" (name=expr), where the expr part is interpreted as a DTML expression, and then saved as name

All of the names/exprs are saved in the "OLD" object for use at trigger firing, but they are interpreted before the object's first change or delete event. (From the POV of that DataManager)

At (sub)transaction commit time, the trigger will fire by executing the "execute" expression, passing in OLD, ORIGINAL, CHANGED, and CHANGED_ATTRS.

OLD is the memento of the "keeping" stuff. CHANGED(x) tells you if attribute x was changed. CHANGED_ATTRS() returns a list of changed attribute names. And ORIGINAL is a dictionary of the original values of changed attributes. But ORIGINAL is not necessarily accurate if the original value was mutable.

(Or contained mutable values)

Set Attrs field

Last, the "Set Attrs" field...

pje: One of the most common uses for a Generic Trigger is to handle saving changed attributes in a database. However, you need to tell the DataManager what attributes this service will be provided for, hence the field.

tsarna: Note that you don't have to provide all of that stuff to get something useful. Many of the "clauses" are optional

pje: The Trigger object handles _SetAttributeFor()/_DelAttributeFor() operations by caching the new values in the DataSkin, where your trigger expression can then get them at (sub)transaction commit.

tsarna: A very simple but useful trigger is "UPON DELETE EXECUTE LDAPDelete(self.dn)" (where LDAPDelete is an external method that deletes something)

pje: To amplify on Ty's earlier comment re: complexity is something that we add to ZPatterns regularly, then factor out into simpler approaches and forms as time goes on.

Simple designs are harder, therefore they take more time. We don't wait until we have the simplest possible design to build something, because we will know better how to simplify it after we've made one for real.

stevea: So without skinscript, you can just read left-to-right and top-to-bottom down the Generic Trigger properties.

tsarna: right, Steve

stevea: skipping what is left blank.

pje: Exactly, Steve. We made GAP's and GT's with the understanding that they'd be replaced by SkinScript eventually. But we needed something working first, and it's easier to debug just the working part first. Then when we debug SkinScript, we'll know it's a parse or compile issue. :)

Moving ZPatterns along

stevea: What is the best way for people outside Verio to help move ZPatterns along?

pje: Lobby DC for better encapsulation in Zope internal API's? :)

roche: what about this pattern: write example and doc before release (to quickly evolve zpatterns)

pje: Roche, please understand that although ZPatterns is a labor of love, it's also a labor of paycheck. :)

What doesn't work in ZPatterns

pje:I personally haven't done a thing with it. Ty is my "user" here, and the folks on the list have submitted plenty of bug reports and feature requests for other usage patterns. I'll try instead to point out what doesn't work as well or is not as complete...

• First, security. If you want proxy-ish behavior, you have to do it outside the GAP's and GT's, or roll your own. (But I plan to work on proxy roles support for GAP's and GT's this weekend)
• Second, property sheets, other than generic, persistent ones, stored in the ZODB. I plan to address that once we have SkinScript, by making a SkinScript sheet provider.
• Third, use of objects outside of a Rack... it basically works, but bug reports are still streaming in. We'll be addressing this area more in weeks to come, however, as we need this for an internal app.
• Fourth, local roles, and containment of non-ZODB objects inside of ZODB or non-ZODB objects.

Application of ZPatterns

JeffH: Phil, what kinds of things have you actually done with ZPatterns so far? So I can get a feel for what it's ready for, and been used for, and been debugged for? :-)

tsarna: I've mainly done two things with it.

• First, I've used ZPatterns in the LoginManager guise to have sites with user logins coming from LDAP.
• Second, I've been building a web-based editor for an LDAP database we have. The LDAP editor is using the ZPatterns database independance to do LDAP, but it's not really a database independant app

Neither of those is a great example, though

Extending LoginManager

roche: How would one go about extending properties for a user with LoginManager

pje: Add Data Plug-ins to the user source

roche: But I can't do that with a ZClass

pje: True. As of alpha5, however, you can merge DataSkin with LoginUser to create a ZClass that will allow it. (Ty: we should probably make _ZClass_for_LoginUser support that also)

Rack brain objects persistently

sRp: Something to do with zpatterns: create a Brain object, then Rack Brain persistantly

JeffH: As in a property sheet whose values are stored in a SQL database?

tsarna: You can use the secret, undocumented "User Source" user source, which takes Data Plug-Ins.

Other applications of ZPatterns

stevea: I'm using it to support a Product Line Infrastructure -- a bunch of related applications that reuse bits of each other but in differnet contexts. However, I've only got the first app so far :-)

JeffH: We need to build a workflow system. We need to build a content management system. We need to build a customer relationship management system. I only want to build each one once. They must, at some point, cooperate with each other depending on the context in which they are being used.

roche: we also need to build a customer relationship system

rdmurray: We could probably use a community collaboration on a CRM project.

SWARM

pje: ZPatterns is the basis for another system Ty and I will be building, called SWARM... State-based Workflow And Resource Management So, it is probably a good match for other workflow apps.

stevea: I've seen that acronym somewhere inthe Wiki I think. Will SWARM be open source or verio internal?

pje: Open source.

tsarna: (pje and I are arguing as to weather propertysheets on users already work or not... hold on :)

pje: The idea is that there will be Plans, which have Phases, Events, and Roles. Then "ObjectsWithPlans?" will use the plan as a kind of state-based attribute provider. You can actually do state-based stuff now, using just GAPs.

jean: I'm champing at the bit to show off something like SWARM at work. Stuff like Journyx and an inhouse issue tracker are being pushed, and I feel they're steps backwards. We're going to have to zopify some anyway, better to invest more in it, not disperse across other cruft. But what's available now isn't quite for prime time and i'm not the one the fix it :(

pje: Okay, I've checked in a change so that LoginUser can have DataSkin property sheets. (And thus, attribute providers)

public cvs

tsarna: security concerns

pje: Perhaps we should look into setting up a sourceforge project, and do it as SWARM, with LoginManager and ZPatterns as components. Then we won't have to make more than one, or change later.

mindlace: Host it at digicool? basically it's part of the fishbowl process

Simple working example

msegarra: Hi, all. Please, anyone can explain in detail how to get an very very Simple working example say: storing a ZClass Person in ZODB and using a Specialist/Rack combination all done from Zope management interface, I mean no python !!

pje:

• First, make your ZClass, with "DataSkin" as the first base class.
• Second, add a Specialist, with a Rack.
• Third, go to the Rack's management interface, select the "Storage" tab, and select the ZClass you want to use.

That's it. Call "getItem(key)" to retrieve an item from the Specialist, and "newItem(key)" to create a new item in the specialist.

To delete an item, call its manage_delete() method.

From DTML, you'd probably say >dtml-call "specialist.newItem(key)"<

for example:

The "key" becomes the "id" value of the created object.

tsarna: You can also go to it through the web: .../specialistname/key

stevea: Also, you can access the objects through URL: if your specialist is called /specialist/ and you called newItem(one) on it, you can get to object one with /specialist/one

pje: Whoops, I misspoke on that example... You probably want >dtml-let newobj="specialist.newItem(key)"< so you can do something with the new object. :)

msegarra: good, is right to add a Person via button Add ?

Shane and others have proposed adding interfaces to make the contents visible/manageable.

msegarra: thanks a lot, I am unsderstanding a bit now ...

Simple example but in Python

roche: Do you do the second step also in python?

tsarna: If you want to use a python class as the thing to store in the rack, you can do that too, if that's what you mean?

pje: It just has to be registered as a ZClass base.

tsarna: You just have to register it as a ZClass base in order to be able to pick it on the Storage tab

pje: No, msegarra. The "Add" button would only add it to the specialist's Methods, not its contents. The contents of Racks are not visible through the Zope management interface. At least not yet, anyway. :)

roche: are there any ZPattern classes registered as ZClass bases?

tsarna: Yes. DataSkin. Also PlugInBase?/PlugInContainer, Specialist, and Rack. How useful it is to subclass them, I dunno

roche: I can't see it? But i also couldn't do the Makefile part of the installation

tsarna: Roche, if you can't build the DynPersist module, ZPatterns won't work at all.

stevea: not even in 2.2? goes to look at code

Ty: it isn't very useful right now for anything but Specialist and DataSkin.

pje: That's right Steve. We haven't abandoned 2.1.x yet, and until then we can't get rid of DynPersist, even for 2.2.

roche: i can't build the DynPersist module - lost with Makefile.pre.in part

• When you type make -f Makefile.pre.in boot what do you get?
• what platform are you on?

tsarna: (both from a "does it work" and a "what for" perspective)

Ran out of time here. Some one else please continue.