this is made by a python newbie, and zope in the middle oldbie
:)

Please email updates, comments, requests, "hello world" to me.  I'll work on KebasData if
there's interest from the community.  Otherwise, it will remain as is.

KebasData will get a url, and parses the page for the pattern you specified.  If there's no start and end pattern, the whole page will gulped.  If start and end pattern is specified, part of the page will be used.

To use existing Kebas Data instances from Zope2.3.x in Zope2.4.x, we need to
reset all Kebas Data instances before exporting it.  

Put this DTML Method in the Zope root.

######DTML Method##########
<dtml-in "ZopeFind(this(),obj_metatypes(['Kebas Data'],search_sub=1)">
<dtml-call reset>
Reset <dtml-var id_or_title>
</dtml-in>
################################
To get data daily, you can setup crontab.  or use Xron.
Use get_matched() method

From v0.0.1a3 onwards, you can set refresh limit in seconds.  That way, after x seconds, any new view by clients will trigger object updates.

I would like to compile all the URLs and regex patterns used by everybody, and include it with the tar ball.
so, mail me your URLs and regex patterns, and I'll include it with KebasData.tar.gz

This is areally an exercise in making a Zope Product, and pointers, comments, good coding style, etc 
are requested.

The lowdown
------------
-Url is the url you want to retrieve
-pattern is the regular expression pattern that will match data in the page
-start pattern is a unique string in the page that will determine your start location.
if left blank, it will be the first character in the page.
-end pattern is a unique string that determines your end location. if not specified, it will be the last character in the page.

Note, if you define start pattern, you *must* define end pattern.

-render_method: a dtml method to associate rendering of matches.

if you have a DTML Method with id "use_this", then put use_this in
render_method field.
Checkout the userfriendly example.

-request method; either GET or POST
-parameter list: a per line key and value.
e.g : suppose we want to supply name=kedai, character=real macho, then in the parameter list put

----start parameter list text area----
name
kedai
character
real macho
-------end---------------------------
all url encoding will be taken care off.

in a GET method, no key:value pairs will be looked up.  use query strings.
I'm not sure whether we can get the key:value pairs from query string.  is there any difference?  
I may include that capability if many wants it.

Also, checkout the exported demo/example folder.
-copy dll.zexp into zopedir/import
-start browser and go into management view.
-click root folder, or any folder you want.
-import dll.zexp
-you may need to initialize all Kebas Instance first.

starting with KebasData-0.0.1a2, you can pass url to get_matched(), and do stuff with the results.

e.g, to get daily userfriendly strip
url: userfriendly.org/static
pattern: <img.*?>
start pattern: <!--Start Current Strip--> 
end pattern: <!--End Strip-->
render_method: show

somewhere in your acquisition path, create a DTML Method call show with this codes inside:

---------------start of show-----------------
<dtml-var standard_html_header>
<dtml-var match>
<dtml-var standard_html_footer>
---------------end---------------------------

You can then access "http://server.com/userfriendly"
or <dtml-var "userfriendly.view()">

get daily dilbert strip
url: http://www.dilbert.com/
pattern: <a href=(.*?)>(.*?)</a>
start pattern: <!--COMIC STRIP BEGIN-->
end pattern: <!--COMIC STRIP END-->
render method: dilbert_method

create a DTML Method with id dilbert_method
-----------------start----------------------
<img src=<dtml-var "_.string.replace(match[0][0],'/comic','http://www.dilbert.com/comic')">>
--------end---------------------------------

you can access the strip at http://server.com/dilbert/
or <dtml-var "dilbert.view()">


getting top news from cnn.com/CNNI

url: www.cnn.com/CNNI
pattern: <a href=(.*?)>(.*?)</a>
start pattern: <!-- Profil
end pattern: <!-- FN and SI Content --> 

will give you a list of tuples.  you can then access any tuples or items from a dtml-method.
<dtml-var expr="myid.match"> will give all items grabbed/matched.
<dtml-in expr="myid.match"> will iterate thru list
<dtml-var expr="myid.match[0][0]"> will give the first group match.
etc


This view helps determine how you can manipulate data retrieved. 
You can have a list of items, or a list of tuples if your regular expression uses grouping.
e.g:
	<img.*?> will return a list of data
	<img(.*?)alt(.*?)> will return a list of tuples.

You can access data as you would any list.