Planet CherryPy

The next MichiPUG meeting will be on Thursday, July 2nd at 7PM. Ryan Burns will talk about the Impressive presentation software (which was used at Tuesday’s Ignite Ann Arbor) and Terry Howald will talk about Python’s use in scripting visual effects.We may also talk about the Python 3.1 release.

The meeting will be in downtown Ann Arbor at the SRT Solutions office.

I was listening to my beloved Asa today (Up Nigerian soul!) "Fire on the Mountain", an incredible monument of a song:

So you say you have a lover and you love her like no other
So you buy her a diamond that someone has died on
Don't you think there's something wrong with this?

Hey Mr. soldier man, tomorrow is the day you go to war
Boy you are fighting for another man's cause and you don't even know him
> 


 
Ooooh!
> 


 
What did they say to make you so blind to your conscience and reason?
Could it be love for your country or for the gun you use in killing?

Permalink | Leave a comment  »

In my poetical wandering over the weekend I ran across Karl Shapiro's "The Tongue". He starts by getting the conceit all wrong, and even though it bears the execution of a fine craft piece, the result comes off a bit of a mess.
> 


 
As a slug on the flat of the sun-heated clay, With the spit of its track left behind it like glass, Imperceptibly voyages, licking its way In the sinuous slime of itself to the grass,   So my tongue on the white-heated wall of your thigh Licks its belly across, and the path of my slime Lies in ribbons of passion, the wet and the dry Overlapping to mount to the leaf of its climb.   And the mouth and the mouth and the tongue and the tongue And the fishes that feed in the joy of our oil And the slug of our wetness finds green food among The hair-forests of longing where serpents uncoil.
> 


 
You can see how the cleverness dampens the sense, something I often struggle with myself. This is a large part of the reason why Shapiro, despite his technical skill, has never been as celebrated as he should be. He tries to use a sprinkling of words ("passion", "longing") to mend the detachment of the conceit of the slug, which could never hope to transport the idea of a tongue inching towards cunnilingus.
> 


 
The piece pretty much cries out for a rival metaphysical poet's response.  And it should serve as a lesson to me.

Permalink | Leave a comment  »

Turning a story in text form into an animated movie is a long and complicated procedure. [...] many parts of this process could be automated by using artificial intelligence techniques [...] So we decided to explore the possibility of a generation process of computer animation from a childrens story in natural language text form to the final animated movie.

 Bookmark this on Delicious - Saved by chimezie to ai theory logic literature semantic - More about this bookmark

Open source Python modules, linguistic data and documentation for research and development in natural language processing, supporting dozens of NLP tasks

 Bookmark this on Delicious - Saved by chimezie to python nlp code programming linguistics - More about this bookmark

"I've written a textbook entitled 'Machine Learning: An Algorithmic Perspective'. [...] There are lots of Python code examples in the book, and the code is available here." Yummy

 Bookmark this on Delicious - Saved by chimezie to code python machine-learning programming - More about this bookmark

So, I've started a part-time Ph.D program at the EECS department of
Case Western University. I'm hoping to write a thesis on higher
education information science as patient advocacy. I've taken a
course on Machine Learning, database systems, and now on model theory.
 Luckily Case has a class on model theory in the philosophy
department. But it also counts as credit from the graduate
mathematics department.
> 


 
The class on model theory is about Kurt Godel's (I know I'm
misspelling it) theory on the incompleteness theory of peano's
arithmetic ( an axiomatization of number theory ). The book for the
class, It is supposed to be written by mathematical geniuses, our
professor proclaims. The book is ".."
> 


 
Models are states of absolute (binary) logical certainty: things that
are or things that aren't. Statements in this language are 'sound'
(they follow from our understanding of logic). The consist of
domains: sets of 'things' denoted by terms in a written language (a
first-order language). The model also consists of 'interpretations'
of phrases (or formulas) in this language, some of which are 'closed'
(i.e., all variables refer to constants interpreted through this
model) or open. The axiomatic nature of basic science presumes
logical certainty in its canon. The language consists of constants
that can be interpreted 'against' the domain of a model, sets of
constants composed form constants interpreted against the domain of
the model, functions that map sets of members of the domain to members
of the domain, 'relations' over the domains, and a determination of
'equality' over members of the domain. Model theory is the basis of
first-order theory, logic-based knowledge representation, numeric
theory, etc..
> 


 
Godel's theory says that any axiomatic system that is as expressive as
some computable representation of number theory is not complete (i.e.,
there is at least one question you can formulate in the language for
which you cannot say with certainty that it is so or it isn't so).
> 


 
Models 'entail' sets of formulas when they entail all the members of
the set. Models satisfy sentences and 'terms' in languages. Some
formulas can be said to be valid in every model (i.e., they are
satisfied by every model of a language) or they specifically are
entailed by a particular model. Axiomatic machinations (finite state
automata, etc..) are systems about languages and a set of axioms that
can be used to 'derive' expressions in the language via a finite set
of valid theories, specifically given sentences in a theory, or
sentences that follow from modus ponens (common sense if / then
conditionals) and sentences in a theory (a scientific theory).
> 


 
As our prof puts it, all of logic programming (and database theory) is
spawned by this known limitation to model theory and first-order
logic. They are restricted forms of it that are complete (unlike
number theoretic languages), and sound and thus 'decidable' by a
finite state automaton or turing machine.
> 


 
Systems that describe their 'formal semantics' - the meaning of
statements made in their languages often describe them using model
theory (all of semantic web theory does this: RDF, RDFS, OWL-DL,
etc..). More on this later

Permalink | Leave a comment  »

We've been using SPARQL, OWL, and N3 lately to prototype the reporting
of common research variables to the Society of Thoracic Surgeon's
(STS) National Database. The reports are being run against our large
RDF dataset of abstracted electronic patient records from the
Cleveland Clinic's Electronic Health Record system. Our dataset
consists of about 200,000 patients each represented as statements in
named RDF graphs. The STS variables we are responsible for deriving
are represented using a combination of OWL-DL and Notation 3. The
constraints that do not benefit from the restricted, tree-like nature of description logic are captured using secondary plain Horn clauses
(or rules) represented in Notation 3.
> 


 
We use an open source logic reasoning system for the semantic web that
converts the constraints and a SPARQL query for an RDF dataset
governed by these OWL-DL constraints into provably optimal sets of
rules used to calculate an entailed RDF graph (the specifics of this
method is a subject of a paper I'm working on for the RuleML 2009
conference). Such an entailed RDF graph can then be targeted with
SPARQL queries to answer for the STS variables. A recent challenge
has been to try to capture the semantics of negation in order to
implement 'exclusion criteria'. This is typically of the form of a
class of procedures that do not involve combinations of one or more
kinds of other procedures. A recent update to FuXi includes the
ability to convert OWL-DL expressions that use owl:complementOf into
general, stratified, logic programs that can be evaluated using SPARQL
in order to implement the semantics of stable model negation (which is
quite different from the way owl:complementOf is meant to be
interpreted: according to the negation of first-order logic).
> 


 
In particular, statements of classical (first-order) negation are
making assertions about the lack of existence of models that satisfy
the positive form of such a statement in a theory. I prefer this
explanation to the way the term 'open world' assumption is often
used to describe this interpretation of negated terms in a
description logic language. Database theory, ofcourse, does not interpret negated terms
in this way, but instead (intuitively) understands statements of negated terms to be
'true' if the (ground) positive form is not in the set of known facts (the
database).

Our use of negation, and the nature of knowledge
recorded in a computer-based patient record system seems (so far) to
lend itself more to the database interpretation where there is an
understanding that a curated medical information system would have its
data entered under the governance of policies that would allow
medically useful inferences to be made from the absence of certain
facts about patient care.
> 


 
In particular, if a fact is known to not be true about a patient or
some activity involving a patient, it is not recorded. This common
understanding can be used to make inferences about whether facts in a
patient record satisfy an exclusion criteria. Below is an example of
this:
> 


 
Consider the following OWL descriptions of a class of operations:
 

SubClassOf(
 IntersectionOf(
  Operation
  PostOpInHOspitalEvent
  ObjectAllValuesFrom(
  involves
  ComplementOf( UnionOf( CardiacProcedure ThoracicControlBleeding ) ) )
 )
 sts:ReopForOtherNonCardiac )

The syntax above is the OWL2 functional-style syntax. We can
paraphrase the general class inclusion (GCI) axiom above as saying:
".. all operations that followed another operation and do not involve any 
cardiac procedures or thoracic control bleeding procedures." 
" 
The original documentation for this variable in the STS adult cardiac
database manual says: 

Indicate whether the patient returned to the operating room for
other non-cardiac reasons

Now, if we assume that all operations of interest and the involved
procedures are explicitly recorded in our patient RDF dataset. This
general class inclusion axioms can be reduced into a set of rules that use negated
'literals' (as they are called); understood to capture the semantics of
default negation (or the 'closed world assumption').  It is worth noting that this is exemplary of a class of expressions that description logic, tableaux-based reasoning algorithms often have problems with.

Conjunctive query answering for stratified datalog is a well-studied class of
problems in database theory. It is through the insight of this canon of theory that FuXi is now able to reduce
OWL-DL expressions that use owl:complementOf into sets of rules (or
logic programs) that can be efficiently processed in order to
implement SPARQL entailment regimes for combinations of OWL and
rule-based representations for the semantic web such as
Notation 3 or RIF core.
> 


 
The current FuXi implementation converts the GCI into the following
two RIF rules:

Forall ?X ?QrjeKHuq961 (
?X # sts:ReopForOtherNonCardiac

  :- And(

    ?X # PostOpInHospitalEvent
    ?X # Operation,
    Naf ?X[involves -> ?QrjeKHuq961] ) )

Forall ?X ?QrjeKHuq961 (
?X # sts:ReopForOtherNonCardiac
  :- And(

    ?X # PostOpInHospitalEvent ,
    ?X # Operation,
    ?X[involves -> ?QrjeKHuq961],
    Naf ?QrjeKHuq961 # CardiacProcedure,
    Naf ?QrjeKHuq961 # ThoracicControlBleeding ) )

 Note, Naf is in the (current) 30 July 2008 version of the "RIF
Framework for Logic Dialects" 

The first rule describes members of the clas of ReopForOtherNonCardiac as those post-operative operations (i.e., operations that follow another operation in the same patient hospital visit or episode) that do not involve other procedures.

The second rule applies to those post-operative operations that do involve other procedures where these other operations are not either cardiac procedures or thoracic control bleeding procedures.

These RIF rules can be exchanged with other RIF-compliant rule-based
systems that implement any of the well-accepted semantics for negated
formulas in horn clause logic (stable models, well-founded models,
stratified models, etc.). A recent modification to FuXi makes
use of a programmatic SPARQL interface for Python that a colleague of
mind has been working on called telescope. It works with
rdflib (same as FuXi) and is used as a control layer that converts
negated RIF rules into a series of SPARQL queries involving
OPTIONAL/FILTER/!BOUND that are used to calculate "stratified models"
(i.e., the finite set of facts that can be inferred from the set of
rules that include negated literals).
> 


 
Renzo Angles et al. (2008) and Polleres, A. (2007) have since
demonstrated that the expressive power of SPARQL coincides with that
of datalog with negation, so it comes as no suprise that certain
datalog clauses (or rules) can be converted into SPARQL queries using
so-called copy-patterns and the introduction of a MINUS operator. For
the details of how this operator works and how its semantics are
equivalent with that of datalog, the reader is urged to read any of
the above mentioned papers.
> 


 
telescope is used to programatically convert MINUS operators into a
SPARQL queries that answer for RIF rules with the corresponding
negated frame formulas below:
 

SELECT ?X
WHERE {
 ?X a PostOpInHospitalEvent .
 ?X a Operation
 #The post-operative operation does not invlolve any procedures
 OPTIONAL { ?X involves ?QrjeKHuq961 }
 FILTER (!bound(?QrjeKHuq961))
}

SELECT ?X
WHERE {
 ?X a PostOpInHospitalEvent .
 ?X a Operation .
 ?X involves ?QrjeKHuq961
 #In the case where the post-operative operation involves a procedure
it is *not* either a
 # cardiac procedure or a thoracic control bleeding
 OPTIONAL {
  ?QrjeKHuq961 a CardiacProcedure .
  ?QrjeKHuq961 a ThoracicControlBleeding .
  ?QrjeKHuq16542 a CardiacProcedure .
  ?QrjeKHuq16542 a ThoracicControlBleeding
  FILTER (?QrjeKHuq961 = ?QrjeKHuq16542)
 }
 FILTER (!bound(?QrjeKHuq16542))
}

I'll be adding a wiki shortly (on the python-dlp google code wiki)
describing the explicit APIs that can be used for this purpose, but I
wanted to give the feature some context in the recent work I've been
doing on applications of semantic web for medical informatics
> 


 
-- Chimezie

Permalink | Leave a comment  »

Okay. I think it needs to be said that RDF / OWL tool-makers do not
do a good job of respecting the mechanics of xml:base . It seems
almost a given that most of them mangle the URI base resolution
conventions they find in existing RDF/OWL files, overwriting them with
their own. Often, this happens in a very destructive way that can be
even more annoying for a person who has much experience using XML and
thus has a better appreciation (perhaps) of the value of a base URI of
an OWL document for more than just owl:Ontology/@rdf:about.
> 


 
This is the typical scenario: I create an OWL document like so:
> 


 
<rdf:RDF>
 <owl:Ontology rdf:about="">
   <owl:imports rdf:resource=".. relative path .."/>
> <owl:Ontology>
 </owl:Ontology>
 <owl:Class rdf:about="tag:info@example.com#Stuff"/>
"/>
</rdf:RDF>
> 


 
First, let me describe the intent here. I think it is perfectly
reasonable to modularize large ontologies into fragments (if you will)
that are bundled together and linked via relative imports. Since they
are bundled together, I often like to use the power of URI resolution
to make relative references to imported OWL documents (as you can see
above) in a way that is completely independent from where the bundle
is being deployed (file system or on the web).
> 


 
The downside of not using xml:base explicitly is that the URIs of my
classes need to be fully qualified, but as far as I'm concerned this
is outweighed by the advantage of knowing that I can deploy my bundled
import network on a website or on a filesystem and have any
self-respecting tool know how to resolve the relative URIs.
> 


 
One of the more involved technical points during the development of
GRDDL was regarding the use of empty URI references in GRDDL results.
The primary importance of empty, relative URI references (in RDF serializations) is the
ability to refer to the containing document without necessarily having
the URI on hand. It was this particular dialogue that made me better
appreciate the power and simplicity of the URI base resolution process
that sits at the bottom of many of the important W3C specifications.
> 


 
From RFC 3986 (the process of determining the Base URI to use when
resolving relative URIs):
> 


 
* The base URI is embedded in the document's content.
* The base URI is that of the encapsulating entity (message, document, or none).
* The base URI is the URI used to retrieve the entity.
* The base URI is defined by the context of the application.
> 


 
Getting back to my example above (this was supposed to be a short
rant). Too often what happens is that when I load an OWL document
such as the oneabove into an OWL editor and save it (even after not
making any changes), it results in:
> 


 
<rdf:RDF xml:base="tag:info@example.com#">
">
 <owl:Ontology rdf:about="">
   <owl:imports rdf:resource="file:///path/to/OWL/document"/>
> <owl:Ontology>
 </owl:Ontology>
 <owl:Class rdf:ID="Stuff"/>
</rdf:RDF>
 
<insert appropriate expletive>
> 


 
Now, it's not so much the 'forced' use of rdf:ID that bothers me as
the fact that the relative paths used for the imports are now replaced
with absolute URIs. This sabotages the advantage I once had of being
able to rely on the URI resolution chain to be sufficient for clients
that need to resolve my relative URIs. Essentially, the OWL tool has
monopolized the opportunity to resolve relative URI references.
> 


 
Now, in fairness, it seems much of the motivation of doing this is to
associate an explicit base URI with the ontology itself and often
ontology tools will complain if they are unable to determine an
absolute URI to use for this purpose. I think a better mechanism for
doing this would be explicit attributes or elements rather than
bastardization of the ability to give an explicit URI base in content.
> 


 
It seems to me that if the author wanted to associate an explicit URI
with the ontology he or she would use one in the place of an empty,
relative URI reference. In fact, I would go as far as saying that it
is bad practice to forcibly insert an @xml:base in a document that
doesn't have one *and* uses empty, relative reference!
> 


 
So, for any developers of RDF / OWL tools, please take care in trying
not to enforce a base resolution scheme despite the document author
providing one of their own for reasons that are orthogonal to
associating a URI to an ontology (and more important as far I'm
concerned): deployability.

Permalink | Leave a comment  »

Parking

via xkcd.com on 3/29/09

Permalink | Leave a comment  »

Zattoo’s Marshall Weir will be talking at this week’s MichiPUG (Thursday evening at 7PM at SRT Solutions in downtown Ann Arbor). In his own words:

I’ve been working on a python module for running reports in Hadoop. Its sort of a wrapper around the pig data processing language and some smarts for running reports on a hadoop cluster and pushing and pulling data to it. It’s designed primarily to make it easier and more efficient to run complex sets of interdependent reports - I’ve been using it to do business reporting on our customer behavior at Zattoo.

This should be very interesting for folks like me who have never seen Hadoop in action!

Toshio Kuratomi's How to Build Applications Linux Distributions will Package. As a web framework dev, this was priceless.

The next MichiPUG meeting will be on Thursday, May 7th at 7PM. The topic for this week is newfangled non-relational databases (with demos of Couch DB, Mongo DB, Tokyo Cabinet, Persevere, and Redis promised)

via Welcome - Michigan Python Users Group | Google Groups.

As usual, the meeting will be at the SRT Solutions office in downtown Ann Arbor.

For PyCon 2009, I'm giving two talks! One on extending CherryPy and one on the innards of Dejavu/GeniuSQL. I think I've finally reduced my talks to the required time slots (I could easily have made 4-hour talks for each and posted my presentations:

• http://www.aminus.org/rbre/pubs/pycon/2009/cpintro/cpintro.html
• http://www.aminus.org/rbre/pubs/pycon/2009/djvlinq/djvlinq.html

Use the arrow keys or mouse-click to proceed through them. The images don't load as fast over the network as they will when I present, so be patient if you preview them yourself. Also, try to use 1024 x 768 fullscreen--they're laid out specifically for that resolution.

Update: video is now available thanks to the great people who put on PyCon:

• CherryPy: http://blip.tv/file/1957163/
• Dejavu: http://blip.tv/file/1949237/

Given a class defining one or more ordering methods, this decorator supplies the rest. This simplifies and speeds-up the approach taken in recipe 576529.

 Bookmark this on Delicious - Saved by chimezie to recipe python code comparison - More about this bookmark

"Pipe viewer is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion."

 Bookmark this on Delicious - Saved by chimezie to article tool unix console - More about this bookmark

At long last, I’ve released Paver 1.0. Here’s the announcement that I sent to python-announce:

After months of use in production and about two months of public testing for 1.0, Paver 1.0 has been released. The changes between Paver 0.8.1, the most recent stable release, and 1.0 are quite significant. Paver 1.0 is easier, cleaner, less magical and just better all around. The backwards compatibility breaks should be easy enough to work around, are described in DeprecationWarnings and were introduced in 1.0a1 back in January.

Paver’s home page: http://www.blueskyonmars.com/projects/paver/

What is Paver?

Paver is a Python-based software project scripting tool along the lines of Make or Rake. It is not designed to handle the dependency tracking requirements of, for example, a C program. It *is* designed to help out with all of your other repetitive tasks (run documentation
generators, moving files about, downloading things), all with the convenience of Python’s syntax and massive library of code.

If you’re developing applications in Python, you get even more… Most public Python projects use distutils or setuptools to create source tarballs for distribution. (Private projects can take advantage of this, too!) Have you ever wanted to generate the docs before building the source distribution? With Paver, you can, trivially. Here’s a complete pavement.py::

    from paver.easy import *
    from paver.setuputils import setup
 
    setup(
        name="MyCoolProject",
        packages=['mycool'],
        version="1.0",
        url="http://www.blueskyonmars.com/",
        author="Kevin Dangoor",
        author_email="dangoor@gmail.com"
    )
 
    @task
    @needs(['html', "distutils.command.sdist"])
    def sdist():
        """Generate docs and source distribution."""
        pass

With that pavement file, you can just run “paver sdist“, and your docs will be rebuilt automatically before creating the source distribution. It’s also easy to move the generated docs into some other directory (and, of course, you can tell Paver where your docs are stored, if they’re not in the default location.)

Installation

The easiest way to get Paver is if you have setuptools_ installed.

easy_install Paver

Without setuptools, it’s still pretty easy. Download the Paver .tgz file from Paver’s Cheeseshop page, untar it and run:

python setup.py install

Help and Development

You can get help from the mailing list.

If you’d like to help out with Paver, you can check the code out from Googlecode:

svn checkout http://paver.googlecode.com/svn/trunk/ paver-read-only

You can also take a look at Paver’s project page on Googlecode.

I’ve just released Paver 1.0b1. This new release adds a fun little feature. A typical setup.py script looks like this:

from distutils.core import setup
 
setup(name="Foo", ...)

With the new version of Paver, you can now rename setup.py to pavement.py (or run paver -f setup.py) and then do:

from paver.setuputils import setup
 
setup(name="Foo", ...)

That gets you going quite quickly, doesn’t it? Of course, you’d likely want to add from paver.easy import * and to start making tasks that take advantage of Paver.

1.0b1 includes some other bug fixes, brings back the call_task function (particularly useful for distutils tasks), and makes the help output more consistent.

There’s one more bug on my list to fix (the distutils output is not showing up), and then I need to rework the docs for Paver 1.0.

Update: It occurred to me a bit later that my example above doesn’t work, because you need to call install_distutils_tasks to get Paver to pick up all of the distutils/setuptools commands. However, this will be fixed in rc1.

I have just released Paver 1.0a3. This has some good refinements on the way to 1.0 final and a couple of nice, new features (the “auto” task and the “pushd” context manager):

• Added automatic running of “auto” task. If there’s a task with the name “auto”, it is run automatically. Using this mechanism, you can write code that sets up the options any way you wish, and without using globals at all (because the auto task can be given options as a parameter).

• When generating egg_info running “paver”, the full path to the Paver script was getting included in egg-info/SOURCES.txt. This causes installation problems on Windows, at the very least. Paver will now instead place the pavement that is being run in there. This likely has the beneficial side effect of not requiring a MANIFEST.in file just to include the pavement.

• the options help provided via the cmdopts decorator now appears

• pavements can now refer to __file__ to get their own filename. You can also just declare pavement_file as an argument to your task function, if you wish.

• call_pavement now switches directories to the location of the pavement and then switches back when returning

• if you try to run a function as a task, you’ll now get a more appropriate and descriptive BuildFailure, rather than an AttributeError

• paver can now again run tasks even when there is no pavement present. any task accessible via paver.easy (which now also includes misctasks) will work.

• added the pushd context manager (Python 2.5+). This will switch into another directory on the way in and then change back to the old directory on the way out. Suggested by Steve Howe, with the additional suggestion from Juergen Hermann to return the old directory:

  <span class="k">with</span> <span class="n">pushd</span><span class="p">(</span><span class="s">'newdirectory'</span><span class="p">)</span> <span class="k">as</span> <span class="n">olddirectory</span><span class="p">:</span><br />    <span class="o">...</span><span class="n">do</span> <span class="n">something</span><span class="o">...</span><br /><br />

The monthly Michigan Python Users Group (MichiPUG) meeting is coming up tomorrow! This month, we’re going to do something a little different: sprinting on a project to help out a new coworking space that is being set up in downtown Ann Arbor (see below for the full description from Bob Kuehne).

As usual, the meeting starts at 7PM at SRT Solutions.

proposal for 03/02/09 - Michigan Python Users Group | Google Groups

background:

* a guy named mike kessler is opening a co-working space on main
street, tween washington and huron, in the
old arcadian too space, right next door to (south) vinology. the
space is designed as a place where people
can pop in for a day, hang out, work with other people, get a
permanent desk, host trainings, etc. it’s
going to be pretty cool.

* as part of that, he needed some sort of door lock/authentication
system. i volunteered to put one
together, and it’s coming along fine, from the hardware and
software side. it’s basically a bag of
scripts (python, natch), and some csv files for manager users,
logging, whatever. oh, and a door latch
that gets controlled by the above.

* but from a ui perspective, i know better things could be done, and
this is where you come in.

pug topic/sprint:

* if people are up for it, i’d like to do three things:
* 30m : setup tasks, review / design schema
* 60m : split into groups and build an interactive site to do a few
tasks
* 30m : each group discuss results, demonstrate (10-15m@)

Googling “embarrassment driven development” (EDD) does not return as many hits as it should. I think it’s a very powerful development technique. I first heard the expression from the Plone guys at PyCon 2006, and Google did turn up this match:

[ArchipelagoSprint] Time to get cracking on Plone 3.0!

Wrt. timelines, I was hoping that we could try to have a “Tech preview” release before the Plone Conference 2006 in Seattle (October 25-27) - I’m going to be on-stage there talking about the exciting new features of Plone 3.0 - and I’d like to not be booed off stage. Yes, this is embarrassment-driven development - as usual.

That’s Alexander Limi illustrating the prime motivator for EDD.

The idea behind EDD is simple: if you have to demo something in front of an audience, and that something sucks, you will move hell or high water to make sure you don’t look like an idiot.

Every product has rough edges and warts, but no one wants a demo to be all warty and to have to say “yeah, I know you shouldn’t have to click to the left of the button, but we just haven’t gotten to that yet”. EDD ensures that, at least for the parts you have to get up and show, the rough edges will be smoothed in time for the show.

I’m going to be practicing EDD leading up to JSConf. I want to be able to show some useful, non-trivial bits of ServerJS work by then.

I’m happy to announce that Paver 1.0a2 has been released! And, unlike Paver 1.0a1, it installs (thanks to Greg Thornton for the patch for that!). I’ve been quite busy with other projects over the past month, so I appreciate the help of Marc Sibson and Greg Thornton in making 1.0a2 a nice improvement over 1.0a1.

Paver 1.0 is still for the slightly adventurous, because it has not yet seen testing by many people. Paver is not complex code (and Paver 1.0 is, I think, less complex than Paver 0.8 was), so it’s not hard to dig in if you have a problem.

Assuming nothing major comes up, I expect Paver 1.0 final to be out by PyCon.

Speaking of PyCon, I won’t be attending PyCon this year as I have a lot of other things on my plate at work this time around. Mark Ramm will be doing the Paver talk in my stead, just as he did at PyOhio last year.

The most basic case is very simple.

import time
import cherrypy
 
class WebSvc(object):
    @cherrypy.tools.caching(delay=300)
    @cherrypy.expose
    def quadruple(self, number):
        time.sleep(1) # make the real call somewhat costly
        return str(int(number) * 4)
 
cherrypy.quickstart(WebSvc())

That uses an in-memory cache and defaults to items expiring from the cache in 300 seconds (5 minutes). If you want to tweak that setting or others you can configure the caching tool to your liking.

cw

This is in response to a post that asks if setting up caching in other web frameworks is as easy as in Rails Ruby with Sinatra.

It’s been less than two weeks since Bespin was introduced, and there’s already been an impressive amount of activity around the open source project. There are at least 3 entirely new Bespin servers that I’m aware of.

The current Bespin server that we at Mozilla are maintaining is written in Python and appears in the backend/python directory in the Bespin source. To help people get up to speed with the code, I have created a screencast to help people get started with the Python backend and give them an idea of how the code is set up.

I look forward to hearing your feedback!

Bespin Python Backend Overview from Kevin Dangoor on Vimeo.

Creating a web framework with WSGI on Vimeo

The Michigan Python Users Group (MichiPUG) meeting topic from February 2009, presented by Kevin Dangoor. This screencast video shows us using WSGI components to build up a web framework piece-by-piece.

By the way, for people who are interested in working on the Bespin server, this is the kind of “web framework” that the server is built upon.

Despite working for an open source company, I have been pretty quiet here about what I’ve been doing in the Mozilla Labs web developer tools group. No more. We’ve gone public!

Mozilla Labs » Blog Archive » Introducing Bespin

Bespin proposes an open extensible web-based framework for code editing that aims to increase developer productivity, enable compelling user experiences, and promote the use of open standards.

I started working on Bespin as soon as I joined Mozilla, hitting the ground running with a new Python server. Ben and Dion had already done a lot of work and experimentation on Bespin prior to joining Mozilla in December, so I must confess that I am still fairly ignorant about the Canvas-based magic that they’re doing in the UI. But, Bespin has an architecture that lends itself well to selective ignorance: the server provides a RESTful API, and the client is responsible for all of the presentation. For their part, Ben and Dion have been able to be blissfully ignorant about the inner workings of the Python server.

Of course, I’m not a JavaScript noob and have done some work in the client, but my focus has been the server. Now that we’re out in the open, you can definitely expect that we’ll be talking more about how things work and how you can bend Bespin to your will. Bespin is honest to goodness open source (MPL-licensed), so it becomes an open and collaborative effort starting right away.

The initial reaction has been fantastic. There are tons of people hanging out in #bespin on irc.mozilla.org, and the mailing list has grown to a couple hundred members already. Thanks to everyone for jumping in with your thoughts and patches!

Here’s some of the coverage:

Dion’s post at Ajaxian:

From Dion’s personal blog:

Foolish chaps and companies have come to me in the past thinking that open source will be a silver bullet for “getting other people to do our work.” Those that have been involved in open source know that it isn’t the case. It is often more work. But, it is worth it.

From Ars Technica:

The project is still at an early stage of development and there is clearly a lot of work to be done before it will be able to deliver the same practical value as existing desktop editors. Despite the limitations, it shows an enormous amount of promise and has the potential to eventually deliver a user experience that rivals even the best text editors.

From Five Questions with Dion Almaer:

Now the browsers are moving fast again and building a first class platform for us to develop, the Open Web Platform. Instead of getting bogged down thinking about what IE 6 gives you, take some time to think about what you could build with the latest technology. I realise that you have to be pragmatic and get things working with your audience, but browsers are changing, and so are expectations.

From What Mozilla’s Bespin Bespeaks (ComputerworldUK):

You can see that Bespin is ticking all the Mozilla boxes, but what’s also striking is that this is a Web-based project: Mozilla is entering the cloud. It’s a further shift to viewing the Web as a platform for doing, well, just about anything. Clearly, against that background, open standards are even more important. And not only for the code: another issue that Mozilla will need to start addressing publicly is that of open data. As more stuff moves into the cloud, it become imperative to establish minimum standards for access to the data that is held there. I look forward to hearing Mozilla’s views on the subject.

While I certainly don’t speak for Mozilla, I would be extremely surprised if there’s anyone at Mozilla that believes that users should have anything less than full access and ability to take their data with them. There can be technical issues involved in providing the data, but the data should be available in some reasonable form. Bespin, for its part, makes it easy to export a project in a tarball or zipfile.

I was surprised to see Bespin covered even on Lifehacker:

Primarily, Bespin is a text editor—the kind you’d use for editing code or managing text-based todos. Using Bespin, developers could collaborate on projects through a unified interface (that still supports plugins!) no matter where they are—so long as they’ve got a browser.

cnet has the story, too:

For example, what about integration with open-source software repositories? If it’s flexible enough, Bespin could essentially act as a source code viewer that repositories such as SourceForge or Google Code could employ.

A nice writeup on the ReadWriteWeb as well:

It’s clear that a great deal of thought and attention went into this early version - and it’s a safe bet that it will only get more impressive as time goes on.

RWW last month surprised me with their coverage of me joining Mozilla.

I’m having a great time at Mozilla so far, and it’s great to be out in the open working with so many people now on Bespin and ServerJS.

Reminder of Thursday’s MichiPUG meeting:

The next MichiPUG meeting will be on Thursday, February 5th at 7PM. Kevin Dangoor will lead a discussion/demo on building your own custom web frameworks with WSGI. As usual, the meeting will be at the SRT Solutions office in Ann Arbor..

It turns out that both pip and easy_install have trouble with the Paver 1.0a1 package. I’m going to look into that, but in the meantime I have removed the Paver 1.0a1 tarball from the cheeseshop so that people don’t accidentally get it. Sorry about that!

As a side note, it is unfortunate that easy_install and pip both will pick up alpha and beta releases in preference to release versions. I would think that a nicer default would be to prefer the current release version unless there’s a flag saying “give me the test release”.

I have just pushed Paver 1.0a1 up to the Python Cheeseshop. This is very much an alpha release, so beware. That said, I’m successfully using it in my own projects and have been for some time.

I have significantly changed the way Paver works, and I think the new structure is simpler and more flexible because it is no longer tied to distutils. In fact, Paver’s task running capability is in paver/tasks.py and that module could conceivably be used standalone. And, the namespace of tasks is not muddied with a bunch of distutils tasks if you’re making a pavement for some other purpose.

But, the distutils/setuptools integration still exists and is easy to turn on.

Paver 1.0a1 has a handful of useful new features, including the ability to run separate sub-pavements (in the same process, no less).

The docs on the Paver site are still for 0.8, and I will leave it that way for now. When you install Paver, you can run “paver paverdocs” to see the docs for the version of Paver you are running.

Please report any bugs you find either to the googlegroup or directly to the tracker at googlecode.

Thanks to Marc Sibson, Bryan Forbes, Juergen Hermann and others who have contributed to this release.

In Python, we’ve been enjoying the convenience of mixing and matching various things that follow the WSGI convention for a while. More recently, Rack brought those good features to a very similar interface for Ruby. And now I see:

Jack: Rack for JavaScript

echo “Rack provides an minimal interface between webservers supporting Ruby and Ruby frameworks” | sed -e s/Rack/Jack/g -e s/Ruby/JavaScript/g

Jack comes from Tom Robinson, one of the developers of Cappuccino/Objective-J.

"Although many Americans heard Barack Obama's inauguration speech, they probably weren't listening for plyptotons and catachresis — but Geoff Nunberg was." This is the best reflection I've seen on Obama's speech. It's not that I agree with everything Nunberg says (for example, I think antimetabole can still be "vessel of deep ideas" to the modern ear, and even the cited examples from McCain and H.R. Clinton worked), but it's that Nunberg bases his points on basic, good sense, that's unfortunately rare these days in discussions of language.

Saved By: Uche Ogbuji | View Details | Give Thanks

Tags: politics, language, news, culture, audio

"Grid computing introduces a new concept to IT infrastructures because it supports distributed computing over a network of heterogeneous resources and is enabled by open standards. Grid computing—which helps optimize underutilized resources, decrease expenses and reduce costs—has helped organizations accelerate business processes, enable more innovative applications, enhance productivity, and improve resiliency of IT infrastructure."

 Bookmark this on Delicious - Saved by chimezie to grid parallel paper infrastructure - More about this bookmark

"The purpose of this IBM Redpaper is to provide discussion material about grid computing, concepts, use, and architecture. Grid computing represents unlimited opportunities in terms of business and technical aspects."

 Bookmark this on Delicious - Saved by chimezie to parallel grid paper infrastructure - More about this bookmark

With my new job comes a new MacBook Pro (the latest generation, very slick!). The Setup Assistant did a great job of making my new MBP just like my old one, carrying across all of my applications and settings from my latest Time Machine backup. Everything in /usr/local/bin made it. Ditto for my MacPorts stuff in /opt.

One thing that didn’t make it was the Python libraries that I had installed in the system-wide site-packages directory (/Library/Python/Version/2.5/…). The Setup Assistant is designed to not pull OS parts across when you are loading up a new machine. That makes sense, but unfortunately the Python stuff seems to fall into that category.

Perhaps the site-packages directory should be moved to /usr/local or something so that it gets copied over? A change like that would be useful for Apple to make.

Thankfully, the impact on me is pretty minimal, because I tend to do my work in virtualenvs and don’t have very many packages installed system-wide.

At the next Michigan Python Users Group (MichiPUG) meeting, we’ll be talking about building custom web frameworks using WSGI bits and pieces. I’ve done this a couple of times now and think it’s a powerful and straightforward approach that works well for certain kinds of projects.

I’m figuring we’ll take a look at some available components and glue stuff together to build a framework during the session.

As usual, the meeting is the first Thursday of the month (February 5th, in this case) at 7pm at the SRT Solutions office in downtown Ann Arbor.

Recipe 576613: Easy State Pattern - support for implementing state machines "Provides is a module that gives support for implementing state machines. States are implemented as subclasses, derived from the state machine class. Methods that are state dependant or which cause transitions are declared using decorators. "

 Bookmark this on Delicious - Saved by chimezie to recipe code python state-machine - More about this bookmark

Recipe 576611: Counter class: Bag/multiset class for convenient tallying of hashable elements.

 Bookmark this on Delicious - Saved by chimezie to recipe python code reference datastructures - More about this bookmark

"Last year, when lenders started dropping like flies as foreclosures rose and margins were called, the problems of Wall Street became more and more apparent, and lending guidelines were tightened to the point that many individuals were stuck in their time-bomb loans, and thus began a vicious cycle. But what led to this? Here is a visual guide to help you understand the events leading up to the bailout."

 Bookmark this on Delicious - Saved by chimezie to visualization finance diagram history - More about this bookmark

"ExpanDrive gives you perfectly transparent access to open, edit, and save files with your favorite programs, even when those files are half a world away."

 Bookmark this on Delicious - Saved by chimezie to ssh network tool mac - More about this bookmark

"Capture HTTP conversations regardless of the browser or server technology you use."

 Bookmark this on Delicious - Saved by chimezie to http tool mac software network web - More about this bookmark

'UML is applying an abstraction at the wrong end of the problem. It is primarily used to sketch object models for inferior languages. As such, it tends to explode into incomprehensible patterns of accidental complexity in order to accommodate the various “design patterns” that are used work around the lack of essential language features.' Wow. So harsh (written by a "lispy", mind you), yet so, so true. I's a much sharper version of something I've been saying for years. Too many architects focus on implementation artefacts rather than the core domain model, and this is one of the most expensive mistakes in software engineering.

Saved By: Uche Ogbuji | View Details | Give Thanks

Tags: modeling, programming

I just finished some changes to python-dlp including a modification to
FuXi that includes an implementation of the Magic Set Transformation
(MST) method for RIF-like horn clauses. The most useful, immediate
value this has for me is to be able to (essentially) implement a DLP
(description logic programming) entailment regime for a SPARQL query
service.
> 


 
Consider the test case for the SymmetricProperty OWL test.
> 


 
The base facts are:
> 


 
  first:Ghent first:path first:Antwerp .
  first:path a owl:SymmetricProperty
> 


 
The goal we are trying to prove is:
> 


 
  first:Antwerp first:path first:Ghent
> 


 
I.e., a user wants to query an RDF dataset that includes an RDF graph
with the above statements and is expected to implement an entailment
regime for OWL-DL RDF such that the following query gives a positive
answer:
> 


 
  ASK { first:Antwerp first:path first:Ghent }
> 


 
The general pD* rule that would normally apply in helping answer this query is:
> 


 
  {?P a owl:SymmetricProperty. ?S ?P ?O} => {?O ?P ?S}.
> 


 
Re-written in a familiar (prolog-like) RIF-BLD syntax:
> 


 
  Forall ?P ?O ?S ( ?P(?O ?S) :- And( owl:SymmetricProperty(?P) ?P(?S ?O) ) )
> 


 
In order to maintain consistency, a rule-based engine that used this
clause to implement the definition of a symmetric property would need
to fire it for *every* triple in the fact base (in order to properly
calculate the herbrand base) because of the 2nd triple pattern in the
body / antecedent / left-hand-side of the rule: ?S ?P ?O
> 


 
However, the DLP approach that converted tree-based OWL axioms into
colloquial horn clauses would allow us to use (instead) a
domain-specific rule:
> 


 
  Forall ?Y ?X ( first:path(?Y ?X) :- first:path(?X ?Y) )]
> 


 
This rule is domain-specific in the sense that it only applies to
instances of the first:path predicate rather than for every predicate.
 As a result of this transformation, the procedural evaluation of the
rule for symmetry has been reduced from the worst case to only the
fraction of the RDF dataset concerning first:path statements.
> 


 
So, a knowledge base that could exhaustively evaluate rules in a
top-down fashion (via 'forward chaining') prior to bringing up the
SPARQL service could answer that question against the (smaller)
entailed RDF graph.
> 


 
However, with the MST implementation, if the query was known a priori
the ruleset can be modified into a version that further restricts the
amount of redundant work done during the inference process. For
example, even if the SPARQL service is known to never have to answer
that query, the colloquial rule above would still be needed by a niave
implemenation and would apply to every statement that used the
first:path predicate.
> 


 

Essentially, performing a top-down (or
forward chaining) evaluation of the rules and the facts simulates a
backward-chained proof.
> 


 
Below are the 3 rules that replace the original domain-specific rule:
> 


 
  :path_magic(?LOC1 ? LOC2) :- And( :path_magic(?X ? LOC2)
:path_bf(?X ? LOC1) :path_magic(?X) )
  :path_magic(?X) :- :path_magic(?X ?LOC)
  :path(?X ?LOC1) :- And( :path_magic(?X ? LOC1) :path_magic(?X)
:path(?X ?LOC2) :path_magic(? LOC2 ? LOC1) :path(?LOC2 ?LOC1) )
> 


 
And finally, the trigger for the proof is the following RDF statement:
> 


 
  first:Antwerp first:path_magic first:Ghent
> 


 
The first two rules, pass through information about the sub-goals of
the query and essentially block the final rule from taking effect
until the trigger is added to the fact graph. It is clear to see that
the 3rd rule, will no longer apply to every RDF statement with a
first:path predicate, but rather only statements of that kind where
the subject and / or object terms are part of a query. So, for a
SPARQL service where we do not expect to answer queries that rely on
supporting symmetric properties in the first:path predicate, no
calculations will be performed and no unnecessary RDF statements will
be entailed.
> 


 
I hope to write a bit more about some of the benefits of a Python
toolkit for building Semantic Web expert systems. I touched a bit on
these in my InfixOWL write-up and presentation, but haven't really put
the whole picture together.
> 


 
-- Chimezie

Permalink | Leave a comment  »

Need to setup a harness to run these against FuXi

 Bookmark this on Delicious - Saved by chimezie to rules w3c testing - More about this bookmark

"In this note I present a concise proof of the existence and uniqueness of the limit distribution of an ergodic markov chain. This nice proof was described to me by David Gilat of Hebrew University during a hot summer afternoon in Perugia"

 Bookmark this on Delicious - Saved by chimezie to academic machine-learning, logic, state, mathematics - More about this bookmark