Carlos de la Guardia looks at python web projects: do they eat their own dogfood by running their main website on their own software? Like plone.org runs on plone. Or did they pick something else when that was better suited to the specific task? Like the great new grok website that runs on plone.

Another way of looking at it: do individual developers (instead of projects) eat "their" dogfood? I'm definitively eating my own dogfood: I program with plone and my website runs plone. The weblog is Quills, which I also contributed to.

Especially when you only run a weblog, a ready-made blogging tool (wordpress for instance) seems to be the "the right tool for the right job" choice. For many plone developers. MrTopf also recommends it in his marketing plone the web 2.0 way presentation.

So: different perspectives. Eating my own dogfood suits me fine. For customer projects, we switched from instancemanager to buildout. So I experimented with it first when I switched my site to plone 3.0. Synergy between work and private :-)

I want to close with a great quote from Carlos' article: a Frankenstein old Plone setup.

Tags: plone, python

When updating vanrees.org to plone 3.0, I did not yet include an updated layout product. I wanted to do that one from scratch. Last weekend I found the time to have a first go at the layout; today I updated my server. Rough edges here and there, but at least it doesn't look like standard plone. Some random comments:

Layout product

For the layout I used the latest ZopeSkel one for paster: the plone3_theme. I did it as a lib/python package instead of a Products/... product. No problems there as I use buildout.

Dtml

Following a tip by spliter, I didn't use the stylesheet directory inside browser/, but put it in skins/, just like in the old days. That way I can use dtml, which I like to keep a few variables in check. I dislike having to put #34a7c2 in my css files. I just can't see what color that is. Putting & dtml-secondaryColor; is much more clear in my book. Never mind that there's been a "dtml will be deprecated for css" message for over 4 years or so :-)

Viewlets

The main_template uses viewlet managers and viewlets now instead of a mountain-like pile of interlocking templates and macros. Much cleaner for the integrator and developer. With the @@manage-viewlets template, the TTW part also functions as a charm. Hiding a viewlet (the personal bar in my case) is just a click away. That's much better than delving deep in those templates and customizing them. A quick click in portal_setup gave me a viewlets.xml to put into my layout package's genericsetup directory, so my TTW and my filesystem bits rejoiced in unison.

Next steps

I'm sure I've messed up the lists somehow. Gotta fix that. And I want something extra in the header: that'll be a custom viewlet. And I want to add some viewlet love to quills' (the weblog) templates. Last night I fixed up the regular template so that in-place editing works on a single weblog entry. More to come!

Tags: plone quills

Sunday's sprint wrapup! A nice day of sprinting. Everyone was busy, including all the new first-time sprinters. We had a great round of pastis before the wrapup thanks to AtReal :-) I really enjoyed the Good Stuff :-)

zodb benchmarking

Made some observations. Tried adding lots of simple objects in various batch sizes to get good metrics. Increasing the batch size made it detoriate exponentially. If you stay with a memory-only zodb you can get literally millions of insertions per minute. They suspected a big influence of the mac disk that they did the test on. They're going to continue tomorrow with plone content types, concurrent commits, etc. It also turned out that it is important to pack your Data.fs., as after adding 200k of objects some 30 percent of the database size was taken up with transaction info.

plone survey

Smallest team, 2.5 person. They can have both a 2.5 and 3.0 release out tomorrow. They're still working on KSS stuff to make it more usable, though. Fun to work on. Did a lot of internationalization work.

PrimaGIS

New buildout. Refactoring. Tilecache enable now, really fast now compared to what it is now. New demo creation on the way.

KSS

jquery problems: bugs in the jquery demo code. jQuery events started (success). "Back" button works better now. They tried working on "push" and have a working demo: it really worked, though I don't know how they managed it. Great great stuff.

Translation group

Lots of stuff working. Most of the fields. Hope to have most of it working tomorrow.

sqlalchemy

Bad day. Fighting the stack, including zopetestcase. Ported code to collective.lead (great as it is in Martin's book).

AT rest

Didn't get finished. Saw how to do it. Great help from Ramon: checked in plone.rest tonight. Also fighting against the stack.

getpaid

Completely converted to genericsetup. Got two bugs fixed. Got some 40 tests added, so ready for tomorrow's refactoring. Added some screencasts: they're going to market the hell out of this puppy.

KSS part 2

They've added Django support to kss! Within one day. (Big applause)

workflow tests

They've found out a lot of bugs in the basic product. Worked a lot on tests. Got them running. Now testing workflows is way easier. (Note to self

workflow

State workflow and activity workflow. Trying to switch between the two. Lots of work, but good to have the current results.

Functional tests

They have two new scenarios for generic reusable testing. And they now have a tutorial. Improved comments. Code cleanup. Buildout for selenium.

Vice

Less work for customizers needed, just override the adapter. Should make it easier. Functional tests have been greatly improved. Fixed some basic datetime tests. Polishing of the user interface.

New themes

ZopeSkel was fixed for windows. They worked on concrete designs and Spliter demoed them. They had a great working day. They thought the new guys to create good non-messy themes. And they created documentation for this. They even have disability support guys. The examples are at openplans , for example capri theme , napoli mock . Spliter seems to be doing a hell of a job of shepherding everyone towards good designs. One comment: document your CSS files lavishly with comments, they are filtered out by portal_css.

Commenting

They were stripped down to 2 people today and went over the generic concepts, which turned out to be a good idea. They coded some and can add comments programmatically. They also use parts of easycommenting for KSS and templates. Nothing is really finished, but they have a good foundation. Tomster is looking forward to continuing with this.

Archgenxml

It works. Plone 3.0 compatible, trunk is now safe again. Improvements to the generated tests.

Nuplone

Navigation tree improvements, for instance the whole active section now has a background. The livesearch has great css now. There's also good multi-resolution support, too, so it degrades quite gracefully. When changing position due to resizing of the window, the colors of for instance the calender portlet change. Wild applause because of that feature. Looked like the darkest magic that you can imagine. :-)

plone4artists

Continued with the tests and got most of the audio tests fixed. Audio was released today, Rocky released the video later this day. More translations added, the framework is ready for more translations. The p4a openplans page details the translation mechanism. Calendering added a nice day view, tomorrow they hope to improve the week view. Tomorrow they hope to get it over to plone4artists for a release. Something usable is on the horizon!

Extreme management

We worked on moving portlets to plone 3 portlets. Worked on the workflow (generic setup). Some KSS added for body texts and descriptions. Tried to get macros to work with KSS, but started to work on viewlets instead as that works better. Worked on splitting out bookings into a separate product, that'll be soon into the collective. Reusable for others!

blobfile support

They got conversion of regular files to blobs working. Demo failed as safari crashed :-) He assured it worked :-)

Schema extender

Lots of doc improvements in the doctest. Looks like excellent documentation. There's an additional adapter for modifying existing schemas. I questioned the difference with generic ISchema overriding: they answer was that this product allows you to plug those schema modifications in. No conflicts. Multiple products can do it at the same time.

Technorati tag: ploneconf2007

A real quick writeup of a real quick "what did we do today" session where everyone presented what they did. (Hard to write down so quickly, aargh! Probably made a few mistakes.)

getpaid

Cleanup of tests, new translations, more genericsetup. Excellent way to get everyone

kss

New tutorial which reflects plone 3.0-style development. People worked on resurrecting the event-push system. jQuery work. Django integration for kss: a new framework that they support.

sqlalchemy

Fixed copy/paste bug. New buildout recipe that makes it easier to set up. Lots of discussion about catalog integration. Better ideas about metadata fetching.

extreme management

Exchanged information about how we all handle projects in different organizations. Figuring out how to handle both. Slowly getting there code-wise. Backporting a KSS experiment to trunk. Looked at

ploneGIS

User documentation. A new buildout. Work on a new demo product. Work on layers.

syndication

Some success, some snags. A big timezone bug bit them again, but they made good progress. They can now tag an alpha, mostly due to their remote sprinter.

archgenxml

Polishing, bugfixing, finding a bug in archetypes itself (and fixing it). Documentation is now much better. Probably archgenxml works nice for 3.0 tomorrow. Ideas about breaking the dependency on archetypes.

Functional testing framework

Documentation update. Debugging code improved. Additional component for search testing. Started a buildout for selenium integration.

New themes for plone

Mostly setup today, making sure everyone the whole big team got set up. Worked out what should be handled in a theme. Their new theme has newtheme.pizza as a code name.

Commenting

Team of four, two architects, two coders. Figuring out how to handle comment spam filtering. New project page on openplans. They made a buildout for plone.commenting. They have their first tests running.

plone4artists

Tagging products for a 1.0 release. Fixing the last bugs and getting everything ready. Started on the translations. Working on plone 3.0 support. Work on blobfile support with especially nice handling for plone. The calender team has 10 bugs that need ironing out, 4 of them are done now. And they added a 1-minute screencast. They hope to get more of them done.

plone.relations

Work was done especially on plone.app.relations. Trying to tie into user objects. Adapter experiments and teaching people stuff.

Project name that I missed

Just getting started. Had lunch.

Blobfile support

Figuring how to best integrate it without breaking things. Did groundwork like adding a contenttype and a buildout. They hope to write migration scripts tomorrow morning for ATFile and ATImage. They idea is to add an add-on package for plone 3.0 which will hopefully end up in the next major release. When the 3.0 product is done, they plan a 2.5 product.

Internationalization

Trying to allow customization of translations through the web so that you can make small customizations.

Workflow

Unittesting for workflows basically sucks. They tried to work with Matt Hamilton's (?) idea for using a .csv file for testing.

archetypes schema extender

It is now out of proof-of-concept. Seems to be really working. Improving the interface and expanding the doctest. Will be the way for schema subtyping.

plonegov

They were too hungry to wait for this wrapup :-) Plonegov will merge the three projects that deal with openoffice integration.

I'm off to the plone conference tomorrow morning. A wishlist item I've had for a while was to migrate my website before the start of the conference.

I finally managed: not without some difficulty. It is an old site and it went through quite a number of migrations. Some of them being non-released svn versions. But I was still amazed at the errors I got when attempting migration. I mean, if you can't even get into the ZMI...

Eventual solution: move the parts I wanted to salvage to a "to_copy" folder and make a .zexp of it and import it in a fresh plone 3.0 site. With a bit of care, as you normally need the same products at both ends of the .zexp. An intermediary 2.5 plone site solved that.

I took the opportunity to remove most of the add-on products (except linguaplone and quills), so the site is fresh! It is also missing some bits and pieces here and there (like images and some external html content), which I'll sort out at leisure after the conference :-)

Technorati tag: ploneconf2007

Test for quills: does the atom template show the correct mimetype (application/atom+xml) now? I need to add an entry to make sure the feed refreshes ;-)

I've got quite a number of svn checkins on quills in the last month. I wanted to move my website to plone 2.5 (done) and the latest version of quills at the same time.

Quills was in pretty good shape and I only had to fix a very small number of bugs to get it running reliably here. The two biggest additions:

• Improved migration from 0.9 (latest stable release) to svn trunk (1.5 alpha), including unit tests.
• Large-scale template refactoring to use just two macros in the majority of cases. One to display an individual entry and one to display a list of entries. Both macros can be influenced by two or three config options, for instance to show only a description or a full entry. This got rid of about 200 lines of template code.

A very good move that Quills made was to remove the old quills-internal "topics" and to replace them with the standard plone keyword system. Now you can easily tag your entries with multiple tags, if you want, and you get the luxury of having additional metadata. And I sure like the "tag cloud" portlet :-)

So: thanks all those that made Quills pretty good. Things that'll probably get improved before 1.5 comes out:

• Posting from external applications (using the Blogger API etc.).
• Improved feed management, apparently the upcoming big apple sprint will focus on this as one of their jobs.

I made my website with Plone. Mostly plain plone (2.1 at the moment) with some layout modifications. I do use some extra products, though, and not all of them are readily visible. I was asked to provide the list, so I'll put it up here for all to see.

• LinguaPlone for providing translations of documents. I only use it for a few pages at the moment, but I'm slowly extending it. "Languages" in my case means Dutch and English. For example on the homepage .

  Linguaplone finds the best language based on your browser settings. So if you're Dutch you'll get the Dutch page if available, otherwise the English one. If you click one of the language flags, linguaplone sees it as a direct command and will refuse to display pages not in that language. All in all I'm pretty happy with it.

• Local folder, basically only for including my old graduation thesis that's a set of docbook-exported plain html files somewhere on my harddisk.
• Plone comments for warning me of new comments. You can use it as a moderation tool, but I only use it to get an email for every new comment. Oh, and it adds a "name" field for anonymous comments.
• Plone google sitemaps for notifying the almighty google of all my URLs. Look at google sitemaps for an explanation. It's just a list of URLs, basically. Works out of the box.
• Redirection tool helps me to keep the number of 404s under control. You can add aliases for pages ("this page used to have this URL") and alternatively it will try to guess where the unknown page went. If it finds a page elsewhere with the same ID it will redirect right away.

  I do my best to make a good website and one of the most important things is to prevent dead links. I do move stuff around, for instance when switching weblog software. But I flatly refuse to throw away all that free advertising (read: incoming links) to my old weblog posts, so I've got a pretty impressive set of redirects in my apache config! And for a lot of the other things, the redirection tool is my friend.

• Quills, my weblog tool of choice. I'm using the 0.9 version at the moment with some small local modifications (especially to the atom template). I'll have to look at the current 1.5 development, whether my changes are still needed (and if "yes", feed them back into current development).

Other changes mostly include the portlets. Some adwords portlets, random photo, clustrmap. etc.

Atom is the nice new shiny cleaned-up standard for weblog feeds, so I decided to use that one as my main feed. I had to spend a few hours to get my atom feed displaying OK in bloglines, safari's rss view and on Planet Plone .

Apparently Quills' atom template wasn't too good. Or the standard isn't clear enough. Well, I at least have the entries pointing towards my own weblog instead of into nothingness. But: the summary isn't getting displayed by bloglines (and they're also removing spaces from around links, strangely). I'll have to do some more testing and tweaking and some quills bugreporting/patching, if needed.

So for safety's sake:

• Atom feed
• Atom feed with just plone/zope/python stuff
• RSS feed that should keep everyone happy.

The earliest recorded post here is from April 2003. I know I did some limited blogging before that on a temporary website for about a year or so: mostly research notes.

If you look at the archive list of this shiny new Quills software, you'll see that the number of posts varies greatly. 2006 (1), 2005 (94), 2004 (184), 2003 (67). Somewhere between 5 and 15 posts per month. The frequency is about right for me, but I dislike the variation a bit. Sometimes almost two months go by without a single post. I've decided to take a more active dislike to that :-)

In 11 Ideas to maintain your blog Rajesh Setty has some good tips if you're serious about your weblog. And I'm serious about my weblog, for some of the following reasons:

• Writing well is a very valuable skill. Writing my own weblog hones that skill.
• I get to give away knowledge and information. Giving that away does not make me poorer; it does mean value for other people.
• Part of above value flows back in numerous ways (being contacted by people, getting opportunities, being visible, etc.). My weblog isn't read by a lot of people, but getting at least 7 pats on the back for a well-written 2005 europython summary can't hurt.
• Man, is it ever good for your google ranking! :-)

Some of the highlights of Rajesh's article for me:

Make a long-term commitment

I've been at it since 2003 and I intend to keep it up for a few decades or so.

Respect your audience

Rajesh has a great term: ROII . Return on interaction investment. If you choose to read my weblog, you should get a good return value on that investment. Therefore I try my best to make it worthwile. Checking links afterwards, writing complete English sentences. I think, for instance, that my conference notes are normal readable prose instead of just a collection of short semi-sentences that look like they were typed over directly from the powerpoint sheets. Oh, and adding links like that one just above to ROII :-)

Observe and Listen

Read lots of other weblogs.

So: on to the next few years of blogging!

I copy/pasted an existing piece of migration code out of Quills and modified it for my needs - and stuck it into the the installer for the product I'm using to store my templates and so in:

    def _createWeblogEntry(quills, id, oldid, title, desc, publishDate,
                           urldate, text, topics):
        quills.invokeFactory(id=id,
                             type_name='WeblogEntry',
                             title=title,
                             description=desc,
                             text=text)
        new_obj = getattr(quills, id)
        new_obj.setEffectiveDate(publishDate)
        new_obj.setEntryCategories(topics)
        new_obj.setContentType('text/structured', key='text')
        #wft = getToolByName(self, 'portal_workflow')
        #wft.doActionFor(new_obj, 'publish')
        out.write("Added blog entry %s.\n" % id)
        out.write("apache: %s => archive/%s/%s.\n" % (oldid, urldate, id))

    def migrateSquishdot(portal):
        # These are some values that are used to create
        # the Quills weblog
        quillsBlog = 'weblog'
        quillsTitle = 'Weblog'
        path = quillsBlog + '/drafts/'

        # Create the blog based on the values set above
        if hasattr(portal, quillsBlog):
            out.write("Quills log already exists, deleting it again.\n")
            portal.manage_delObjects(ids=[quillsBlog])
        portal.invokeFactory('Weblog', id=quillsBlog, title=quillsTitle)
        newblog = getattr(portal, quillsBlog)

        target = portal.restrictedTraverse(path)
        try:
            oldblog = portal.oldweblog
        except:
            out.write("Can't find old weblog, quitting.\n")
            return
        qtool = portal.quills_tool

        for entry in oldblog.data.values():
            title = entry.title
            desc = ''
            body1 = '\n'.join(entry.summary)
            body2 = '\n'.join(entry.body)
            body = '\n'.join([body1, body2])
            oldid = entry.id
            id = qtool.idFromTitle(title, 'WeblogEntry', newblog)

            # Using the '%c' stftime was the only way I could
            # successfully migrate the effective time
            ltime = localtime(entry.created)
            date= strftime('%c', ltime)
            urldate = strftime('%Y/%m/%d', ltime)
            # Make sure we don't have the topic already, then add it
            oldBlogCat = entry.subject
            quillsTopic = qtool.idFromTitle(oldBlogCat, 'WeblogTopic', newblog)
            if quillsTopic not in [blogTopic.getId() for blogTopic in
                                   newblog.getTopics() ]:
                newblog.invokeFactory('WeblogTopic',
                                      id=quillsTopic,
                                      title=oldBlogCat)
                out.write("Added topic '%s'.\n" % quillsTopic)
            _createWeblogEntry(target, id, oldid, title, desc, date,
                               urldate, body, [quillsTopic])

Note that I moved the old weblog over to /weblog_old/ for the time being so that I could create the quills weblog directly into a fresh new /weblog.