The first session I attended was FluidDB by Terry Jones. Terry described it as "database with heart of wiki" and its a description I can support from what I have learned. It is a database where objects don't have owners so anyone can add and change metadata(attributes/tags) on them. But the tags have owners and therefore permissions. The big benefit from this approach is that you can add tags or metadata about the object to the object it self instead of creating your own database. Some good examples are all those Twitter apps out there which add or calculate information about the twitter user or the tweets and then store them in their own database. If Twitter used FluidDB everyone could store it's own extra metadata where the original objects where. Now let see if all the promises will be fulfilled when FluidDB is released in a month or so. I really hope they will because I heard Terry sold his apartment to invest in this, that is what I call an entrepreneur! There were two more sessions related to database, one about MongoDB and one about CouchDB from Mozilla. The later uses the language Erlang to get the concurrency speed and it is a replicable and scalable database.

Javascript is THE language for web 2.0 applications and it's much simpler to develop today with all the AJAX libraries out there but still hard to keep up with all the different browsers and it's bugs. That is specially true if you don't use it daily, like a python programmer like me. I was pleasantly suprised to see Pyjamas come this long way. Today it's possible to develop your user interface in Python and have it compiled to javascript that runs in the browser. This is definitely something we will try out because it also enables you to create desktop applications. Luke Leighton who presented "The Zen of Pyjamas" even thought the HTML UI version was much nicer and had better quality then any widget library like PyQ4, PyGTK, wxPython. The main reason for this was that there had gone far more development hours into the web components.

Fulltext search is one powerfull feature in Plone that we use in our daily job. But on large websites it's good to separate the indexing and searching to separate server or service. The "Search with Python and Xapian" was a nice session by Richard Boulton where I saw how to use Python to index and search in Xapian as an external web service.  I also saw Open Object in action from the Open ERP project and I was impressed how simple it is to create enterprise applications with it. Walking back to hotel I thought "What is the difference between Open ERP and SAP?"  SAP has limitations on what it can do and no limitations on the cost, Open ERP has no limitations on what it can do because it's open source and the price well you decide. If you google for Open ERP and SAP this was top hit for me :)  http://fptiny.blogspot.com/2008/10/when-sap-meet-open-erp.html

Valentine Web Systems will be represented at EuroPython 2009 by company founder Sasha Vinčić who will attend the conference this year. Sasha Vinčić is looking very much forward to the EuroPython 2009 conference to get to learn more about what is going on in the wonderful world of Python. Cloud computing and web are interesting subjects.

Meeting old friends who also attend EuroPython 2009 is another highlight. Like Matt Hamilton from Netsight Internet Solutions who is giving the ”Lipstick on a Pig” talk about using Deliverance to skin third party system or Bernd Dorn from Lovely Systems with the talk ”Real World App Engine Projects” about how to get the best out of App Engine, and related technologies such as Bigtable, Amazon S3, EC2 and Cloudfront. There will of course be many more familiar faces to meet in the crowd, be sure about that.

There are a big number of talks that look really interesting to a happy Python fanatic from Sweden, such as Reliable Python Web Sites, Semantic Apps With CubicWeb, Tapping Into The Web Of Data, The Zen Of Pyjamas and Castles In The Cloud.

The week in Birmingham will be a joyride beyond all borders. See you there!

Want to know what’s going on inside the head of a happy Plone user? Now, this is your chance of a lifetime to do that. Netsight is twittering directly from the European Plone Symposium 2009 which is held in Sorrento, Italy this week. Imagine seven days of nothing but ”Ploning” in three different levels: Training sessions, symposiums and sprints. It has to be next door to heaven for a Plone fanatic. 

You’re most welcome to follow Netsights everyday instant reports on Twitter at http://twitter.com/netsight. How about these excited Twitter comments from the European Symposium 2009: ”Just seen 30 second demo of Plone tiles and editing UI for new plone. Wow! Blown away at just how cool it is, really looking forward to it… Missed Underscore meetup :( In Sorrento for Plone Symposium :D Life is good… Plone 4 to be released in 2009, smaller subset of features than originally planned, but bringing cool stuff sooner.”

Netsight is continuously giving you more virtual comments and reports from beutiful Sorrento as the Plone Symposium proceeds throughout this week. Keep close to it at Twitter.

Even if we enviously follow the Big Event in Sorrento, sadly knowing that we couldn't be there this time, we make sure that our dear company Valentine Web Systems always catch up with the latest news about Plone and also secure that all our costumers are well equipped and prepared for the future to come.

Valentine Web Systems, a web development company based in Malmö Sweden, is always on the lookout for new crew members. If you're a skilled programmer looking for new challenges and horizons, this could be your chance.

Our job is to build web portals and web based solutions. We use Python as our main programming language and adopt the power of Cloud Computing. The open source system Plone is therefore our prime working tool.

The team members of Valentine Web Systems all has the privilege of enjoying a big deal of freedom in the job as the customers and the working project always are in our focus. This is how our planning is ruled. To become as efficient as possible in our work, keywords such as SCRUM, agile and pair programming are guiding stars on our heaven. Education is essential to us, which is why we attend international conferences and sprints in order to enrich ourselves with new knowledge about Plone and other interesting web technologies.

Are you a skilled Python programmer and also familiar with Plone and Web 2.0? Then that makes you the most wanted person for us.

Call or email us asap. You may send us your CV right ahead. Our office is located in Malmö, Sweden. You are more than welcome to work together with us there, but you may also work from your home or anywhere else if you like.

We´ve spent the last nine years working a lot with web sites like EEA – European Environment Agency and Jeeves and we´ve got to learn so many good things when it comes to multilingual web sites. So far we´ve just written in technical English on our web site but from now on we would like to proceed and explain to you all why Plone is the most secure CMS and one of the best multilingual working tools.

First of all: Let´s all realize the fact that thousands of happy and satisfied users all over the world all states the same: Plone rules the world! Plone is the most powerful and easy-to-use CMS there is. It´s easy to install, use and extend. Plone lets non-technical people create and maintain information using only a web browser. It´s as easy as 1, 2, 3...

Ordinary people and big companies from all over the world has discovered all the great possibilities about Plone. Many of them are more than just rookies. We´re talking about really big guys like Amnesty International, the Government and the Parliament in Brazil, NASA, Nokia and a big number of universities and museums around the world. As we are long time Plone users we do feel honored to be a part of such a noble society.

Handling many different languages on a web site or a blog demands capability by the system. There are no written rules about what´s right or wrong. Please take a look at these two web sites and you´ll see what we mean. http://www.eionet.europa.eu/software/design/multilinguality/missingpagedilemma
http://www.eionet.europa.eu/software/design/multilinguality/navigation

The multilinguality of Plone is very flexible and creates new possibilities for you as a user to handle the cases when it comes to simple interpretations of the whole web site or complex lingual combinations where only some specific parts are interpreted. This means for example navigation that works in every level without bothering the user and interpreted tags that still means the same.

In handling big web sites like EEA automatic transportation is used for all material that needs to be interpreted or updated. The stuff is sent for interpretation and after that imported with a simple drag-n-drop funktion. According to our knowledge there are no other system that manage to do that!

Are you curious about Plone you might surf in to plone.org. All the needed info about Plone and all the latest updating info are to be found there. Are you looking for blogs and comments by Plone users you might as well visit planet.plone.org. Twitter is another source for info about Plone. twitter.com/valentineweb is where we share our experiences about Plone. If you´re in to statistics please don´t forget to visit the statistic blog about Plone.

When it comes to security there are no other system that can beat Plone. Plone had 10 security holes in a statistic comparison where, for example, Wordpress had 162 security holes, Joomla 324 and the language PHP as many as 9869 documented security holes compared to 65 in the language Python that Plone is written in.

There is no straightforward way to run plone, repoze and buildout yet, atleast not if you want to use a plone/zope version other than the one repoze.plone ships with. Chris McDonough has this ticket assigned to him, so there will hopefully be an official and easy way soon. Until then this is how I got it to work.

Buildout.cfg:

[buildout]
parts = plone repoze zope2 instance addpath
eggs =
find-links = http://dist.repoze.org

[plone]
recipe = plone.recipe.plone

[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}

[repoze]
recipe = zc.recipe.egg
eggs =
     PasteScript
     WSGIUtils
     repoze.zope2
     deliverance
     ${plone:eggs}

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
http-port = 8080
debug-mode = on
verbose-security = on
eggs =
    ${buildout:eggs}
    ${plone:eggs}
products =
    ${plone:products}

[addpath]
recipe = z3c.recipe.runscript
update-script = addpath.py:main
install-script = addpath.py:main

This is not that different from a normal plone buildout. What's new is [repoze] which installs paste, deliverance and repoze eggs and scripts to enable wsgi.

When you want to start the instance you can either start it the old way without enabling repoze or wsgi::

  $ bin/instance fg

  $ bin/paster serve paste.ini

You need do add the paste.ini file to your buildout. The file says where it can find zope.conf, what server should run and what middleware should be in the wsgi pipeline.

Last in buildout.cfg there's an 'addpath' section, which is a little hack to come around the fact that repoze has a dependency on zopelib which is an egg with all of zope2. But here I want to use the version of zope that is officially supported for the current plone version instead of the version that comes with repoze. So addpath runs a script that adds 'parts/instance/lib/python' to the top of the egg listing in bin/paster. The result is that the compiled zope will be used instead of repoze's zopelib.

You can find a buildout with all necessary files in collective trac or collective svn.

To make repoze/wsgi easier with buildout I think we would need to make a repozpe.zope2-like package without the plone and zope dependencies that repoze.zope2 currently has. Then you could pass 'extra_paths = ${zope2:location}/lib/python' to zc.recipe.egg and the compiled zope2 would be found. That would make [addpath] obsolete. This would be a first step to ease the use of repoze in buildout.

I had a great time at the snow sprint. I paired with Martijn Faasen and Robert Marianski and we implemented viewlets for Grok. It's nearly finished. It is working, but it needs a few more tests before it will be merged to trunk.

This is what your viewlet code could look like:

class PortalHeader(grok.ViewletManager):
    grok.name('header')

class Logo(grok.Viewlet):
    grok.viewletmanager(PortalHeader)
    grok.template('logo')

    def update(self):
        self.willBeAvailableInTemplate = 8

A template that wants to render the 'header' viewlet would include this:

<div tal:content="structure provider:header" />

I wrote a sample grok application, grokremoteinclude , to make sure the grok viewlets were compatible with other zope3 packages dealing with viewlets. It's a buildout, running it will install the grok app, nginx and varnish. Read the README.txt for instructions how to get going. The application uses SSI which allows viewlets to be either cached or retrieved from another server. The same technique can be used in Plone which was recently reported by Sasha Vincic.

Snowsprint 2008 is over for this year and many thanks to Lovely Systems and all participants.

Since Lovely Systems has deployed some fast sites I was eager to see if I could reuse some of it with Plone. One of the products was lovely.remoteinclude which enables viewlets to be accessed through own unique urls and therefore cached separately. With nginx you can then use SSI (server side include) to assemble the page instead of doing it all in Plone. The lovely.remoteinclude hooks up adapters for all IIncludableViews if they send the event zope.contentprovider.interfaces.IBeforeUpdateEvent. The adapter then checks if the view is called from within a page or directly. If it is called within a page the render method is changed so it renders the url to the view, and when called directly, no changes are made.

My task was to try to make Plone portlets accessible through urls. This was little bit harder then just marking the PortletManager and PortletRender with IIncludableView since they where not normal viewlets or viewlet mangers and there was no acquisition in portlets. Anyway after some pair programming with Tom Gross we managed to create a package valentine.remoteinclude (we didn't dare to name it plone.remoteinclude). This package is a proof of concept that lovely.remoteinclude can be used on Zope2 and Plone. It will provide urls for portlets that are available on the plone root. While we were learning the plone portlet machinery we found the namespace ++contextportlets++ which we reused to generate urls for our portlets. We also added a namespace for portlets since the ++viewlet++ didn't work for plone portlets. An url to a default calendar portlet will look like:

<!--#include virtual="/++contextportlets++plone.rightcolumn/++portlet++calendar" -->

Together with Whit Morriss and help from Tim Terlegård(who made lovely.remoteinclude work with Grok viewlets) we set up a buildout that will build a nginx and a varnish prepared for SSI and lovely.remoteinclude. The templates were made from a configuration provided by Lovely Systems. With this buildout and the valentine.remoteinclude enabled on a plone trunk builout we where enable to render a normal plone page where nginx assembled the calendar portlet with the rest of the page.

Tom and I couldn't figure out how to get the information from a portlet from which context the "mapping" (portlet storage) and then the portlet was fetched we where not able to generate urls for context specific portlets other then the root. This information seems to be hidden by the PortletFetcher in Plone. Ideas on how this could be solved are welcome.

With your help I think we can create a good SSI solution for Plone which would enable us to include structure and content in the page from different servers and have it assembled before sent to the user. This would be one step in the direction of using the right tool for the job and not put everything in Plone.

Technorati Tags: plone, zope, nginx, ssi, snowsprint

First day of Perfomance sprint in Copenhagen has passed and it really feels like we all have gotten productive from the start. 10 of us are seating at Symbion where Headnet is located and we have two remote sprinters.

Me and and Matt Hamilton started to work on improving the speed of cataloging while you move/rename a tree of objects. Matt had already an idea about using md5 hash on ZCTextIndex which in a quick test gave 30% speed improvement when you edit a document but don't change the body. But when we renamed objects we didn't get this improvement because the object got unindexed and then indexed on the new path so we tried do some magic in ObjectWillBeMovedEvent which would move the index in catalog for this object to the new path instead of unindexing and then get reindexed on ObjectMovedEvent instead of clean index. This way we could benefit from our speed improvement with md5 since the body isn't changed.

This was quickly done and the profiling after this indicated that object_provides indexing is taking way too long. We found out that the calls to zope.component.interface.interfaceToName was to be blamed for this. We tried some simple caching on it with plone.memoize and yeah! We slashed the indexing time by half on rename on a site with 300 random content objects! We let Martijn Pieters take a look at the method and why it was so slow just for an interface name. He found out that Plone really shouldn't use this for object_provides since it was there for a specific use case (I hope Martijn can comment on which) .

Martijn commited the fix, so current plone trunk has an indexing that is twice as fast as yesterday. This commit does not include the magic on ObjectWillBeMovedEvent and md5 so expect more speedups :)

NB! This fix is for trunk (plone 3.x) but me and others have the object_provides index in older Plones so please check the diff (see also) and fix your indexes to get some speed.

Thanks to Nate at Jazkarta for sponsoring, we got a nice sushi delux meal for dinner yesterday.

lovely.remotetask is a zope3 package for running asynchronous tasks. A service is running as a separate thread (not using any of zope's own threads). You define a class that performs a task. The task can be executed by adding it to the service (which adds it to its queue). It can look like this:

   >>> service = getUtility(ITaskService, name='LongRunningJob')
   >>> service.add(u'exampletask', 'input to the task')

The service will execute the task as soon as the other tasks in the queue are completed. The service and the task are registered as named utilities in zcml.

lovely.remotetask is ported to Zope 2.9 and works on 2.10 as well. It depends on zc.queue which runs on 2.9/2.10 without modification. The code is found in zope svn

The zmi (managing services and tasks) is not ported, but there is an example in svn that tells how to do it programmatically. Removing tasks can already be done in zmi as the utilities are added to the utilities folder in the site root.

Headnet sponsored this work.

Time for the annual Plone Conference and I must say I am very happy to see how the preparations are done. Looks like we will have a great conference and probably the largest one. Two days before the trip I looked through the agenda and as always it's hard to choose. First day we have Joel Burton's Building a humane CMS for Plone: updated tutorial and Martin Aspeli's Extending and Customising Plone 3 that are colliding. I am really interested to hear what Alexander Limi has learned at Google, so Simple and effective techniques for better usability is booked. If I choose Joel's talk I'll probably visit Geir Bækholt's Portlets in Plone 3. On the second day I can recommend the talk about the foundation by Paul Everitt but I am also very interested in Duco Dokter's Plone for the enterprise market: technical musing on caching, Clustering and Single Sign-On since this is the knowledge we need for big sites with Plone.

I'll probably end up changing my mind several times during the days and one or two regrets missing some talks but I hope it will be a nice time like all other conferences and a chance to meet friends.

Oh one more thing don't forget board nominations if you know anyone that should be on the board, even your self. Last day tomorrow Monday.

See you at the conference.

/Sasha