Valentine Blog

Merry Christmas and Happy New year

Sasha Vinčić — Mon, 28 Dec 2009 12:24:57 +0100

Working at Valentine Web Systems

Sasha Vinčić — Tue, 01 Dec 2009 11:13:41 +0100

Valentine Web Systems, a web development company based in Malmö Sweden, is always on the lookout for new crew members. If you're a skilled programmer looking for new challenges and horizons, this could be your chance.

Our job is to build web portals and web based solutions. We use Python as our main programming language and adopt the power of Cloud Computing. The open source system Plone is therefore our prime working tool.

The team members of Valentine Web Systems all has the privilege of enjoying a big deal of freedom in the job as the customers and the working project always are in our focus. This is how our planning is ruled. To become as efficient as possible in our work, keywords such as SCRUM, agile and pair programming are guiding stars on our heaven. Education is essential to us, which is why we attend international conferences and sprints in order to enrich ourselves with new knowledge about Plone and other interesting web technologies.

Are you a skilled Python programmer and also familiar with Plone and Web 2.0? Then that makes you the most wanted person for us.

Call or email us asap. You may send us your CV right ahead. Our office is located in Malmö, Sweden. You are more than welcome to work together with us there, but you may also work from your home or anywhere else if you like.

Plone powers 50GB of environmental data, maps and figures

Sasha Vinčić — Mon, 30 Nov 2009 09:59:46 +0100

Recently we upgraded our customers website for large data handling. The main development of the product eea.datatservice was done by our colleagues at Eau de Web in Romania, and our part was to prepare Plone 2.5 to handle large data.

Back to the future - blobs in Plone

The first solution for external storage was iw.fss, but due the cluster setup on our production server, we went with the blob approach because it is more future proof since it is part of the upcoming Plone 4 and doesn't require any shared read/write storage for all instances compared to iw.fss.

We tried out the existing branch of blob support for Plone 2.5 but it was old and lacked functionality, so we had to cut a new branch to backport the latest plone.app.blob. While Andreas Z was working on it we where back-porting the code more or less day by day :) Today the blob support for Plone 2.5 allows files and images to be stored outside ZODB in blobs.

In eea.dataservice we have custom content types with large files and images where some are 18000x16000px! All these maps and figures are converted in to number of different formats. All the images are scaled in different sizes and everything is stored in blobs. To store scales in blobs we backported plone.app.imaging which is used in Plone 4.

Fast but small servers - can't cache everything

In our cluster that is running on blade servers with very little local storage, we have to consider what to cache and where. The large storage on these servers is mounted from SAN, which is a fast and secure storage. The blob storage is mounted on the machine that runs the ZEO and additional to that we have smaller blobcaches for the instances on each machine, 25GB/machine. Since we can't fit the whole blobstorage in cache we clean it manually with a cron. In the newer ZODB3 and Plone there is configuration for automatic cleanup but we can't use it with Plone 2.5. This manual cleaning outside Plone has raised an issue where an instance expects the blob in the cache but it's not there. This happens if the object referring to the blob is in the object cache of the instance. If it is not the problem never arise since the blob is reloaded from ZEO. We are now testing different configurations for the object cache size before we try to catch those exeptions and try to reload the blob from ZEO instead.

Open source

All code used for this website is open sourced and available in Plone or collective repositories and Eionet SVN for i.e eea.dataservice.

Plone grew with 50GB of data and 20% more traffic

www.eea.europa.eu the Plone 2.5 site is now getting 20% more traffic and with 50GB more data than before the migration. Next step will be to migrate the multimedia and other large content to blobs which will probably free almost 10GB from the Data.fs.

Want to know more?

Please contact Sasha Vinčić for more information. Contact details on the right.

Multilingual and high security at Plone Conference 2009

Sasha Vinčić — Tue, 03 Nov 2009 10:37:29 +0100

The grid for the open space was filled up with over 40 talks in just of couple minutes, that is more then the scheduled ones.

Valentine Web Systems team was represented by Per Thulin and me, Sasha Vinčić. I gave a talk about 'Managing multilingual sites' which I hope gave some enlightenment about use cases and pitfalls with multilingual sites. You can see the slides below and recording at ustream.tv.

Manage Multilingual Sites

View more presentations from Valentine Web Systems.

Plone developers and integrators are as always busy working on interesting solutions. I understand why Microsoft's Jeff Tepper thought Plone is Sharepoints open source competitor. Plone may not be the most googled CMS but it is definitly the most secure and the biggest open source competitor to Sharepoint, Websphere and Documentum. Kees Hink and Kim Chee Leong told us how they hardened Plone to military strength. It was audited by Ernst & Young & Pine Digital Security granted certifiedsecure.eu certificate!

Valentine Web Systems provides multilingual and very secure portals built with Plone. If you want to know more, please contact us.

When we made internet history on Tjärö island

Sasha Vinčić — Mon, 31 Aug 2009 23:48:38 +0200

Tjärö is a small and idyllic island in the Karlshamn archipelago in the Swedish southern region of Blekinge. The island is own by the Swedish Tourism Foundation and is for example equipped with a youth hostel, a camping ground, a restaurant and conference facilities. In other words: Most of the things that are needed for an IT sprint with a big gang of happy geeks.

This was how the stage looked like when Tomas Wennström invited some of his friends to a meeting throughout August 21-23, where he expected 10 people at the most to show up. Well, he ended up facing a giant looking interest from people from everywhere to join the party. 286 persons from all the Nordic countries took the boat from Karlshamn to Tjärö to attend Sweden Social Web Camp 2009, as the sprint was called. Sasha Vinčić from Valentine Web Systems was one of them. Valentine Web Systems was, by the way, one of the sponsors of the event.

The main idea behind Sweden Social Web Camp (SSWC) was to give hard working internet geeks from different places a chance to join up and enjoy some nice time together. Just to get a voice and a face on some of them who nowadays turn up on Twitter and other social forums on internet was gold worth. The fact that people from Norway, Finland and Denmark also showed up gave the event a border crossing touch.

The sprint on Tjärö was not only about party and glamour but most of all a brainstorming society where everyone could share experiences and know how about Social Web. The scheduled sessions where the web people could share this valuable stuff with each other were very crowded. Dinner, brunch and peaceful moments by the camp fire were other opportunities for deep, thoughtful and friendly conversations

The aftertaste for every attendant of SSWC has been like enjoying a tasty candy bar. A sprint like this creates everlasting bounds of friendship and the words of joy and appreciation has been numerous after SSWC. Everyone agrees about the thought that this is far from a one hit wonder.

Says Tomas Wennström, the man behind SSWC: “I think we made internet history on Tjärö island.” You can´t describe it better than that.

Sprinting - Agile Development and Amazing Ways to Learn

Sasha Vinčić — Mon, 10 Aug 2009 14:30:56 +0200

Me, Sasha Vinčić(twitter), and Per Thulin(twitter) had an amazing time. This was Per's first sprint and he says it was everything he expected from what I have told him :) Over 20 people met up in Bristol to improve Plone CMS and have fun during the Balloon fiesta.

Performance testing and benchmarking

I worked with Tom Lazar, Alan Hoey and Matthew Wilkes on plone performance testing and benchmarking with funkload. We continued on the collective.funkload by Ross Patterson and added a recipe collective.recipe.funkload wich is a convinient recipe to add funkload to your buildout. With funkload you can then run tests, benchmark and create reports from the results to see how the performance changes. Andreas Zeidler showed me mr.ripley which takes a normal log in combined form and replays it against the domain url you tell it too. This can be used to test new configurations of same buildout or to warm up a site after restart so objects get cached. I then realized this can be used to create funkload tests and use them for benchmarking a site with real life traffic patterns. I added the proxy functionality to mr.ripley (not in svn as of writing) and run the replay through funkload recorder proxy and wola we have tests from real traffic.

TinyMCE replacing Kupu in Plone

Per worked with Ben Mason, James Guest and Owen Curtis-Quick on the TinyMCE track in order to improve the integration with Plone, especially with regard to inserting images. This involved both working out ways to provide available image size dimensions that have been configured to the specific content type, and making the user interface more consistent with the rest of Plone. TinyMCE is the content editor that will be used in future versions of Plone, and a big reason for that is that it produces clean and valid markup. Therefore, another important fix that was done was to add some missing attributes on the image tag being produced. Rob, the maintainer of Products.TinyMCE happily reports that all this work has been merged into trunk and thus will become part of the next release of Products.TinyMCE!

XHTML validation, transmogrifier, 64bit acquisition and much more

I just want to mention some other topics that were worked on during the sprint. XHTML validation through automatic testing, you can write tests to validate your markup. This is important for tools like Deliverance.

Hanno Schlichting worked on some deep stuff like fixing C code for acquisition on 64 bits machines. There was bugfixes for LinguaPlone and release of transmogrifier which is a powerful transformer. Formgen, override skin templates/scripts with view... and probably some things I have missed.

A great sprint organized by TeamRubber in Bristol!

Europython - cloud databases, python in browser, search

Sasha Vinčić — Wed, 01 Jul 2009 14:05:36 +0200

The first session I attended was FluidDB by Terry Jones. Terry described it as "database with heart of wiki" and its a description I can support from what I have learned. It is a database where objects don't have owners so anyone can add and change metadata(attributes/tags) on them. But the tags have owners and therefore permissions. The big benefit from this approach is that you can add tags or metadata about the object to the object it self instead of creating your own database. Some good examples are all those Twitter apps out there which add or calculate information about the twitter user or the tweets and then store them in their own database. If Twitter used FluidDB everyone could store it's own extra metadata where the original objects where. Now let see if all the promises will be fulfilled when FluidDB is released in a month or so. I really hope they will because I heard Terry sold his apartment to invest in this, that is what I call an entrepreneur! There were two more sessions related to database, one about MongoDB and one about CouchDB from Mozilla. The later uses the language Erlang to get the concurrency speed and it is a replicable and scalable database.

Javascript is THE language for web 2.0 applications and it's much simpler to develop today with all the AJAX libraries out there but still hard to keep up with all the different browsers and it's bugs. That is specially true if you don't use it daily, like a python programmer like me. I was pleasantly suprised to see Pyjamas come this long way. Today it's possible to develop your user interface in Python and have it compiled to javascript that runs in the browser. This is definitely something we will try out because it also enables you to create desktop applications. Luke Leighton who presented "The Zen of Pyjamas" even thought the HTML UI version was much nicer and had better quality then any widget library like PyQ4, PyGTK, wxPython. The main reason for this was that there had gone far more development hours into the web components.

Fulltext search is one powerfull feature in Plone that we use in our daily job. But on large websites it's good to separate the indexing and searching to separate server or service. The "Search with Python and Xapian" was a nice session by Richard Boulton where I saw how to use Python to index and search in Xapian as an external web service. I also saw Open Object in action from the Open ERP project and I was impressed how simple it is to create enterprise applications with it. Walking back to hotel I thought "What is the difference between Open ERP and SAP?" SAP has limitations on what it can do and no limitations on the cost, Open ERP has no limitations on what it can do because it's open source and the price well you decide. If you google for Open ERP and SAP this was top hit for me :) http://fptiny.blogspot.com/2008/10/when-sap-meet-open-erp.html

EuroPython 2009 - Python and it's future on the web and the cloud

Sasha Vinčić — Mon, 25 May 2009 22:30:44 +0200

Valentine Web Systems will be represented at EuroPython 2009 by company founder Sasha Vinčić who will attend the conference this year. Sasha Vinčić is looking very much forward to the EuroPython 2009 conference to get to learn more about what is going on in the wonderful world of Python. Cloud computing and web are interesting subjects.

Meeting old friends who also attend EuroPython 2009 is another highlight. Like Matt Hamilton from Netsight Internet Solutions who is giving the ”Lipstick on a Pig” talk about using Deliverance to skin third party system or Bernd Dorn from Lovely Systems with the talk ”Real World App Engine Projects” about how to get the best out of App Engine, and related technologies such as Bigtable, Amazon S3, EC2 and Cloudfront. There will of course be many more familiar faces to meet in the crowd, be sure about that.

There are a big number of talks that look really interesting to a happy Python fanatic from Sweden, such as Reliable Python Web Sites, Semantic Apps With CubicWeb, Tapping Into The Web Of Data, The Zen Of Pyjamas and Castles In The Cloud.

The week in Birmingham will be a joyride beyond all borders. See you there!

Plone 4 – The future looks really great

Sasha Vinčić — Thu, 14 May 2009 14:10:05 +0200

Want to know what’s going on inside the head of a happy Plone user? Now, this is your chance of a lifetime to do that. Netsight is twittering directly from the European Plone Symposium 2009 which is held in Sorrento, Italy this week. Imagine seven days of nothing but ”Ploning” in three different levels: Training sessions, symposiums and sprints. It has to be next door to heaven for a Plone fanatic.

You’re most welcome to follow Netsights everyday instant reports on Twitter at http://twitter.com/netsight. How about these excited Twitter comments from the European Symposium 2009: ”Just seen 30 second demo of Plone tiles and editing UI for new plone. Wow! Blown away at just how cool it is, really looking forward to it… Missed Underscore meetup :( In Sorrento for Plone Symposium :D Life is good… Plone 4 to be released in 2009, smaller subset of features than originally planned, but bringing cool stuff sooner.”

Netsight is continuously giving you more virtual comments and reports from beutiful Sorrento as the Plone Symposium proceeds throughout this week. Keep close to it at Twitter.

Even if we enviously follow the Big Event in Sorrento, sadly knowing that we couldn't be there this time, we make sure that our dear company Valentine Web Systems always catch up with the latest news about Plone and also secure that all our costumers are well equipped and prepared for the future to come.

The shoemakers kid has got new shoes – Plone 3.2 with a blog

Sasha Vinčić — Sun, 29 Mar 2009 22:37:39 +0200

We´ve spent the last nine years working a lot with web sites like EEA – European Environment Agency and Jeeves and we´ve got to learn so many good things when it comes to multilingual web sites. So far we´ve just written in technical English on our web site but from now on we would like to proceed and explain to you all why Plone is the most secure CMS and one of the best multilingual working tools.

First of all: Let´s all realize the fact that thousands of happy and satisfied users all over the world all states the same: Plone rules the world! Plone is the most powerful and easy-to-use CMS there is. It´s easy to install, use and extend. Plone lets non-technical people create and maintain information using only a web browser. It´s as easy as 1, 2, 3...

Ordinary people and big companies from all over the world has discovered all the great possibilities about Plone. Many of them are more than just rookies. We´re talking about really big guys like Amnesty International, the Government and the Parliament in Brazil, NASA, Nokia and a big number of universities and museums around the world. As we are long time Plone users we do feel honored to be a part of such a noble society.

Handling many different languages on a web site or a blog demands capability by the system. There are no written rules about what´s right or wrong. Please take a look at these two web sites and you´ll see what we mean. http://www.eionet.europa.eu/software/design/multilinguality/missingpagedilemma
http://www.eionet.europa.eu/software/design/multilinguality/navigation

The multilinguality of Plone is very flexible and creates new possibilities for you as a user to handle the cases when it comes to simple interpretations of the whole web site or complex lingual combinations where only some specific parts are interpreted. This means for example navigation that works in every level without bothering the user and interpreted tags that still means the same.

In handling big web sites like EEA automatic transportation is used for all material that needs to be interpreted or updated. The stuff is sent for interpretation and after that imported with a simple drag-n-drop funktion. According to our knowledge there are no other system that manage to do that!

Are you curious about Plone you might surf in to plone.org. All the needed info about Plone and all the latest updating info are to be found there. Are you looking for blogs and comments by Plone users you might as well visit planet.plone.org. Twitter is another source for info about Plone. twitter.com/valentineweb is where we share our experiences about Plone. If you´re in to statistics please don´t forget to visit the statistic blog about Plone.

When it comes to security there are no other system that can beat Plone. Plone had 10 security holes in a statistic comparison where, for example, Wordpress had 162 security holes, Joomla 324 and the language PHP as many as 9869 documented security holes compared to 65 in the language Python that Plone is written in.

How to add repoze to your plone buildout

terlegard — Mon, 18 Feb 2008 20:12:16 +0100

There is no straightforward way to run plone, repoze and buildout yet, atleast not if you want to use a plone/zope version other than the one repoze.plone ships with. Chris McDonough has this ticket assigned to him, so there will hopefully be an official and easy way soon. Until then this is how I got it to work.

Buildout.cfg:

[buildout]
parts = plone repoze zope2 instance addpath
eggs =
find-links = http://dist.repoze.org

[plone]
recipe = plone.recipe.plone

[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}

[repoze]
recipe = zc.recipe.egg
eggs =
     PasteScript
     WSGIUtils
     repoze.zope2
     deliverance
     ${plone:eggs}

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
http-port = 8080
debug-mode = on
verbose-security = on
eggs =
    ${buildout:eggs}
    ${plone:eggs}
products =
    ${plone:products}

[addpath]
recipe = z3c.recipe.runscript
update-script = addpath.py:main
install-script = addpath.py:main

This is not that different from a normal plone buildout. What's new is [repoze] which installs paste, deliverance and repoze eggs and scripts to enable wsgi.

When you want to start the instance you can either start it the old way without enabling repoze or wsgi::

  $ bin/instance fg

or you can start with repoze and wsgi enabled::

  $ bin/paster serve paste.ini

You need do add the paste.ini file to your buildout. The file says where it can find zope.conf, what server should run and what middleware should be in the wsgi pipeline.

Last in buildout.cfg there's an 'addpath' section, which is a little hack to come around the fact that repoze has a dependency on zopelib which is an egg with all of zope2. But here I want to use the version of zope that is officially supported for the current plone version instead of the version that comes with repoze. So addpath runs a script that adds 'parts/instance/lib/python' to the top of the egg listing in bin/paster. The result is that the compiled zope will be used instead of repoze's zopelib.

You can find a buildout with all necessary files in collective trac or collective svn.

To make repoze/wsgi easier with buildout I think we would need to make a repozpe.zope2-like package without the plone and zope dependencies that repoze.zope2 currently has. Then you could pass 'extra_paths = ${zope2:location}/lib/python' to zc.recipe.egg and the compiled zope2 would be found. That would make [addpath] obsolete. This would be a first step to ease the use of repoze in buildout.

Viewlets in Grok

terlegard — Thu, 31 Jan 2008 13:42:09 +0100

I had a great time at the snow sprint. I paired with Martijn Faasen and Robert Marianski and we implemented viewlets for Grok. It's nearly finished. It is working, but it needs a few more tests before it will be merged to trunk.

This is what your viewlet code could look like:

class PortalHeader(grok.ViewletManager):
    grok.name('header')

class Logo(grok.Viewlet):
    grok.viewletmanager(PortalHeader)
    grok.template('logo')

    def update(self):
        self.willBeAvailableInTemplate = 8

A template that wants to render the 'header' viewlet would include this:

<div tal:content="structure provider:header" />

I wrote a sample grok application, grokremoteinclude , to make sure the grok viewlets were compatible with other zope3 packages dealing with viewlets. It's a buildout, running it will install the grok app, nginx and varnish. Read the README.txt for instructions how to get going. The application uses SSI which allows viewlets to be either cached or retrieved from another server. The same technique can be used in Plone which was recently reported by Sasha Vincic.

Viewlets in Grok are really nice - no zcml and no macros. The viewlets are in a seperate branch if you want to try them out.

Proof of concept - enabling SSI on Plone 3 portlets

Sasha Vinčić — Thu, 31 Jan 2008 00:36:47 +0100

Snowsprint 2008 is over for this year and many thanks to Lovely Systems and all participants.

Since Lovely Systems has deployed some fast sites I was eager to see if I could reuse some of it with Plone. One of the products was lovely.remoteinclude which enables viewlets to be accessed through own unique urls and therefore cached separately. With nginx you can then use SSI (server side include) to assemble the page instead of doing it all in Plone. The lovely.remoteinclude hooks up adapters for all IIncludableViews if they send the event zope.contentprovider.interfaces.IBeforeUpdateEvent. The adapter then checks if the view is called from within a page or directly. If it is called within a page the render method is changed so it renders the url to the view, and when called directly, no changes are made.

My task was to try to make Plone portlets accessible through urls. This was little bit harder then just marking the PortletManager and PortletRender with IIncludableView since they where not normal viewlets or viewlet mangers and there was no acquisition in portlets. Anyway after some pair programming with Tom Gross we managed to create a package valentine.remoteinclude (we didn't dare to name it plone.remoteinclude). This package is a proof of concept that lovely.remoteinclude can be used on Zope2 and Plone. It will provide urls for portlets that are available on the plone root. While we were learning the plone portlet machinery we found the namespace ++contextportlets++ which we reused to generate urls for our portlets. We also added a namespace for portlets since the ++viewlet++ didn't work for plone portlets. An url to a default calendar portlet will look like:

<!--#include virtual="/++contextportlets++plone.rightcolumn/++portlet++calendar" -->

Together with Whit Morriss and help from Tim Terlegård(who made lovely.remoteinclude work with Grok viewlets) we set up a buildout that will build a nginx and a varnish prepared for SSI and lovely.remoteinclude. The templates were made from a configuration provided by Lovely Systems. With this buildout and the valentine.remoteinclude enabled on a plone trunk builout we where enable to render a normal plone page where nginx assembled the calendar portlet with the rest of the page.

Tom and I couldn't figure out how to get the information from a portlet from which context the "mapping" (portlet storage) and then the portlet was fetched we where not able to generate urls for context specific portlets other then the root. This information seems to be hidden by the PortletFetcher in Plone. Ideas on how this could be solved are welcome.

With your help I think we can create a good SSI solution for Plone which would enable us to include structure and content in the page from different servers and have it assembled before sent to the user. This would be one step in the direction of using the right tool for the job and not put everything in Plone.

Technorati Tags: plone, zope, nginx, ssi, snowsprint

Indexing in Plone got twice as fast

Sasha Vinčić — Fri, 02 Nov 2007 10:43:29 +0100

First day of Perfomance sprint in Copenhagen has passed and it really feels like we all have gotten productive from the start. 10 of us are seating at Symbion where Headnet is located and we have two remote sprinters.

Me and and Matt Hamilton started to work on improving the speed of cataloging while you move/rename a tree of objects. Matt had already an idea about using md5 hash on ZCTextIndex which in a quick test gave 30% speed improvement when you edit a document but don't change the body. But when we renamed objects we didn't get this improvement because the object got unindexed and then indexed on the new path so we tried do some magic in ObjectWillBeMovedEvent which would move the index in catalog for this object to the new path instead of unindexing and then get reindexed on ObjectMovedEvent instead of clean index. This way we could benefit from our speed improvement with md5 since the body isn't changed.

This was quickly done and the profiling after this indicated that object_provides indexing is taking way too long. We found out that the calls to zope.component.interface.interfaceToName was to be blamed for this. We tried some simple caching on it with plone.memoize and yeah! We slashed the indexing time by half on rename on a site with 300 random content objects! We let Martijn Pieters take a look at the method and why it was so slow just for an interface name. He found out that Plone really shouldn't use this for object_provides since it was there for a specific use case (I hope Martijn can comment on which) .

Martijn commited the fix, so current plone trunk has an indexing that is twice as fast as yesterday. This commit does not include the magic on ObjectWillBeMovedEvent and md5 so expect more speedups :)

NB! This fix is for trunk (plone 3.x) but me and others have the object_provides index in older Plones so please check the diff (see also) and fix your indexes to get some speed.

Thanks to Nate at Jazkarta for sponsoring, we got a nice sushi delux meal for dinner yesterday.

lovely.remotetask available for Zope 2.9/2.10

terlegard — Mon, 08 Oct 2007 17:26:47 +0200

lovely.remotetask is a zope3 package for running asynchronous tasks. A service is running as a separate thread (not using any of zope's own threads). You define a class that performs a task. The task can be executed by adding it to the service (which adds it to its queue). It can look like this:

   >>> service = getUtility(ITaskService, name='LongRunningJob')
   >>> service.add(u'exampletask', 'input to the task')

The service will execute the task as soon as the other tasks in the queue are completed. The service and the task are registered as named utilities in zcml.

lovely.remotetask is ported to Zope 2.9 and works on 2.10 as well. It depends on zc.queue which runs on 2.9/2.10 without modification. The code is found in zope svn

The zmi (managing services and tasks) is not ported, but there is an example in svn that tells how to do it programmatically. Removing tasks can already be done in zmi as the utilities are added to the utilities folder in the site root.

Headnet sponsored this work.