jmoiron.net

I've been doing a lot with CouchDB and WSGI the past few months, with positive and negative results. I'm finally getting the "hang" of Document Orited Databases (DODB? Rubbish acronym), and starting to understand what is possible and what isn't, how to do one-to-many and many-to-many relationships without having lots of queries. It struck me that although the per-query time in CouchDB is thusfar a lot slower than relational databases, the overall database time for individual webpage loads is far less, and there are far less queries and complex information joins going on.

There are a few reasons for this. The first is that Django is a general purpose framework, and it's ORM is similarly meant to be general purpose. The result is that getting the comment counts for 10 posts on the front page takes 10 queries. This type of a query will be possible in one view in CouchDB when Reduce is implemented, but the only way to do this without modifying your "data schema" is to get all of the comments and count them manually.

Because data is so fluid, it's no problem to add a "comments_n" field to each of your commented-on documents. This kind of application specific update-on-write hack is probably going to become very commonplace as CouchDB gains mindshare and the "best practices" are discovered, not because there are inherent limitations with the view system (or at least, there won't be, once it's done), but because the lack of a schema makes it extremely easy, and it's always more efficient to write the application this way than to calculate it over and over.

In light of all this tinkering, I've done a few things. I've started to take a closer look at Erlang especially of late, as I've started to use CouchDB trunk instead of 0.7.2. I've also done a lot of python speed & feature related hacking with both the more or less official couchdb-python library and with my own re-implementation using pycurl instead of httplib2 which I've called curlcon. Originally, I had devised an httplib-style replacement object that used curl as it's transport layer (Hence Curl Connection) to be plugged into python-couchdb, but it evolved into a sort of experiment to see how fast I could get the http part of CouchDB to run.

The code is now available in my mercurial repository under the "slipcover" repository. As I start to add testing to some of my "web framework" for the next version of jmoiron.net, "slipcover" will become more of a full project whose purpose (besides to run my website) is to be an example of an idiomatic CouchDB/Python web application.