CouchDb updates

Last weekend, Damien committed the results of a few weeks worth of intensive work on CouchDb to the public Subversion repository.

Fabric is no longer

To speedily gain access to your documents in CouchDb, you create views that define a certain structure. Up to now, Damien’s home-grown Fabric language was used to describe the views. This is no longer the case. You now simply use Javascript functions to do it.

XML’s gone too (thank god)

For a couple of months now, we were discussing the merit of switching from XML to JSON for transferring data from and to CouchDb. The only advantage of XML was that it was in already. JSON, on the other hand, would add so much goodness that we just couldn’t hold it back any longer.

I began working on the transition in June. During August, Damien picked up the work and eventually committed most of it over the weekend, while I spent some time getting the build-process done and looking for ways to integrate Spidermonkey, Mozilla’s Javascript Engine.

The Reasoning

CouchDb now internally uses the JSON representation to handle documents. We got rid of a whole lot of XML-related boilerplate code in the process — and less code is always good. JSON then allowed us to get rid of Fabric too, the advantage being that people interested in CouchDb didn’t have to learn yet another language. Javascript is pretty well known by now and easy to pick up, too. At last, we even added another buzzword to the ever growing list of CouchDb’s features. Views are now created using a map-reduce architecture. An architecture that has already been proven to be massively scalable.

How then?

CouchDb documents are now in fact first class JSON objects:

{
    "id":1234,
    "rev":1,
    "type":"Person",
    "name":"John",
    "age":24
}

Defining a view is now as easy as sending a Javascript function to CouchDb. It receives one document at a time, you do whatever calculations you need and then return either the data you want in the view or nothing, if the document should not be in the view. It’s this easy:

function(doc)
{
    if("Person" == doc.type) {
        return doc.name;
    }
}
    

What happens here? You define an anonymous function that expects a single argument, a document. You then test the document, is it of a Person-type? If it is, put its name-value into the view. If it’s not, do nothing.

This gets repeated for every document you send in and the views you later query get built up in the process, making retrieving your data an instant operation.

Amazing

This allows you not only to get information back in the way of

Give me all the documents from July that have more than 2,500 words in their Text field.

but also

Give me all the documents from July where a the Potato field is a two-element array where the first element is another JSON object with 7 fields, one of which has the value 24.

It not only frees you from the Excel-way of thinking in rows and columns, it lets you model your data in free form. And you can still search through it in no time.

Get your code now (Try the «json» branch to try things out [Update 9/7/07: The JSON branch was merged back to trunk]).

Post Scriptum

The Javascript engine is hooked up in a way that you can easily plug in any other language. So you could as well write your table definitions in your favourite scripting language, or real™ programming language.