I’ve recently been working on an implementation of cookie-based authentication for CouchDB. This is important for pure CouchDB applications (couchapps), where browsers communicate directly with CouchDB. Currently browsers can be authenticated using HTTP basic auth but the popup login box can be disruptive and confusing for users.

Implementation

The cookie part itself was pretty straightforward. The basic idea is that once the user has been authenticated via a traditional form-based login, they are given a timestamped, tamper-proof token in the form of a cookie. Whenever CouchDB receives a cookie with a valid token, it authenticates the user for that request. It then sends a new token with a new timestamp. For performance reasons, it only sends a new token if the timestamp is more than something like 10 minutes old.

No state is stored on the server-side, so that this can be easily used in a cluster. Note that the clocks need to be synchronised somewhat so that the timestamps can be verified.

The cookie is of the form:

username + ':' + timestamp + ':' + HMAC(username + ':' + timestamp)

where HMAC is a secure HMAC signature e.g. HMAC-SHA-256. The timestamp and signature together provide a level of forward-security i.e. a passive eavesdropper cannot re-use a captured cookie in the future outside of the valid timestamp window. This protects against cookies being re-used in the event that a hard disk is stolen and the cookies haven't been erased.

The code is here: http://github.com/jasondavies/couchdb/tree/cookie-auth

Usage

Add the following to local.ini:

[httpd]
authentication_handler = {couch_httpd_auth,cookie_authentication_handler}

I have added per-db _login and _logout handlers to default.ini, but if you want to set up per-node login/logout handlers, use something like the following:

[httpd_global_handlers]
_login = {couch_httpd_auth, handle_login_req, "userdb"}
_logout = {couch_httpd_auth, handle_logout_req, "userdb"}

The final step is to create a special _design/_auth design document containing a users view mapping usernames to {password_sha: base64(sha1(password)), salt: <random salt>, roles: [<role name>, ...]} and a secret member.

Logging in and out should now be as simple as POSTing a username and password to http://127.0.0.1:5984/mydb/_login. An optional query parameter next may also be added to specify a redirect for successful login/logout.

Authentication Protocols

There is a plethora of authentication protocols out there, of varying degrees of security. The vast majority of Web sites use a simple plain-text form-based login though. Why? Firstly for most Web sites, the security requirements are pretty low. Who's going to want to steal my Digg login? For higher security just add TLS/SSL and immediately you are protected against a variety of active and passive attacks. The other reason is that if JavaScript is turned off, the only alternative to form-based plaintext login is HTTP Basic or Digest, where the designer has no control over the login UI. HTTP Basic is essentially equivalent to plain-text, and digest is slightly more secure but still has a bunch of problems.

With CouchDB's strong emphasis on JSON and JavaScript, I wanted to find out if I could crank up security a notch for unsecured HTTP if I assumed that JavaScript is enabled on the client.

Ideal Requirements

1. Reasonable performance/simplicity of JavaScript implementation
2. Mutual authentication
3. Resistance to off-line dictionary attacks based on passive eavesdropping
4. Passwords stored in a form that is not plaintext-equivalent
5. Limited resistance to replay attacks i.e. outside a certain time window

SRP

Invented by Tom Wu in 1998, the Secure Remote Password protocol is highly impressive. It has the following properties:

• Resistant to dictionary attacks by both passive and active network intruders
• Offers perfect forward secrecy, which protects past sessions and passwords against future compromises
• User passwords are stored in a form that is not plaintext-equivalent to the passwords themselves

Wow!

Cryptography in JavaScript

Drunk on the awesomeness of SRP, I went away and implemented it. Unfortunately, I discovered that the performance of modular exponentiation in JavaScript is not good enough for SRP to be usable with a sufficiently large modulus (1024 bits) on today's machines. A 256 bit modulus performs perfectly well but this is not really big enough to resist attacks by an organisation with sufficient resources.

From Tom Wu:

Unfortunately, 256 bits is way to small for any decent amount of security. 256 bits is less than 80 digits, and numbers of that size can be factored on a desktop PC in a few minutes these days. Even 512 bits is starting to look insecure for “casual” use.

Active Attacks

There is another problem here, which is that the JavaScript client itself will be transmitted over the same insecure network over which the authentication protocol is communicated. SRP itself assumes that the client is trusted, but this particular scenario is vulnerable to an active attacker who might simply inject a fake JavaScript client that, for example, sends the user's password in plaintext over the network for the attacker to capture.

The only way to prevent this would be to have an unspoofable browser UI of some kind (e.g. provided by a browser addon). Signing the JavaScript code would be another possibility but this is currently only supported by Mozilla browsers.

SCRAM and other alternatives

Tom Wu suggested looking at CHAP or SCRAM as a potential auth protocol that would still be benefitial to use over HTTPS.

Due to limited time, for now I've opted for a simple plain-text form-based login in my branch, which as I mentioned above is what everyone else uses anyway. It's good enough for most purposes and for everything else we have TLS/SSL. Something like SCRAM would be preferable though, as the password itself is never actually sent to the server, even if TLS/SSL is used.

Conclusion

SRP is impressive, the only other authentication protocol with similar properties that I know of is J-PAKE. However, a JavaScript implementation is unsuitable on today's browsers due to the slow performance of modular exponentiation at 1024 bits. In addition, serving the client implementation over an unsecured connection is itself vulnerable to active attacks, thus for CouchDB, a simpler protocol like SCRAM will give us most of the properties we need over an unsecured HTTP connection, and one can simply add TLS/SSL to protect against active attacks.

Photo credit: Eliot Phillips

Ely Service now runs on CouchDB. Things just got a little simpler: no more Django plus PostgreSQL plus Nginx.

Casual Lofa: the World's fastest furniture

Ely Service is, as J. Chris Anderson put it, “just a very ordinary-looking garage Web site”. It's a simple Web site, which I originally developed using Django. It consists of six pages, one of which has a contact form for sending emails. So the requirements are very straightforward.

Why switch?

This was an experiment to see how easy it is to develop a simple Web site using CouchDB and (almost) nothing else. Ely Service is essentially a static Web site, and hence barely exploits any of the roaring power of CouchDB's B-Tree index or its distributed capabilities.

CouchApp

CouchApp is a set of scripts that make developing standalone CouchDB applications a lot simpler. Using Futon to do this at the moment is far too painful, although I could imagine a lightweight IDE that allows various show/list functions to be previewed as they are developed. Patches welcome!

In a nutshell, CouchApp allows you to store your map/reduce views, lists, shows and validation functions as files in a directory tree. You can also include various helper functions and templates, which are inserted using macros before being pushed to the database.

I put the majority of the Ely Service site into its own app and the contact form handler into a separate app. Complex sites may consist of many apps that work together. Here is the structure of the "elyservice" application:

elyservice/
  _attachments/
  lib/
    helpers/
      ejs.js
  templates/
    layout/
      head.html
      tail.html
  shows/
    contact.js
    page.js

Show Me

As this is a simple site with only 6 static pages, these are all generated using simple "show" functions.

function (doc, req) {
  // !json templates
  // !code lib/helpers/couchapp.js
  // !code lib/helpers/ejs.js
  var body = new EJS({
    text: templates.layout.head + templates.page + templates.layout.tail
  }).render({
    assets: assetPath(),
    doc: doc
  });
  return {
    headers: {'Content-Type': 'text/html; charset=UTF-8'},
    body: body
  };
}

Using CouchDB to send E-mail

This is the most complex part of the site, as it requires the use of an external process to send the emails. Strictly speaking, an external process is not necessary; a cron job would also do the job just fine.

I decided to write this as a generic CouchApp so it could be reused across multiple sites. Pretty much every site has a contact form of some kind.

This works like a standard UNIX mail spooler. New messages are created with a status of "spool", and the notification script sets the status to "sent" when it has finished sending. Unsent messages are retrieved by the notification script by calling the "mail_spool" view:

function (doc) {
  if (doc.type == 'mail' && doc.status == 'spool')
    emit(null, null);
}

The actual sending of email is done by the send_emails.py script, which is launched as an external process.

I've put the contact form code here, including the mail spooler: http://github.com/jasondavies/couchdb-contact-form/tree/master

Nginx Configuration

One of the only remaining hurdles to truly pure CouchApps is support for clean URLs. I wanted to retain the clean URLs of the original Ely Service site, and in order to do this I had to rewrite them using a reverse proxy. Nginx was ideal for this task.

server {
    listen 89.145.97.172:80;
    server_name www.elyservice.co.uk;
    set $projectname elyservice;

    location / {
        if ($request_method !~ ^(GET|HEAD)$) {
            return 444;
        }

        proxy_pass http://127.0.0.1:5984/elyservice;
        proxy_redirect default;
        proxy_set_header X-Orig-Host '$host:$server_port';

        rewrite ^/media/(.+)$ /$projectname/_design/elyservice/$1 break;
        rewrite ^/$ '/$projectname/_design/elyservice/_show/pages' break;
        rewrite ^/(.*)/$ '/$projectname/_design/elyservice/_show/pages/pages:$1' break;

        return 404;
    }

    location /contact/ {
        if ($request_method !~ ^(GET|HEAD|POST)$) {
            return 444;
        }

        proxy_pass http://127.0.0.1:5984/elyservice;
        proxy_redirect default;
        proxy_set_header X-Orig-Host '$host:$server_port';

        if ($request_method = POST) {
            rewrite ^/contact/$ /$projectname/ break;
        }
        rewrite ^/contact/$ '/$projectname/_design/elyservice/_show/contact' break;

        return 404;
    }
}

It turns out that Nginx automatically decodes URL-encoded characters in rewrite URLs before passing them through the proxy. Hence I couldn't use "pages/foo" for my docids. No problem here, I simply elected to use "pages:foo" instead.

It's worth noting that support for a CouchDB rewrite handler is under active discussion at the moment, so watch this space.

Security and Validation

Validation is very important; I don't want d00dz being able to edit any document in the database. All it takes is a POST or a PUT and anyone can create or update any document. To prevent this, first of all I added an admin user to local.ini. This user is given the special role of "_admin", which has the special priviledge of being able to create and modify design docs.

However, this level of security is not enough, as someone could still PUT malicious text to the home page doc for example.

A simple way to prevent this is to configure Nginx to reject any requests that aren't HEAD or GET:

if ($request_method !~ ^(GET|HEAD)$) {
    return 444;
}

Note: the non-standard error code 444 causes nginx to drop the connection (see https://calomel.org/nginx.html). The standard "forbidden" error code 403 could be used instead.

There may be cases, though, where we want users to be able to create/modify some documents but not others. For Ely Service, we want anonymous users to be able to create new documents of type "mail", but nothing else. This is where validate_doc_update comes in handy.

function (newDoc, oldDoc, userCtx) {
  // !code _attachments/validate.js
  if (userCtx.roles.indexOf('_admin') != -1) {
    return;
  }
  if (oldDoc == null) {
    return validate(newDoc);
  }
  throw {
    forbidden: "Invalid operation: existing messages cannot be modified."
  };
}

Conclusion

Although CouchDB is still alpha software, developing and deploying a simple Web site using CouchApp was very straightforward. The real benefits of CouchDB were not exploited at all, but we'll see some of that in a future post.

Several people have noted that Ely Service loads very quickly. This is a combination of CouchDB's raw speed and the simplicity of Ely Service's design.

After some hacking about with the rather badly documented Python FUSE bindings, I have finally managed to mount a CouchDB document's attachments directly onto a virtual filesystem!

Use Cases

• If you've read My Couch or Yours? Shareable Apps Are The Future by jchris, this is a great time-saver if you want to edit HTML, JavaScript, CSS or even image files directly using your favourite editor.
• Uploading large numbers of files repetitively through Futon or even via a Python prompt becomes tedious very quickly: drag'n'drop or cp * is the way forward!

Installation

You need the following:

1. Python FUSE bindings
2. CouchDB-Python 0.5 or greater
3. CouchDB-FUSE

Running python setup.py install should install a couchmount script on your path.

Usage

$ mkdir mnt
$ couchmount http://localhost:5984/jasondavies/_design%2Flinks mnt/
$ ls mnt/
$ touch mnt/foo
$ ls mnt/
foo
$ 

...you get the idea...

Happy Couching!

Update: Paul Davis has a nice alternative for automatically synchronising files if you're using couchapp:

inotifywait -m -r --exclude "\.swp$" -e modify . | xargs -n 1 -I {} echo "couchapp push . wbm-targets" | bash

My productivity this week, according to RescueTime:

Looks like I've been spending too much time on the CouchDB...

Finally, Django 1.0 is released!

After countless hours of hard work by many talented people, it is a pleasure to see the long-awaited version 1.0 released to the world. Finally, a version number that reflects the quality of the codebase, which in fact I have been using in production for many months.

New features that I am excited about include:

Re-factored admin application

Developed under the newforms-admin branch for a long while, the re-factored admin application is much more customisable and means I don't have to do so many ugly hacks and workarounds any more. See the admin reference for details.

Re-factored ORM

The internals of the ORM were pretty much rewritten from scratch in the queryset-refactor branch, resulting in many annoying bugs being fixed. This also includes support for model inheritance, which allows for more elegant models. See the wiki for more details.

Have fun!

Awesomeness, from Brian Rosner:

Django code_swarm from Brian Rosner on Vimeo.

DebugFooter is a handy little piece of middleware that Simon mentioned during his talk on Monday.

Adds a hidden footer to the bottom of every text/html page containing a list of SQL queries executed and templates that were loaded (including their full filesystem path to help debug complex template loading scenarios).

To use, drop in to a file called 'debugmiddleware.py' on your Python path and add 'debugmiddleware.DebugFooter' to your MIDDLEWARE_CLASSES setting.

Twubble searches your Twitter friend graph and picks people you might like to follow.

This is the kind of thing that Facebook should let developers do. Due to the restrictive developer API, however, you can only see friends of friends who are friends with you.

According to Twubble, I should make friends with factoryjoe!