Django and Thrift

Django Models Exposed

I've been looking for a better way to expose django models to other systems. I've generally preferred to use XML-RPC for its simplicity and ease of use within Python. Most of the time its fast enough, but there is some overhead whenever you are parsing XML. Coding XML-RPC clients and servers in languages like C# and Java, while the libraries are pretty good, feels verbose and brittle when compared to Python. There's a lot of work building objects that represent the exact input and outputs of your methods.

Thrift

Thrift is a framework for modeling the objects and interfaces for cross-language services. Originally developed at Facebook, Thrift is now an Apache Incubator project. Out of the box, you get the Thrift IDL (Interface Description Language), and the thrift compiler, which generates the objects and service implementation code for the languages you want to use for clients and servers.

Django as Thrift Service Host

While Thrift comes with high-performance socket servers out of the box, leveraging an existing highly-available web application stack has advantages. Here's a quick working example of a Django model exposed as a Thrift service.

Our (very simple) models.py:

from django.db import models

class Thing(models.Model):
    name = models.CharField(max_length=255)
    description = models.TextField()

Our corresponding Thrift interface tmodels.thrift:

namespace java net.redmoxie.examples

struct Thing {
    1: i32 id,
    2: string name,
    3: string description,
}

typedef list<Thing> Things
typedef i32 IdNum

service ThingService {
    Thing getThing(1:i32 id),
    IdNum saveThing(1:Thing thing),
    Things getAllThings(),
}

Here we've defined a struct to contain our object 'Thing', defined a couple of custom types, and defined our service contract with three methods.

Now we can call the thrift compiler on our IDL file:

thrift -gen py -gen java tmodels.thrift

Now we have 2 new folders in our current directory: gen-py and gen-java. Inside of the gen-py directory, there is a generated Python module 'tmodels'. Let's copy that up one level to our app directory.

Now for the view code we'll create tviews.py:

import sys,traceback

from django.http import HttpResponse, HttpRequest

from thrift.protocol import TBinaryProtocol
from thrift.transport import TTransport

from MyApp.tmodels import ThingService
from MyApp.tmodels.ttypes import Thing as TThing
from MyApp.models import Thing

class ThingServiceHandler:
    '''
    Naive thrift service handler for Thing objects
    '''
    def getThing(self, id):
        print id
        thing = Thing.objects.get(pk=id)
        return TThing(name=thing.name.encode('utf-8'),
                        description=thing.description.encode('utf-8') )

    def saveThing(self, t_thing):
        thing = Thing.objects.create(name=t_thing.name,
                                     description=t_thing.description)
        return thing.id

    def getAllThings(self):
        t_things = []
        for thing in Thing.objects.all():
            t_things.append(TThing(name=thing.name.encode('utf-8'),
                        description=thing.description.encode('utf-8') ) )
        return t_things

def ThingServiceView(request):
    '''
    Process thrift ThingService requests
    '''

    ## Request deserialization
    transportIn = TTransport.TMemoryBuffer(request.raw_post_data)
    protocolIn = TBinaryProtocol.TBinaryProtocol(transportIn)

    ## Response serialization
    transportOut = TTransport.TMemoryBuffer()
    protocolOut = TBinaryProtocol.TBinaryProtocol(transportOut)

    ## Processing
    handler = ThingServiceHandler()
    processor = ThingService.Processor(handler)
    processor.process(protocolIn,protocolOut)
    transportOut.flush()

    ## Response
    data = transportOut.getvalue()
    return HttpResponse(data, content_type='application/x-thrift')

Not much to it. We create a class that implements the methods we defined in our Thrift IDL interface and wire up our view to deserialize the request, process the service call, serialize the response, and return it to the client.

Then we add a line like this to our urls.py:

    (r'^thrift/', 'MyApp.tviews.ThingServiceView'),

In part II we'll look at the Java Thrift client that can connect to our service.