Broken Django behind a Load Balancer

Having problems with your seven Django servers behind a single load balancer or proxy, and you have no idea which of the servers is giving your that “one in eight requests” error?

The solution is simple, add some information about the instance to the HTTP response headers!

It doesn’t really matter which load balancing or proxy solution you use, Amazon ELB, HAProxy, Varnish, Pound, Nginx, this would just work with most of them without any modification.

Here is a simple example of just such a middleware:

Simple Amazon AWS API queries

After using the excellent python boto Amazon Web Services library, I felt a tingle of unease where boto was taking away the transparency and clarity of AWS APIs. Amazon have excellent documentation that contains detailed API References, and the pace of their new features released is staggering. It is a pity that boto is written in such a way that it needs to re-implement every new feature Amazon releases in their own wrappers and a foreign language (method names).

So as the first step in making it more transparent to use the original Amazon AWS API reference documentation in a copy-and-paste fashion, I had to write the same reinvent-the-wheel piece of boiler plate that everyone wrote a hundred times before (google for it, its there). But mine is shinier and prettier, or so I would like to think.

I present to you, in all its glory — The Python Amazon AWS API queries class!

Google AppEngine URLFetch in Unit Tests

Started using Google AppEngine for a personal project of mine some time ago, and noticed that like everywhere else in python, the state of testing (tdd) is really poor.

There are several “solutions” that provide stubs that can be used in unit testing Google AppEngine applications, including something called a “testbed” which is part of the API itself. But the problem with these is that they provide functional bits of API implemented on your local environment just like it would work on a deployed AppEngine application.

It sounds quite good to have a local personal instance of something similar to the datastore you get in deployed applications, but unfortunately for the urlfetch service it is not exactly what I was looking for in tests.

The thing I need is an object that will not urlfetch anything, will not access the network at all. The requirement in this case is an object that I can tinker with its state before and after my own methods have used the urlfetch facility. After a lot of digging in the current implementation of the stub, I ended up writing a very simple mock for this myself. It is far from perfect, but its a start.