dot.Using Making technology about computers, and computers about usability.

Broken Django behind a Load Balancer

Having problems with your seven Django servers behind a single load balancer or proxy, and you have no idea which of the servers is giving your that “one in eight requests” error?

The solution is simple, add some information about the instance to the HTTP response headers!

It doesn’t really matter which load balancing or proxy solution you use, Amazon ELB, HAProxy, Varnish, Pound, Nginx, this would just work with most of them without any modification.

Here is a simple example of just such a middleware:
https://gist.github.com/2393220#file_hostname_middleware.py

Simple Amazon AWS API queries

After using the excellent python boto Amazon Web Services library, I felt a tingle of unease where boto was taking away the transparency and clarity of AWS APIs. Amazon have excellent documentation that contains detailed API References, and the pace of their new features released is staggering. It is a pity that boto is written in such a way that it needs to re-implement every new feature Amazon releases in their own wrappers and a foreign language (method names).

So as the first step in making it more transparent to use the original Amazon AWS API reference documentation in a copy-and-paste fashion, I had to write the same reinvent-the-wheel piece of boiler plate that everyone wrote a hundred times before (google for it, its there). But mine is shinier and prettier, or so I would like to think.

I present to you, in all its glory — The Python Amazon AWS API queries class!

Google AppEngine URLFetch in Unit Tests

Started using Google AppEngine for a personal project of mine some time ago, and noticed that like everywhere else in python, the state of testing (tdd) is really poor.

There are several “solutions” that provide stubs that can be used in unit testing Google AppEngine applications, including something called a “testbed” which is part of the API itself. But the problem with these is that they provide functional bits of API implemented on your local environment just like it would work on a deployed AppEngine application.

It sounds quite good to have a local personal instance of something similar to the datastore you get in deployed applications, but unfortunately for the urlfetch service it is not exactly what I was looking for in tests.

The thing I need is an object that will not urlfetch anything, will not access the network at all. The requirement in this case is an object that I can tinker with its state before and after my own methods have used the urlfetch facility. After a lot of digging in the current implementation of the stub, I ended up writing a very simple mock for this myself. It is far from perfect, but its a start.

I think I really dislike Python

Ruby:

> irb
irb(main):001:0> a = [1,2,3]
=> [1, 2, 3]
irb(main):002:0> a[10] = 20
=> 20
irb(main):003:0> a
=> [1, 2, 3, nil, nil, nil, nil, nil, nil, nil, 20]

Perl:

> perl
@a = (1,2,3);
$a[10] = 20;
use Data::Dumper;
print Dumper(@a);
$VAR1 = 1;
$VAR2 = 2;
$VAR3 = 3;
$VAR4 = undef;
$VAR5 = undef;
$VAR6 = undef;
$VAR7 = undef;
$VAR8 = undef;
$VAR9 = undef;
$VAR10 = undef;
$VAR11 = 20;

Python:

> python
>>> a = [1,2,3]
>>> a[10] = 20
Traceback (most recent call last):
  File "", line 1, in ?
IndexError: list assignment index out of range
>>> a
[1, 2, 3]

A bright idea in the middle of the day

I was reading a blog yesterday about “The sad state of open source monitoring tools” and was thinking about it for some time. Coincidently today I had a chance to look at my CruiseControl configuration files, which I wrote quite a long time ago.

I really love the DSL that CruiseControl is using for it’s configuration, it’s extremely powerful at describing how to build projects. Especially powerful are the variables, that unlike in Ant are not immutable, and the way plugins can be pre-configured with your own defaults, as well as renamed to other names. It’s really easy to configure it in such a way that adding a new version for a project is just 1-3 lines of XML, for example

<xxx-project name="XXX v6.66">
  <property name="version" value="6.66"/>
</xxx-project>

Just in those 3 lines, the pre-configuration already includes all the information about the project. Where it is at, who to send e-mail to, where is the version control, EVERYTHING! If the only variable that changes over time is the version number, then that is all you need to leave as a variable … everything else is just a template that can be re-used. And these templates are extremely easy to combine from smaller templates, it’s a template-in-the-template kind of configuration.

IMHO this would very much apply to configuration of monitoring software, like nagios for example. And the way the (CC) plugins are written in java – adding new plugins that check all kinds of esoteric things is really easy to do.

If it would also have the XML/XSLT configuration of how the web-interface looks like (the way CruiseControl does), and the super-easy installation (again like in CruiseControl). It would be a really really really great product, extremely powerful, easy to configure, and potentially great looking.

If only ThoughtWorks would write such a thing … I would be thrilled!

Actually nagios is already extremely similar to what I described, but for some strange reason I find the rigid configuration of nagios a large PITA. Maybe some-day when time stops and I will have unlimited time to code, I will do it myself.

Testing messy bash scripts

GMail is broken

What happened to GMail lately?
It steals focus!
Contacts don’t work most of the time!
Mail is often not sent because the button pushing has no effect.

Is this GMail 2.0?
Fuck 2.0, I want the working GMail back!
And stop it from stealing focus to it’s own tab, that is SO annoying.

A small feng shui for the office

Just bought this today. It’s compact. Made these nice fire things with a red gift band. If only I could get rid of my room-mate and all his junk, this office could be so much better. It could be perfect.

The Motherload of all switches

When I read a news article with something like this in it:

Thankfully, Sun realizes that not everyone is looking for a 3,456 port InfiniBand switch. So, it has prepped a 72-port InfiniBand switch that will sit on top of the Sun Blade 6048 rack.

I just can’t stop being amazed.

Invite the world to GMail

The other day I noticed that I have this “Invite a friend” on my GMail. I didn’t use that thing for more than a year, so I took the chance of this re-discovery to get rid of it. Within several minutes I sent 98 invitations to an imaginary friend whose mail bounced at some noreply@somewhere… address. And it worked – I got rid of that little blue box that serves no purpose on my GMail page!

Imagine my surprise when today I find that box at the same place, with 50 new invitations to give away.

No, really – I don’t have any friends who don’t have a GMail account. I don’t want to use this “feature”, ever. And it does not contribute a thing to my GMail experience – get rid of it Google! Put some AdSense there or something.