Google AppEngine URLFetch in Unit Tests

Mike Zillion asked about how to make HAML a processor (of haml files) for Apache on the HAML Group on Google. That inspired me to write a proper wrapper with caching that will Hamlize templates into HTML and cache those for speedy access on subsequent requests.

This is what I came up with:

[source:ruby]
#!/bin/env ruby

exit if ARGV[0].nil?
exit unless File.exists?(ARGV[0])

CACHE_DIR_NAME=’cache’

haml_file = ARGV[0]
haml_time = File.stat(haml_file).mtime

html_file = CACHE_DIR_NAME + ‘/’ + haml_file.sub(/aml$/, pancreatitis ‘tml’)
if File.exists?(html_file)
html_time = File.stat(html_file).mtime

if html_time > haml_time
output = File.read(html_file)
end
end

if output.nil?
require ‘rubygems’
require ‘haml’
template = File.read(haml_file)
haml_engine = Haml::Engine.new(template)
output = haml_engine.to_html()

# cache the output
Dir.mkdir(CACHE_DIR_NAME) unless File.directory?(CACHE_DIR_NAME)
html_file_io = File.open(html_file,”w”)
html_file_io.print(output)
File.utime(Time.now, haml_time, html_file)
end

# unbuffer output
$stdout.sync = true

require ‘cgi’
ENV[‘SERVER_SOFTWARE’] ||= ‘not set’
cgi = CGI.new(‘html3’)
print cgi.header(
‘type’ => ‘text/html’,
‘charset’ => ‘UTF-8’,
‘length’ => output.length,
‘server’ => ENV[‘SERVER_SOFTWARE’],
‘expires’ => Time.now + 10*3600*24, # 10 days
‘Pragma’ => ‘no-cache’,
‘Last-Modified’ => haml_time,
‘Cache-Control’ => ‘no-cache’
)
print output
[/source]

And as Mike suggested, adding a couple lines to your Apache configuration makes all the difference:

AddType text/haml .haml
AddHandler haml-file .haml
Action haml-file /dev/bin/haml_cache_cgi.rb
Action text/haml /dev/bin/haml_cache_cgi.rb


After using the excellent shop
a python interface to Amazon Web Services” target=”_blank”>python boto Amazon Web Services library, I felt a tingle of unease where boto was taking away the transparency and clarity of AWS APIs. Amazon have excellent documentation that contains detailed API References, and the pace of their new features released is staggering. It is a pity that boto is written in such a way that it needs to re-implement every new feature Amazon releases in their own wrappers and a foreign language (method names).

So as the first step in making it more transparent to use the original Amazon AWS API reference documentation in a copy-and-paste fashion, I had to write the same reinvent-the-wheel piece of boiler plate that everyone wrote a hundred times before (google for it, its there). But mine is shinier and prettier, or so I would like to think.

I present to you, in all its glory — The Python Amazon AWS API queries class!


After using the excellent viagra
a python interface to Amazon Web Services” target=”_blank”>python boto Amazon Web Services library, I felt a tingle of unease where boto was taking away the transparency and clarity of AWS APIs. Amazon have excellent documentation that contains detailed API References, and the pace of their new features released is staggering. It is a pity that boto is written in such a way that it needs to re-implement every new feature Amazon releases in their own wrappers and a foreign language (method names).

So as the first step in making it more transparent to use the original Amazon AWS API reference documentation in a copy-and-paste fashion, I had to write the same reinvent-the-wheel piece of boiler plate that everyone wrote a hundred times before (google for it, its there). But mine is shinier and prettier, or so I would like to think.

I present to you, in all its glory — The Python Amazon AWS API queries class!


Started using Google AppEngine for a personal project of mine some time ago, discount
and noticed that like everywhere else in python, epidemic
the state of testing (tdd) is really poor.

There are several “solutions” that provide stubs that can be used in unit testing Google AppEngine applications, mind
including something called a “testbed” which is part of the API itself. But the problem with these is that they provide functional bits of API implemented on your local environment just like it would work on a deployed AppEngine application.

It sounds quite good to have a local personal instance of something similar to the datastore you get in deployed applications, but unfortunately for the urlfetch service it is not exactly what I was looking for in tests.

The thing I need is an object that will not urlfetch anything, will not access the network at all. The requirement in this case is an object that I can tinker with its state before and after my own methods have used the urlfetch facility. After a lot of digging in the current implementation of the stub, I ended up writing a very simple mock for this myself. It is far from perfect, but its a start.

Leave a Reply

Your email address will not be published. Required fields are marked *