James on Software

Jonathan Barket just sent me a patch that gets resource_controller's scaffold_resource generator to spit out RSpec specs! I quickly merged it in to master, and wanted to announce it to the world, since it's something I've been hoping to get added to the plugin for a while.

r_c's scaffold_resource generator now supports vanilla test/unit, Shoulda, and RSpec for tests, and erb, and haml for templating. It will sense which of those plugins or gems you have installed in your app, and generate accordingly — it's absolutely 0 configuration.

So, if you're an RSpec user, grab the latest master from github, and get generating! :)

Recently, I realized that something was wrong with all of the conventional testing and specing libraries.

One of the things I loved about RSpec, when I read thin's specs, for the first time, was how easy it was to read the assertion code:

some_var.should == 5

But, when I tried using RSpec on a project, it continually frustrated me that I had to describe each test twice (by naming it, and writing pseudo-english code):

it "should augment the count by one" do
  some_var += 1
  some_var.should == 5
end

So, I went back to the familiar Shoulda. But, then, a couple of weeks ago, I came to a realization: Shoulda has exactly the same problem — it's just hidden under the awesome set of macros. Hell, even test/unit has this problem. Test names are comments. And, frankly, many, if not most of the tests that I write (like the example above) just don't need commenting — they're simple enough that a comment is unnecessary verbosity. A comment may even be distracting the reader from actual test code. Then, a couple of days later, Jay Fields blogged about exactly that. That's what made me take a look at his testing framework: expectations.

Naming Tests with Code

When I was working with RSpec, I started to wish that I could do something like this, to avoid the naming penalty:

before do
  some_var += 1
end

some_var.should == 5
some_var.should be_an_instance_of?(Fixnum)

Shoulda can do something like that, but it doesn't work in all situations, because it gets called at the class scope, so you have to go guessing about what the name of the instance variable will be. Besides, since we're sticking to one assertion per test, couldn't the assertion be the name of the test?

some_var.should == 5 do
  some_var += 1
end

Oops, we just derived the syntax for expectations.

Assertions are King

Except that expectations takes it a few steps further. It provides a ton of niceties for describing your tests in sensible language, and great mocking facilities. Our last example would look like this:

expect 5 do
  some_var  = 4
  some_var += 1
end

If we wanted to test for Fixnum, like in our second assertion, above, there's a nice shorthand for that.

expect Fixnum do
  some_var  = 4
  some_var += 1
end

As I write this, I can hear somebody out there screaming: "That's not very DRY!!". Even though I prefer the duplication in this particular example for readability reasons, let's DRY it up, just for fun.

setup = lambda do 
  some_var  = 4
  some_var += 1
end

expect(Fixnum, &setup)
expect(5, &setup)

Looks a lot like what I was wishing for with RSpec, doesn't it?

Next, let's look at a mocking example. One of the things that makes mocking a little bit weird with conventional testing and specing frameworks is that assertions come last, but mocks come first. So, when you're trying to follow the one assertion per test pattern, you end up with two different flows: setup, assert, for assertions, and: mock, setup, with mocks. Since assertions always come first with expectations, there's only one possible flow, making tests more readable.

expect mock.to.receive(:some_method).with(1) do |my_mock|
  my_mock.some_method(1)
end

Finally, spec junkies can even write expectations using BDD-style language.

expect klass.new.to.have.finished do |process|
  process.finished = true
end

Caveats

Expectations is pretty new, so I haven't yet seen any niceties for testing rails. It would be great to be able to do something like:

expect controller.to_render('index') do |c|
  get :index
end

As a big fan of Shoulda, I'd love to see some of the same types of macros for expectations, too (not in the core framework, but as an add-on).

Get It

$ sudo gem install expectations

Check out the RDoc for more examples, and a full set of documentation.

While I was figuring out how to embed a webserver in to my new open source project, I had an epiphany: REST is the real deal, and thin, rack, and invisible are the platform of the future. It turns out that most daemon processes require a lot of code just to implement a protocol, daemonize, and manage things like PID files. From what I can tell, most of the code in many of these utilities is there to do things that aren't their primary responsibility.

Take Starling, Twitter's queueing agent, for example. A quick sloccount of starling's lib directory shows that there are 737 lines of code in that gem. Further investigation shows that the service (that is, everything but the client) is comprised of 5 files.

1. server.rb is responsible for things like starting the server, processing requests at the socket level, and managing Starling's thread pool. server.rb contains 146 LoC.
2. runner.rb is responsible for parsing command line options, daemonizing, switching to a specified user and/or group, forking, etc. runner.rb contains 199 LoC.
3. handler.rb implements the memcache protocol. It contains 143 LoC.
4. persistent_queue.rb actually implements — well — the persistent queue. It accepts new queue items, and stores them on disk for later retrieval. persistent_queue.rb contains 103 LoC.
5. queue_collection.rb is a proxy to a collection of PersistentQueue instances. I took that last sentence from their documentation, because I'm actually not 100% clear on what this file does. At any rate, queue_collection.rb contains 79 LoC.

As we can see, most (66%) of Starling is completely unrelated to queueing. Sure, implementing the memcache protocol was a good idea, since it's well supported, but at what cost? The author of Starling was forced to write quite a bit of new code.

All new code is immature. There are always bugs to fix, and issues that arise. But, this kind of new code has got a much bigger problem: most of it is unrelated to Starling's domain, and it isn't generic enough to be useful for anybody but them. So, Starling's daemonizing code, and memcache implementation are only being looked after by Starling's developers.

While Starling is fairly popular, it's a pretty specific tool. So, the base of contributors, bug finders, and bug fixers is far smaller than for, say, a web server.

Thin & Rack as a Platform

A web server needs to do all of the same ugly stuff as Starling. It needs to know how to daemonize, and speak a protocol (REST!). The difference? That stuff is a web server's primary responsibility.

Thin gives you event-driven listening, daemonizing, and stability — all for the price of:

Thin::Server.new '0.0.0.0', 5432 do  
  map('/') { run Invisible::Application.new }
end

Thin's mailing list has over 250 subscribers, and nearly 100 people watch it on github. Plenty of people are running their websites on thin (and, those are just the ones we know about). And, while most of the work is still being done by MA, there have been plenty of contributions from other people. And, given that a web server is a much more widely needed application, thin has a much larger potential user base than Starling.

Piggybacking on thin's technology is a good idea, and it's incredibly easy to do. Thin has built-in support for rack, a generic layer for connecting a web server to an application. It does all the stuff that every web framework used to have to do. Rack makes writing a framework insanely easy.

Invisible: A Web Framework in 40 sloc

The last time I saw MA speak, he coded a version of invisible live, during an hour long talk. Earlier today, I forked it, and made some changes to tailor it toward writing web services that are consumable by ActiveResource. It is an MVC framework, but my fork has no support for templating, since most responses will be of the to_xml variety.

The controllers look a lot like rails controllers. Like in merb, the return value of an action becomes the response body. However, things like changing the status code are decidedly more bare bones than in most frameworks. Also, there is no user-specified routing at all: all routes map to /controller/:id, with the action being dependent on the REST method.

These two hashes determine which action gets called (i.e. exactly how ActiveResource expects them):

MEMBER_ACTIONS     = { 'PUT' => :update, 'DELETE' => :destroy, 'GET' => :show }
COLLECTION_ACTIONS = { 'POST' => :create, 'GET' => :index }

If you follow invisible's conventions (and, it's hard not to), your service should be consumable by active_resource, out of the box. That means you get a free client!

Since we all know that a framework is nothing without a sample app...

Scrawny: A Lightweight, RESTful Persistent Queue on Top of thin, rack and invisible in < 60 sloc

Before we get started, I want to make something clear from the get go: THIS IS A PROOF OF CONCEPT — I KNOW IT ISN'T FINISHED — DON'T USE IT!

It's unfinished, but scrawny displays the amazing power of thin, rack, invisible, and active_resource. With an incredibly small amount of code, I was able to implement a stable, lightweight, persistent message queue, and its client. The code for doing all of the ugly stuff I talked about earlier in this article is a mere ~10 lines. I was able to focus on actually creating the queue. The other stuff was an aside, as it should be.