About

James Golick

James Golick's software experience ranges from artificial intelligence to web front-end and JavaScript development. Most recently, James has fallen back in love with web development thanks to Ruby on Rails.

Since discovering Rails, James has become a prolific contributor to its open source ecosystem. He is the author of several popular plug-ins and gems, and a contributor to countless others, including the framework itself.

James is an advocate for well-written, well-tested code and he blogs regularly about the practice of developing software. He speaks regularly at software development conferences and user groups. James is a partner in Nine Lives, Inc.

Latest Tweets

follow me on Twitter

James on the Web

Monkey-Patching, Single Responsibility Principle, and Scala Implicits

Feb 08 2010

When it's impossible to extend core classes, there's no choice but to write a whole bunch of classes with names like StringUtil to house your utility methods. Every namespace winds up having at least one StringUtil class, and it gets really ugly.

In ruby, it's possible to add methods to absolutely any class including String, Integer and other core classes. Rather than calling StringUtil.pluralize("monkey"), you call "monkey".pluralize. The technique is known as monkey-patching. Compared to utility classes, it's a hell of a lot more convenient, and it reads better.

But, monkey-patching isn't without flaw. When you add a method to String, you add it to everybody's String. You're polluting a global namespace.

A lot of rubyists will tell you that in practice, monkey-patching related namespace pollution isn't a problem, and they're mostly right. But, sometimes, they're wrong. The json gem doesn't work properly with activesupport, for example. That's a real problem that gives me real headaches on a daily basis.

In his CUSEC talk, and follow-up blog post, Reg Braithwaite raises the issue that monkey-patching Integer to add duration methods (like 1.hour or 1.day) is a violation of single responsibility principle. He goes on to justify monkey-patching somewhat:

A Ruby program with extensive metaprogramming is a meta-program that writes a target program. The target program may have classes that groan under the weight of monkey patches and cross-cutting concerns. But the meta-program might be divided up into small, clean entities that each have a single responsibility...In effect, the target program's classes and methods have many responsibilities, but they are assembled by the meta-program from smaller modules that each have a single responsibility.

I think this is a mostly adequate justification for violating SRP. But, needing such an in depth justification at all feels wrong. There must be a better way.


It turns out there is, in scala at least, in the form of something called implicits. Implicits are a way to tell the scala compiler how to convert from one type to another in the event that a method is needed from the second type. Implicits can be definted at (or imported in to) arbitrary scope, and their effects are entirely localized. In fact, they don't do anything at all unless they're needed.

So, let's say we wanted to add a k-combinator type method (like ruby's Object#tap) to all scala objects. First, we need a class that implements the tap method in a way that can be applied to any type.

class Tapper[A](tapMe: A) {
  def tap(f: (A) => Unit): A = {
    f(tapMe)
    tapMe
  }
}

This class has one method that taps the object supplied to the class's constructor. The details of its implementation are interesting, but unimportant here. This REPL session should explain everything:

scala> val tapper = new Tapper("hello!")
tapper: Tapper[java.lang.String] = Tapper@5e53bbfa

scala> tapper.tap { s => println(s) }
hello!
res0: java.lang.String = hello!

Now we need to get objects responding to #tap. We do that by defining an implicit.

object Tap {
  implicit def any2Tapper[A](toTap: A): Tapper[A] = new Tapper(toTap)
}

We wrap the implicit's definition in an object, which is scala for singleton. Those methods can then be imported in to arbitrary scope using the import statement. This REPL session should make everything clear:

scala> "hello!".tap { s => println(s) }
<console>:5: error: value tap is not a member of java.lang.String
       "hello!".tap { s => println(s) }
                ^

scala> import Tap._
import Tap._

scala> "hello!".tap { s => println(s) }
hello!
res2: java.lang.String = hello!

This Tapper certainly doesn't violate SRP. The addition of the #tap method to all Objects is localized and doesn't affect other running code. Libraries your code depends on can implement their own tap method without collision.

In the end, Reg's argument about meta-programming might still apply here. But, if it does, the kind of meta-programming introduced by implicits is limited in scope and prevents the kind of global namespace pollution that can bite you in the ass in ruby. And the resulting code doesn't have to violate SRP.

In practice, I've found scala's implicits a lot more pleasant to work with than ruby's monkey patching. They're more explicit which provides additional clarity without sacrificing much in the way of terseness. Like anything else, scala isn't a perfect language, but implicit type conversions are a really elegant solution to an ugly problem.


Friendly 0.5.0: Offline indexing and more

Jan 30 2010

There've been a few quiet releases of Friendly since I blogged about 0.4. Mostly, they were bug fixes, except for the addition of change tracking which is mostly an internal feature that will support arbitrary caches. You can see all the notable changes in the changelog.

This week, I released 0.5 which includes built-in support for building a new index in the background without interrupting your app. Here's how it works:

First, declare the new index in your model:

class User
  # ...snip...

  indexes :name, :created_at # this wasn't there before
end

Then, make sure to run Friendly.create_tables! to create the index table in the database. Don't worry, this won't overwrite any of your existing tables.

Friendly.create_tables!

Now that the the new table has been created, you need to copy the .rake file included with Friendly (lib/tasks/friendly.rake) to somewhere that will get picked up by your main Rakefile (lib/tasks if it's a rails project). Then, run:

KLASS=User FIELDS=name,created_at rake friendly:build_index

If it is a rails project, you'll need to prefix friendly:build_index with a rake task that loads your rails environment. For our app, the full command looks like this:

KLASS=User FIELDS=name,created_at rake environment friendly:build_index

If you're running this in production, you'll probably want to fire up GNU screen so that it'll keep running even if you lose your SSH connection. When the task completes, the index is populated and ready to go.

We've already built a couple of indexes with this code in production and it worked great!

Get It!

As always, install Friendly as a gem:

sudo gem install friendly

If you're not already following the github project, it's a great way to keep up with Friendly's development. Finally, if you feel so inclined, I'd appreciate a recommendation on Working with Rails.


Trend All the Fucking Time (TRAFT?)

Jan 10 2010

My new years resolution was to measure more. For a while now, I've wanted to get a better picture of our systems and our business, and hopefully, how they relate.

So, my first day back at work after the holidays, I started looking for the right tool to gather data with. After investigating some of the options, I wound up settling on munin.

I say settling because I was quite dissatisfied with the available options. I tried everything from collectd to reconnoiter and found all of the solutions horribly lacking in some way. This is an enormous market just waiting for a startup to revolutionize it.

In any event, we were already using munin to trend our system metrics. So, now it was just a matter of figuring out how to get our business metrics in there. Here's how we did it.

Custom Graphs

It's actually relatively easy to write a munin plugin. All you need is an executable that responds to a config command and emits a specially formatted value when it's called with no parameters.

Most of the examples I could find were implemented using multi-line strings, which seemed ugly to me. So, I wrote a little ruby DSL to make my plugins easier on the eyes.

Here's an example plugin written with munin_plugin. I won't go in to what all the parameters mean. The official documentation does a good enough job of that.

#!/usr/bin/env ruby

require 'rubygems' # or rip or whatever
require 'munin_plugin'

munin_plugin do
  graph_title  "Load average"
  graph_vlabel "load"
  load.label   "load"

  collect do
    load.value `cat /proc/loadavg`.split(" ")[1]
  end
end

Everything outside the collect block gets emitted as configuration. When the above script is called with config, it produces the following output:

graph_title Load average
graph_vlabel load
load.label load

When it's called without any parameters, it would produce something like the following:

load.value 0.03

As you can see, the DSL just emits whatever you give it, essentially verbatim. Nothing fancy, just a little syntactic sugar.

Let's trend some business metrics.

Trending Business Metrics

One of our most popular features is picture uploads. I wanted to get a sense of how quickly pictures were being uploaded at different times of day. Since munin polls nodes every 5 minutes, I wasn't sure exactly what kind of value it was going to need to get this going. Do I need to calculate the rate myself?

It turns out munin has an option called DERIVE, which turns your monotonically increasing value in to a per unit of time graph. So, I created a little REST API that returns the total number of pictures on the site. Then, all I had to do was scoop it up with a fairly simple munin plugin.

#!/usr/bin/env ruby

require 'rubygems'
require 'munin_plugin'
require 'open-uri'

munin_plugin do
  graph_title    "Picture Upload Rate"
  graph_vlabel   "Pictures / ${graph_period}"
  graph_category "FetLife"
  graph_period   "minute"
  pictures.type  "DERIVE"
  pictures.min   "0"
  pictures.label "pictures"

  collect do
    pictures.value open("http://an.internal.ip/stats?id=pictures").read
  end
end

Here's the result (actually for a different metric, but it uses roughly the same script):

We use a nearly identical plugin to chart the all the critical objects in our system. The graphs are starting to give us a nice look at exactly what happens during peak load, and as time goes on, hopefully they'll assist us in identifying problems, too.

The moral of the story is that seting up custom graphs is easy. You should do it.


The Problem with User Stories

Jan 04 2010

Back when I was running a consultancy, I was a big fan of user stories as a method for gathering requirements. Having left the consulting world, though, it's become really clear to me that stories are the wrong approach.

Why consultants like stories

With a freshly inked contract, the client typically wants to begin work immediately. If you're smart, you've sold them on the value of good design. But, they probably haven't engaged a designer just yet. So, starting the requirements gathering process by story carding is a great way to get the ball rolling.

In fact, an iteration or two worth of story cards are a really great thing to put in a proposal. They demonstrate to the client that you have listened to them during your initial meetings. A reiteration of the requirements proves that you're (at least basically) on the same page.

The problem

The thing is: applications are not collections of words. They engage us in completely different ways than reading story cards.

Words mean different things to different people. Even meticulously written user stories that follow a standard form leave plenty of room for interpretation. That's why, even with your carefully crafted stories, it still takes several tries to get it right with the designer.

The design process exposes all the holes in your requirements. You discover missing, irrelevant, or grossly mis-estimated stories. With something to look at, everybody sees the application far more clearly. In fact, this is the first time they actually see the application at all.

Now that the stakeholders have seen the designs, the inevitable rounds of changes begin. A good design process should yield important refinements to the product. Which means that by the end of it, the stories are even less relevant.

The solution

Start with UI.

User interface assets (xhtml / css — not photoshop documents) leave very little to the imagination. They are as close as you can get to a working application without actually implementing it.

A good designer (and supporting team) will gather requirements and incorporate feedback throughout the process. All the while, the project stakeholders will be able to actually point and click at the interface, even if it's not completely functional.

So, for application development, the designer belongs on the team rather than as an external resource. They should play a central role in bridging the gap between stakeholders' brains and the development team.

The reality is that designers play that role either way. It is during the design process that the requirements become truly understood. So, come to terms with that, and save everybody a lot of time and money.


Friendly 0.4: scopes, named_scopes, associations, and more

Dec 22 2009

Friendly has gotten a bunch of attention since we released it last week. We're already over 200 watchers on the github project. Thanks to everybody who wrote or tweeted about it.

Since then, I've been hacking away on features that were sorely missing. Today, I've got a handful of those to release.

Scopes

The star of Friendly 0.4.0 is scopes. Here's how they work.

Assuming the following model:

class Post
  attribute :author, String

  indexes   :author, :created_at
end

To create a scope on the fly, just call YourDocument.scope.

@scope = Post.scope(:author => "Stewie Griffin")

You'll get back an instance of Friendly::Scope. That object supports almost all of the query and creation API that a Document class does.

@scope.all == Post.all(:author => "Stewie Griffin")
@scope.first == Post.first(:author => "Stewie Griffin")

# these won't actually evaluate to true, but I mean to say that the syntaxes are identical
@scope.build == Post.new(:author => "Stewie Griffin")
@scope.create == Post.create(:author => "Stewie Griffin")

It's also possible to create reusable, named scopes.

Named Scopes

The named scope syntax is pretty well identical to ActiveRecord's:

class Post
  # ... snip ...
  named_scope :by_stewie, :name => "Stewie Griffin"

Then, you can get an instance of that scope with:

Post.by_stewie

A named scope instance is just another instance of Friendly::Scope. Unlike with ActiveRecord, you must call scope.all to get the target of the scope. You can also use any of the other scope methods:

Post.by_stewie.all == Post.all(:name => "Stewie Griffin")
  Post.by_stewie.create.name == "Stewie Griffin"
  # ... and so on

Named scopes are also chainable arbitrarily.

class Post
  # ... snip ...
  named_scope :recent, :order! => :created_at.desc
end

Post.by_stewie.recent == Post.all(:name => "Stewie Griffin", :order! => :created_at.desc)

If there are conflicts, the right-most scope takes precedence.

class Post
  # ... snip ...
  named_scope :by_peter, :name => "Peter Griffin"
end

Post.by_stewie.by_peter == Post.all(:name => "Peter Griffin")

Associations

Currently, only has_many associations are supported.

class User
  has_many :posts
end

class Post
  # ... snip ...
  attribute :user_id, Friendly::UUID

  indexes   :user_id, :created_at
end

The cool thing is that associations are just another instance of Friendly::Scope.

User.new.posts # => #<Friendly::Scope>

So, all of the Friendly::Scope methods are supported. You can also chain named_scopes on an association.

@user = User.create
  @user.posts.recent == Post.all(:user_id => @user.id, :order! => :created_at.desc)

Those are the basics. Check out the docs for Friendly::Scope and Friendly::Associations::Association for more info.

Misc Stuff

Jeff Rafter contributed a few awesome patches.

Most notably, Friendly was previously unable to create an index table for a field of type Friendly::UUID. Jeff added support for mapping custom types to database field types.

There are no docs yet, but the method signature for Friendly::Attribute.register_type should be clear enough. For most purposes, though, the big win is that Friendly should now be able to create tables for you more consistently.

Jeff also added a spec/config.yml for putting your spec db settings in.

Get it while it's hot!

  • Install Friendly:
    sudo gem install friendly
    
  • Watch the github project.
  • Join us in irc at #friendlyorm on freenode.