Closures (Ruby v Python)

I'm a Ruby guy. I'm not religious about it, but when it comes to scripting, prototyping, parsing, or automation my instinct (read: comfort zone) is Ruby.

I've been working on a project lately where the dominant language is Python. This has required me to both learn a new language and break habits built in other environments. I like learning new things and I think that having your comfort zone invaded every once in a while keeps you on your toes so this experience hasn't been too painful so far. It's interesting, though, to evaluate what I consider habits against what I think are more along the lines of principles.

One thing that prompted this was the way Ruby and Python differ in handling closures. I could claim that this started back when I was a TA and had to grade dozens of projects written in Python over many disparate editors (tabs vs. spaces, anyone?) but I digress. In this case, I was using what was basically a callback and I really like they way Python handles this syntactically. Consider the following:

def foo():
    print("in foo")

fun = foo   # fun is now a function object
fun()       # this invokes foo()

Which is not possible (at least with a similar syntax) in Ruby. In Ruby what you get is

def foo
    puts "in foo"
end

fun = foo   # invokes foo because foo takes no arguments
            # would be an error if foo expected arguments

And since foo gets the return value of puts (nil) if you try to use it later in your code you get an error. You'd either have to use a lambda or Proc to get around the fact that the method is invoked. In Ruby, you can elide the parentheses when calling a method which makes the above behavior perfectly rational.

Advantage Python. Let's try and use closures to do something...

There is a cool feature in Ruby that I've found useful in the past in that I can create a function generator similar to the following:

def foo_gen
    x = 0
    return lambda { x += 1 }
end

Where the returned function would represent the list of integers starting at 1 and increasing each time the function is invoked. Something like:

foo = foo_gen
3.times { puts foo.call }

Which would print out the numbers 1, 2, 3. In fact, each new call to foo_gen creates a new infinite sequence starting at 1. In Python, this turned out to not be so easy. There is a subtle difference in how the lexical scope is represented when defining a Python closure. Consider what I thought was an equivalent Python construct:

def foo_gen():
    x = 0
    def foo():
        x += 1  # Error: x not in scope here
        return x
    return foo

Unfortunately, the variable local to the scope that the closure was defined in is not visible from the closure itself. This subtlety is actually related to why a Python class method definition requires an explicit this argument when it is defined and Ruby class methods do not. In either case, the workaround to this problem is something similar to:

def foo_gen():
    x = 42
    def foo(x=x):
        while 1:
            x += 1
            yield x
    return foo

Two things are important in the above change:

• I am required to provide an argument to the function that receives the value of the locally (to the defining scope) bound variable. (The alternative is to define x as an array and increment the first index of that array - I don't fully understand why that is valid, however)
• I am now using a generator and yielding the values explicitly requiring an infinite loop to enable the generator

I should note that, in Ruby, the yield still happens it's just hidden behind the syntax and without the need for an explicit loop construct.

I'm a fan of list comprehensions and functionality such as enumerate. Other features of Python are certainly growing on me quickly (though, ' '.join(lst) is still counter intuitive compared to lst.join ' '). And, while I can accept the white space requirement, the necessary parenthesis for function calls, explicit self argument in class methods and some of Python's other assembly as uncomfortable, I find this to be more or less broken.

I'm not intending to bash Python or praise Ruby. I've got miles to go before I am a Python master (or Ruby master, for that matter) and I am entirely open to the fact that in my journey I may come to understand this behavior. Until that point, however, I still feel icky writing code like that.