a return to yield

I remember when, almost a decade ago, I was first discovering generators. It was a heady time, and I saw applications everywhere.

def fib_gen():
    x, y = 1, 1
    while x < 100:
       x, y = y, x + y
       yield x
    return

I also remember the first time I tried to mix a return value into my generator.

def fib_gen():
    x, y = 1, 1
    while x < 100:
       x, y = y, x + y
       yield x
    return True

Imagine my surprise, as I'm sure countless others experienced as well:

SyntaxError: 'return' with argument inside generator

A rare compile-time error! Only the decorative, bare return is allowed in generators, where they serve to raise StopIteration.

Now, imagine my surprise, so many years later when I import that same code in Python 3.


...

Nothing! No error. So what happened?

Turns out that the coroutine and asyncio machinery of Python 3 has repurposed this old impossibility.

If we manually iterate to skip over our yield:

fib_iter = fib_gen()                                                                                                                                                                                                    
for i in range(11):                                                                                                                 
    next(fib_iter)                                                                                                                  
next(fib_iter)

We see what's really happening with our return:

Traceback (most recent call last):
  File "fib_gen.py", line 13, in <module>
    next(fib_iter)
StopIteration: True

That's right, returns in generators now raise StopIteration with a single argument of the return value.

Most of the time you won't see this. StopIterations are automatically consumed and handled correctly by for loops, list comprehensions, and sequence constructors (like list). But it's yet another reason to be extra careful when writing your own generators, specific to Python 3.

List Comprehension Code Golf

Ah code golf, pastime of our navelgazing alter egos. Being designed for readability and maintainability, Python doesn't always show well in this sort of sport, but occasionally we get thrown a bone. For instance, for nonzero even numbers less than 10:

>>> [x for x in range(10) if x and not x % 2]
[2, 4, 6, 8]

is equivalent to

>>> [x for x in range(10) if x if not x % 2]
[2, 4, 6, 8]

A whole character saved! Yes, a close reading of PEP 202 will show that one of the canonical examples of list comprehensions uses this pattern for... some reason.

Either way, now you know. Sally forth and do what must be done with all code golf tricks: Never, Ever Use Them For Production Code.

-- Mahmoud
http://sedimental.org/
https://github.com/mahmoud
https://twitter.com/mhashemi

__dict__ and vars()

vars() is the lesser-known and arguably more Pythonic version of the __dict__ attribute (available on all classes that do not use slots). From the docs:

vars([object])

Without an argument, act like locals().

With a module, class or class instance object as argument (or anything else that has a __dict__ attribute), return that attribute.

As core Python contributor Raymond Hettinger points out on Twitter:

"Most folks prefer len(s) to s.__len__(), prefer next(it) to it.next(), but forget to use vars(s) rather than s.__dict__"

However, I think there's room to argue that s.__dict__ may be a little clearer than vars(s). After all, (almost) everything in Python is an object, and very nearly all objects are backed by dicts. Furthermore, s.__dict__ is faster:

>>> from timeit import timeit
>>> timeit('object.__dict__;object.__dict__')
0.2776460647583008
>>> timeit('vars(object);vars(object)')
1.276643991470337
>>>

Four million iterations later, we see that grabbing the __dict__ is over four and a half times faster. This should actually come as no surprise: accessing built-in functions (like len()/vars()/etc.) is always slower than object attribute access.