Loopy references

This may not come as a surprise to experienced Python programmers, but objects in Python can reference or "contain" themselves and/or their parent objects.

A trivial example, the self-containing list:

>>> a = []
>>> a.append(a)
>>> a
[[...]]

Kind of a cool repr, at least. How else could you flatly represent this infinitely nested object? What happens when you take two of these and smash them together with an equals comparator?

>>> b = []
>>> b.append(b)
>>> a == b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: maximum recursion depth exceeded in cmp

Like two mirrors pointed at one another, they compare back and forth forever, or at least for...

>>> sys.getrecursionlimit()
1000 

... loops.

collections.deque random access is O(n)

>>> r = range(int(1e6))
>>> li = list(r)
>>> de = collections.deque(r)
>>> pos = len(de) / 2
>>> timeit.timeit(lambda: de[100])
0.1355267132935296
>>> timeit.timeit(lambda: li[100])
0.14215966414053582

Great, list and deque performance are identical.

>>> timeit.timeit(lambda: li[pos])
0.1369839611421071
>>> timeit.timeit(lambda: de[pos])
56.227819584345696

Oops.  O(1) vs O(n).

necromancy, lesson one

There is a dark, mysterious, and dangerous power of Necromancy in the Python language; used only at great peril: the __del__ method.  Consider this class:

>>> class Living(object):
...    def __del__(self):
...       undead_hoard.append(self)
...
>>> undead_hoard = []

Let's use it.

>>> a = Living()
>>> undead_hoard
[]

So far so good.  What happens if we delete one?

>>> del a
>>> undead_hoard
[<__main__.Living object at 0x020C01D0>]

The object got resurrected by __del__.  No problem, it can be removed from the list:

>>> del undead_hoard[0]
>>> undead_hoard
[<__main__.Living object at 0x020C01D0>]

Hmm, maybe reset the list?

>>> undead_hoard = []
>>> undead_hoard
[<__main__.Living object at 0x020C01D0>]

Lesson one of Python Necromancy: __del__ may be called many times on the same object; a safe __del__ must be re-entrant.  In this case, the resurrection was the fault of the __del__ method itself, but that is not always the case.

Who weakrefs the weakrefs?

>>> import weakref
>>> class Target(object): pass
...
>>> t = Target()
>>> r = weakref.ref(t)
>>> r2 = weakref.ref(r)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create weak reference to 'weakref' object

decoding base64.decode

base64 is Python's module for providing various standard encoding functionalities, especially the eponymous base64. Little quirk about the decoding in Python 2.7, it ignores invalid characters!



>>> import base64
>>> test_str = base64.b64encode('123456789')
>>> base64.b64decode(test_str[:-4] + '....' + test_str[-4:])
'123456789'
>>> base64.b64decode(test_str[:-4] + '...' + test_str[-4:])
'123456789'
>>> base64.b64decode(test_str[:-4] + '.' + test_str[-4:])
'123456789'
>>> test_str[:-4]
'MTIzNDU2'
>>> base64.b64decode(test_str[:-4])
'123456'

Python 3 adds a flag, validate, but it's off by default. Oh well!

better translate()

translate is a simple, efficient function for doing fast byte-by-byte replacement on a string in Python.

However, Python Does What is always on the lookout for dangerous alternatives to simple things.  Instead of a function call, isn't it better to mutate global interpreter state so that the same result is achieved as a side-effect of a completely different operation?  (The answer is yes.)

>>> import ctypes
>>> def maketrans(inp, outp):
...    inp = list(inp)
...    outp = list(outp)
...    for aa, bb in zip(inp, outp):  # there's a fun reason single letter variable names are a bad idea here
...       ctypes.memset(ctypes.pythonapi.PyString_AsString(id(aa)), ord(bb), 1)
...
>>> maketrans("jkl", "xyz")
>>> "".join("jkl")
'xyz'

This method also has application to single character variable names that bears further exploration.

yade (yet another dict extension)

>>> from collections import defaultdict
>>> class TypeDict(dict):
...    def __missing__(self, key):
...       self[key] = defaultdict(key)
...       return self[key]
...
>>>
>>> td = TypeDict()
>>> td[float]['test']
0.0