necromancy, lesson one

There is a dark, mysterious, and dangerous power of Necromancy in the Python language; used only at great peril: the __del__ method.  Consider this class:

>>> class Living(object):
...    def __del__(self):
...       undead_hoard.append(self)
...
>>> undead_hoard = []

Let's use it.

>>> a = Living()
>>> undead_hoard
[]

So far so good.  What happens if we delete one?

>>> del a
>>> undead_hoard
[<__main__.Living object at 0x020C01D0>]

The object got resurrected by __del__.  No problem, it can be removed from the list:

>>> del undead_hoard[0]
>>> undead_hoard
[<__main__.Living object at 0x020C01D0>]

Hmm, maybe reset the list?

>>> undead_hoard = []
>>> undead_hoard
[<__main__.Living object at 0x020C01D0>]

Lesson one of Python Necromancy: __del__ may be called many times on the same object; a safe __del__ must be re-entrant.  In this case, the resurrection was the fault of the __del__ method itself, but that is not always the case.

Who weakrefs the weakrefs?

>>> import weakref
>>> class Target(object): pass
...
>>> t = Target()
>>> r = weakref.ref(t)
>>> r2 = weakref.ref(r)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create weak reference to 'weakref' object

decoding base64.decode

base64 is Python's module for providing various standard encoding functionalities, especially the eponymous base64. Little quirk about the decoding in Python 2.7, it ignores invalid characters!



>>> import base64
>>> test_str = base64.b64encode('123456789')
>>> base64.b64decode(test_str[:-4] + '....' + test_str[-4:])
'123456789'
>>> base64.b64decode(test_str[:-4] + '...' + test_str[-4:])
'123456789'
>>> base64.b64decode(test_str[:-4] + '.' + test_str[-4:])
'123456789'
>>> test_str[:-4]
'MTIzNDU2'
>>> base64.b64decode(test_str[:-4])
'123456'

Python 3 adds a flag, validate, but it's off by default. Oh well!

better translate()

translate is a simple, efficient function for doing fast byte-by-byte replacement on a string in Python.

However, Python Does What is always on the lookout for dangerous alternatives to simple things.  Instead of a function call, isn't it better to mutate global interpreter state so that the same result is achieved as a side-effect of a completely different operation?  (The answer is yes.)

>>> import ctypes
>>> def maketrans(inp, outp):
...    inp = list(inp)
...    outp = list(outp)
...    for aa, bb in zip(inp, outp):  # there's a fun reason single letter variable names are a bad idea here
...       ctypes.memset(ctypes.pythonapi.PyString_AsString(id(aa)), ord(bb), 1)
...
>>> maketrans("jkl", "xyz")
>>> "".join("jkl")
'xyz'

This method also has application to single character variable names that bears further exploration.

yade (yet another dict extension)

>>> from collections import defaultdict
>>> class TypeDict(dict):
...    def __missing__(self, key):
...       self[key] = defaultdict(key)
...       return self[key]
...
>>>
>>> td = TypeDict()
>>> td[float]['test']
0.0

ctypes short string shenannigans

>>> >>> >>> 42105668 >>> 42105668 >>> 'f' >>> 'ab' >>> 'f'

import ctypes b_char = 'b' ctypes.pythonapi.PyString_AsString(id(b_char)) ctypes.memset(_, ord('f'), 1) b_char 'ab' 'ab'[1:]

Single character strings are interned in a special array in CPython.  This means mutation to the internals of a single character string will have global consequences.

(http://www.laurentluce.com/posts/python-string-objects-implementation/ has more information.)

assignment order

Variable assignment in a chained "=" goes left-to-right.

>>> a = [None]

>>> a[0][0] = a  # fails, as expected

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

TypeError: 'NoneType' object has no attribute '__getitem__'

>>> a[0] = a[0][0] = a  # succeeds

The second form is equivalent to:

>>> a[0] = a

>>> a[0][0] = a

With a bit of code we can explicitly see that the assignments are happening in left-to-right order:

>>> class Squawker(object):

...    def __init__(self, name): self.name = name

...    def __setitem__(self, key, val): print self.name

...

>>> Squawker("a")[0] = Squawker("b")[0] = 1

a

b