Software Blag

In the Python programming language, metaclasses are used to specify how classes are created. Note that we are talking about class creation, not class instance creation. A few months ago I wrote some code which dynamically creates metaclasses. This turns out to be quite a useful trick.

Say you have a number of classes in a source file. To take an example from a project I am working on, each class opens and parses a particular file. Each class implements a common interface, and contains metadata which allows a manager to instantiate and “run” the class.

Of course, you want to run each class, and in the correct order. You might do something like this:

class Loader(object):
    [ ... ]

class RouteLoader(Loader):
    [ .... ]

class ShapeLoader(Loader):
    [ .... ]

loaders = [ RouteLoader, ShapeLoader ]

for loader in loaders:
    run(loader)

This sucks. Firstly, you are forced to manually maintain a list of loaders. Secondly, as you maintain the code, you might find that the order the loaders are defined in the file doesn’t match their order in loaders. I find the code much clearer if the order things run is the order they appear in the file; scrolling through the file shows you things in a logical order.

Instead, let’s define a function which creates a metaclass. This metaclass doesn’t do much, it just appends classes it constructs to a list:

def append_meta(lst):
    class AppendMeta(type):
        def __init__(cls, name, bases, dct):
            super(AppendMeta, cls).__init__(name, bases, dct)
            if hasattr(cls, "_ignore") and cls.__name__ in cls._ignore:
                return
            lst.append(cls)
    return AppendMeta

I’ll explain how this works in a second. Here’s how to use it:

loaders = []
mclass = append_meta(loaders)

class Loader(object):
    __metaclass__ = mclass
    _ignore = ('Loader', )
    [ .. ]

class RouteLoader(Loader):
    [ ... ]

class ShapeLoader(Loader):
    [ ... ]

for loader in loaders:
    run(loader)

What’s going on? append_meta is a function that takes a list reference as an argument, and returns a class. When used as a metaclass, it appends any class it creates to the provided list. The list reference is kept by the returned metaclass through normal closure behaviour. We have a little complexity to make sure that the Loader class itself doesn’t show up in the list, simply by providing a list of class names to ignore in class member _ignore. If someone can think of a more elegant way to do this, please let me know.

We no longer have to worry about maintaining the loaders list manually. As classes are created in the order they appear in the file, as we move them around in the file, the order they appear in the list is updated. Pretty neat.

I’m sure at this point you are wondering: why not just put all the loaders in a module, import it, and use dir(module)? Well, sometimes that doesn’t feel very nice; in any case, this is just an example. The approach of dynamically creating metaclasses to track and manipulate your classes is extremely useful. I have more complex examples that allow me to dynamically generate metaclasses which do several things, composing functionality. I’m using this to both track my classes and (using code pinched from Django’s ORM) also let certain class member variables know the name they have been given in the class. If there interest, I’ll blog about that - it is possible to come up with a library of useful operations performed by metaclasses, and then at runtime generate a metaclass to suit.

Anyway, that’s my first meaty post on this blog. Flame away!