Autoloading in python

June 21, 2009 at 04:14 PM | categories: python, oldblog | View Comments

Before I started using python, I'd used perl for several years, and one thing which I'd liked about perl was its autoload facility. Now in python the closest equivalent that I've seen is __getattr__ for classes, but not __getattr__ for a module. This seemed like a real shame since there are times when autoload can be incredibly useful.
If it seems chaotic, consider the Unix PATH variable. Any time you type a name, the shell looks in lots of locations and runs the first one it finds. That's effectively the same sort of idea as autoloading. Yes, you can do some really nasty magic if you want, but then you can do that with the shell to, and generally people get along find.
Anyway, vaguely curious about it I decided to do some digging around, and came across this post by Leif K Brookes, which suggests this:
You could wrap it in an object, but that's a bit of a hack.

import sys

class Foo(object):
     def __init__(self, wrapped):
         self.wrapped = wrapped

     def __getattr__(self, name):
         try:
             return getattr(self.wrapped, name)
         except AttributeError:
             return 'default'

sys.modules[__name__] = Foo(sys.modules[__name__])

That looked reasonable, so I created a file mymod.py which looks like this:
import sys

def greet(greeting="Hello World"):
   print greeting

class mymod_proxy(object):
    def __init__(self):
        super(mymod_proxy, self).__init__()
        self.wrapped = sys.modules["mymod"]
    def __getattr__(self, name):
        try:
            return getattr(self.wrapped, name)
        except AttributeError:
            def f():
                greet(name)
            return f

sys.modules["mymod"] = mymod()
And tried using it like this:
~> python
Python 2.5.1 (r251:54863, Jan 10 2008, 18:01:57)
[GCC 4.2.1 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mymod
>>> mymod.hello()
hello
>>> from mymod import Hello_World
>>> Hello_World()
Hello_World
And as you can see, it seems to work as expected/desired.

Now the reason I'd been thinking about this, is because I'd like to retain the hierarchy of components in Kamaelia that we have at the moment (it's useful for navigating what's where), but given we tend to use them in a similar way to Unix pipelines it's natural to want to be able to do something like:
from Kamaelia import Pipeline, ConsoleReader, ConsoleWriter
Pipeline(
    ConsoleReader(),
    ConsoleWriter(),
).run()
Rather than the more verbose form specifically pulling them in from particular points. Likewise, we don't really want to import every single module in Kamaelia.py, because of the large number of components there (Kamaelia is really a toolbox IMO where things get wired together, and Axon is the tool for making new tools), the majority of which won't be used in ever application!

Now, I haven't done this yet, and wouldn't do it lightly, but the fact that you can actually make autoload functionality work, seems kinda cool, and and a nice opportunity. But I'm also now wondering just how nasty this approach seems to people. After all, Leif describes it as "a bit of a hack", and whilst it's neat, I'm not taking in the positive view. I'm interested in any views on better ways of doing Autoload in python, and also whether people view it as a nice thing at all. (One person's aesthetic is another person's pain after all...)
blog comments powered by Disqus