Python packages & modules — how to loop through all subpackages

I have been playing around with python and I wanted to do the following things. Suppose we have a file test.py and a package foobar containing 2 modules foo.py and bar.py and 2 subpackages foo_mod and bar_mod. In these subpackages, there are some modules foo1.py, foo2.py… and bar1.py, bar2.py…
Now, I want to loop through each submodules (foo1.py, bar1.py…) without writing their names in test.py (or foo.py), so I can add/remove subpackages/submodules without changing every files

So, I have this hierarchy:

test.py
[foobar]:
          __init__.py
          foo.py
          bar.py
          [foo_mod]:
                    __init__.py
                    foo1.py
                    foo2.py
                    ...
          [bar_mod]:
                    __init__.py
                    bar1.py
                    bar2.py
                    ...

There is only 1 line in __init__py in foobar:
__all__ = ["bar", "foo"]
And in the subpackages, only:
__all__ = ["foo1", "foo2"]

In each submodules (foo1.py…), I have this code (but it should be different, otherwise, we can put it in __init__.py files):

def getInModule():
    return "I'm in "+__name__+" getInModule"

def getSub():
    return "I'm in "+__name__+" getSub"


Now, in foo.py:

import foo_mod
from random import choice
import sys

def getThings():
    fooList = []
    for i in foo_mod.__all__:
        stuff = [choice(range(10)) for _ in range(10)]
        fooList += [(stuff, "foo_mod." + i)]
    return fooList

def getSubmodule():
    fooList = []
    for i in foo_mod.__all__:
        __import__("foobar.foo_mod." + i)
        fooList += [sys.modules["foobar.foo_mod."+i].getSub()]
    return fooList


And finally in test.py:

import foobar
import sys

someList = []
for i in foobar.__all__:
    __import__("foobar." + i)
    someList += sys.modules["foobar."+i].getThings()
    print sys.modules["foobar."+i].getSubmodule()

for (l, m) in someList:
    __import__("foobar." + m)
    print sys.modules["foobar." + m].getInModule()


The output of python test.py is :


["I'm in foobar.bar_mod.bar1 getSub", "I'm in foobar.bar_mod.bar2 getSub"]
["I'm in foobar.foo_mod.foo1 getSub", "I'm in foobar.foo_mod.foo2 getSub"]
I'm in foobar.bar_mod.bar1 getInModule
I'm in foobar.bar_mod.bar2 getInModule
I'm in foobar.foo_mod.foo1 getInModule
I'm in foobar.foo_mod.foo2 getInModule


If I want to add a new foo3.py in foo_mod, I just have to add , "foo3" in foo_mod’s __init__.py and for a new subpackage baz, , "baz" in foobar’s __init__.py

Now, you can also create __all__ by listing the .py in the directory, but I prefer not to.

Oh, and if you want all foox.py and barx.py to have access to the same function fff(), you can write def fff()... in the __init__.py of foobar and do from .. import fff.
And for functions in the __init__.py of foo_mod, from . import fff.

If you know better way to do these things, please let me know ! It’s my first attempt and I’m sure I’m missing clever tips.

Advertisement

~ by loquehumaine on 2011, April 5 - Tuesday.