Creating/making directories in python (complex)

Question

I am trying to create a bunch of directories/subdirectories that I can copy files into. I am working with Python and I can't seem to find a good way to do this. I have a main path that I will branch off of. Then after that, I have Weights and No_Weights. Male and Female following. Within each of Male and Female folders, I have each ethnicity (Caucasian, African-American, Asian, Hispanic, Indo, Other, Unknown). Within each of those folders, I have age ranges from Below 20, all the way to 70+ (B20, 20, 30, 40, 50, 60, 70).

I have tried to generate all of the paths so all I would have to call is mkdir about 50 times, but that is about 150 lines of code (almost).

Is there any simple way to create all of these folders without having to do it by hand?

well I'm working on a Mac with Python 2.6...(yes I know I'm still using Python 2.6 just because there was a significant change between this version and 3) — Brandon
– Brandon, Commented Jul 1, 2010 at 15:20

Tim Pietzcker · Accepted Answer · 2010-07-01 16:02:22Z

18

import itertools
import os

dirs = [["Weights", "No_Weights"],
        ["Male", "Female"],
        ["Caucasian", "African-American", "Asian", "Hispanic", "Indo", "Other", "Unknown"], 
        ["B20", "20", "30", "40", "50", "60", "70"]]

for item in itertools.product(*dirs):
    os.makedirs(os.path.join(*item))

itertools.product() will construct all possible path variations, then os.path.join() will join the subpaths together using the correct syntax for your platform.

EDIT: os.makedirs() is needed instead of os.mkdir(). Only the former will construct all the intermediate subdirectories in a full path.

edited Jul 1, 2010 at 16:02

answered Jul 1, 2010 at 15:00

Tim Pietzcker

337k59 gold badges520 silver badges572 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Brandon Over a year ago

this was a great idea...I've never used itertools, but it works like a charm and of course it's fast! Thanks!

Gary Kerr · Accepted Answer · 2010-07-01 15:02:04Z

3

This example should get you started:

import itertools
import os.path

ROOT = r'C:\base\path'

sex = ('male', 'female')
ethnicity = ('Caucasian', 'African-American', 'Asian')
ages = ('B20', '20', '30')

for path in itertools.product(sex, ethnicity, ages):
    print os.path.join(ROOT, *path)

The itertools module is your friend: http://docs.python.org/library/itertools.html#itertools.product

answered Jul 1, 2010 at 15:02

Gary Kerr

14.6k4 gold badges52 silver badges52 bronze badges

Comments

Wayne Werner · Accepted Answer · 2010-07-01 15:02:20Z

2

Just do something like this:

main = 'somedir'
weight = ['weights', 'No_weights']
ethnicity = ['Caucasian', #the rest]
ages = ['B20'] +  range(20, 71, 10)

for w in weights:
    os.mkdir(os.path.join(main, w)
    for e in ethnicity:
        os.mkdir(os.path.join(main, w, e))
        for a in ages:
            os.mkdir(os.path.join(main, w, e, a))

and that should take care of it for you...

answered Jul 1, 2010 at 15:02

Wayne Werner

52.3k35 gold badges213 silver badges304 bronze badges

Comments

Donald Miner · Accepted Answer · 2010-07-01 14:55:42Z

Have a few nested for-loops, then os.mkdir for each one. Use os.path.join to concatenate the directory path together.

Something like:

loop weights
    mkdir weight
    loop sexes
        mkdir weights + sex
        loop ethnicities
            mkdir weights + sex + ethnicity
            loop ages
                mkdir weights + sex + ethnicity + age

here loop is just a normal for-loop:

for weight in ('weights', 'no_weights'):

mkdir is os.mkdir

'+' is os.path.join:

os.mkdir(os.path.join(weights, sex, ethnicity, age))

Edit: dir_util might be of some use here so you don't have to make each subdirectory:

http://docs.python.org/release/2.5.2/dist/module-distutils.dirutil.html

loop weights
    loop sexes
        loop ethnicities
            loop ages
                mkpath weights + sex + ethnicity + age

Alex Martelli · Accepted Answer · 2010-07-01 15:06:12Z

os.makedirs can help -- it makes all intermediate directories all the way down to the "leaf" one you specify.

The other issue (generating all the "one from column A, one from column B, ..." combinations) is best approached as a problem of "counting in mixed bases" -- roughly, s/thing like...:

choices = [ ['Weights', 'Noweights'],
            ['Male', 'Female'],
            ['Caucasian', 'AfricanAmerican', ...
            ...
          ]
Ls = [len(x) for x in choices]
ct = [0] * len(Ls)

while True:
    p = [c[i] for i, c in zip(ct, choices)]
    os.makedirs(os.path.join(p))
    n = -1
    while n > -len(Ls):
      ct[n] += 1
      if ct[n] < Ls[n]: break
      ct[n] = 0
      n -= 1
    else:
      break

itertools.product is the modern and concise approach to generating all the "one from column A, etc etc" picks, and what I would advise in production software -- just:

for p in itertools.product(*choices):
    os.makedirs(os.path.join(p))

can replace all of the above code (!). I think it's also worth being aware of the "counting in mixed bases" lower-abstraction-level approach because it comes in handy in many cases (including times in which one is stuck using a Python release < 2.6;-).

Collectives™ on Stack Overflow

Creating/making directories in python (complex)

5 Answers 5

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related