Making python imports more structured?

Question

The code works but looks messy so this might be a code review question where I didn't study enough of pythons conventions to know how to structure and organize the beginning of my file more pythonic. I basically just pasted in imports so they could be duplicates, not needed anymore or wrongly ordered. Can you advice anything how to structure my imports or can I leave code like this to focus on my own functions?

File 1:

from __future__ import with_statement
import logging
import os
from google.appengine.api.users import is_current_user_admin, UserNotFoundError
import time
import cgi
import geo.geotypes
import main
import captcha
from google.appengine import api
from google.appengine.runtime import DeadlineExceededError
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.ext.blobstore import BlobInfo
from google.appengine.ext.db import djangoforms
from django import forms
from django.core.exceptions import ValidationError
from django.utils import translation
from datetime import datetime, timedelta
os.environ['DJANGO_SETTINGS_MODULE'] = 'conf.settings'
from django.conf import settings
from django.template import RequestContext
from util import I18NHandler
import util
from google.appengine.api import urlfetch, taskqueue
from django.template.defaultfilters import register
from django.utils import simplejson as json
from functools import wraps
from google.appengine.api import urlfetch, taskqueue, users, images
from google.appengine.ext import db, webapp, search, blobstore
from google.appengine.ext.webapp import util, template
from google.appengine.runtime import DeadlineExceededError
from random import randrange
import Cookie
import base64
import cgi
import conf
import datetime
import hashlib
import hmac
import logging
import time
import traceback
import urllib
import twitter_oauth_handler
from twitter_oauth_handler import OAuthClient
from geo.geomodel import GeoModel
from django.utils.translation import gettext_lazy as _
webapp.template.register_template_library('common.templatefilters')

File 2 (there's are several instructions here I don't understand):

from __future__ import with_statement
                # -*- coding: utf-8 -*-
import facebookconf
import os, wsgiref.handlers
os.environ[u'DJANGO_SETTINGS_MODULE'] = u'conf'
import util
import time
import logging
import urllib
import wsgiref.handlers
import appengine_admin
import cgi
import captcha
import re
import hashlib
import string
import hmac
import twitter_oauth_handler
from twitter_oauth_handler import OAuthClient
os.environ['DJANGO_SETTINGS_MODULE'] = 'conf.settings'
from geo.geomodel import GeoModel
from google.appengine.dist import use_library
from google.appengine.ext import blobstore, webapp, db, search
# template import must be run before other Django modules imports
from google.appengine.ext.webapp import blobstore_handlers, util, template
from google.appengine.ext.blobstore import BlobInfo
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.api import files, images, mail, memcache, users
from django.conf import settings
# Force Django reload
settings._target = None
from util import I18NHandler, FacebookBaseHandler
from google.appengine.ext.db import djangoforms
from django.utils import translation
from django.utils import simplejson as json
from django.contrib.formtools.preview import FormPreview
from random import choice
from urllib import quote
from google.appengine.api.users import is_current_user_admin, UserNotFoundError
from google.appengine.api import urlfetch
import random
import datetime
from datetime import timedelta
from django.utils.translation import gettext_lazy as _
from django.template import defaultfilters

How do I know when an import is no longer used since the function was moved or removed? Why can't I specify the same import for multiple files at one place and I must spec the same import in both files? I can imagine moveing handling imports to a separate file i.e. imports.yaml to specify imports for all python files in that directory or likewise.

How do I know when an import is no longer used since the function was moved or removed? use pylint (pypi.python.org/pypi/pylint/0.24.0) — mouad
– mouad, Commented Sep 10, 2011 at 20:17
To follow up on what delnan said. If most of these imports are actually used within the file, then it's a sign that your code could use some significant restructuring. It will be very difficult to maintain any code that is actually using all of those imports. — Alec Munro
– Alec Munro, Commented Sep 10, 2011 at 20:20
I could lint the code whic is about 2300 lines with 2000 lines in one file and 300 lines in one file where much is web request handlers. And these imports are already the "restructured" version. — Niklas Rosencrantz
– Niklas Rosencrantz, Commented Sep 10, 2011 at 20:29

Vukašin Manojlović · Accepted Answer · 2017-02-26 17:35:04Z

27

PEP 8 - Style Guide for Python code recomends to order your imports in following order:

1. Standard library imports
2. - blank line -
3. google sdk imports
4. - blank line -
5. django imports
6. - blank line -
7. your own code imports

Import only things you use in code. Remove unused imports. You can use one of these tools to detect unused imports: Pydev on Eclipse / pyflakes / pylint

You have quite a lot imports. How big is your actual code? It might be a good idea to split it into few modules.

Why can't you import single time in single file? Well you actually could do it like this:

WARNING: THIS EXAMPLE ILLUSTRATES BAD CODING PRACTICES

import_all.py:

    import a
    import b
    import c

other.py:

     from import_all import *

But please don't do that. It is against all good practices of Python development and against The Zen of Python:

Explicit is better than implicit.

...

Namespaces are one honking great idea -- let's do more of those!

I also recommend you to read the Python documentation on modules and something about Python namespaces.

edited Feb 26, 2017 at 17:35

Vukašin Manojlović

3,8573 gold badges23 silver badges33 bronze badges

answered Sep 10, 2011 at 20:18

Ski

14.5k4 gold badges58 silver badges67 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Alec Munro Over a year ago

I read PEP8 as suggesting Google SDK and Django imports belonging together, in the "related third party" section. Not to say it wouldn't be cleaner this way. :)

Niklas Rosencrantz Over a year ago

I'm actually refactoring to use 3 files: main.py i18n.py and util.py moving code that both main.py and i18n.py needs to util.py. The files main.py and i18n.py are about 2300 lines together. I'm glad it works but it sure looks unstructured and I don't even understand all my imports.

Marius Mucenicu Over a year ago

there's no blank line between google and django imports as they are both 3rd party libraries, and it would only cause more confusion to somebody who sticks to the conventions.. standard library -> blank line -> 3rd party -> blank line -> application specific (anything imported from the same repository)

Colin · Accepted Answer · 2011-09-10 20:52:47Z

12

Once you've used pylint to identify duplicate and unused imports, and organized them according to PEP8 as the other answers suggest, you can further clean it up by changing the way you import packages.

Instead of

from google.appengine.api import urlfetch, taskqueue, users, images

you could just do

from google.appengine import api

then you would need to put "api.urlfetch", "api.taskqueue", etc. wherever you use those.

This isn't the "right" way to do it, it's just another way. You'll have to choose which one you prefer.

Also note that you can use aliases:

from google.appengine import api as gaeapi

now you would put "gaeapi.urlfetch". This is useful if you need to import modules called "api" from multiple packages.

Also, to answer your question "Why can't I specify the same import for multiple files at one place and I must spec the same import in both files?", if you're importing the same packages in multiple files, that could indicate those files are closely related, and should be merged into a single file. Unlike C++ or Java, where every class is its own file, the pythonic way is to make each module (file) as self-contained as possible, which usually means they contain multiple classes and functions.

answered Sep 10, 2011 at 20:52

Colin

10.9k11 gold badges51 silver badges62 bronze badges

1 Comment

Niklas Rosencrantz Over a year ago

This is 3 answers in one so I learnt a lot. I started with one file main.py and now I have 'main.py' with models, `i18n.py' with request handlers and 'util.py' with base classes and common functions. The actual code I'll restructure you may find from this link pastebin.com/EPRfCTpS

Muriel · Accepted Answer · 2019-03-18 19:11:34Z

9

Adding to the answers above, what we like to do in addition to the PEP8, is organize things alphabetically (but import comes before from) like this:

# 1. standard libraries alphabetical with 'imports' before 'from' like this:
import csv
import logging
from collections import defaultdict
from datetime import date
#(blank line)

# 2. third party packages alphabetical with imports first
# next is the third party packages (also alphabetical as described above)
import numpy as  np 
import pandas as pd
from statsmodels import api as sm
#(blank line again)

# 3. your own stuff (also alphabetical) 
from my_other_folder import my_other_file

edited Mar 18, 2019 at 19:11

answered Mar 18, 2019 at 18:55

Muriel

6138 silver badges25 bronze badges

4 Comments

Niklas Rosencrantz Over a year ago

I am actually still working on this in new projects... + 1 for "import before from" but if I have from foo import bar and from baz import bletch which one should be first? Is it the import or the from which determines the order of those the imports (other things equal)?

Muriel Over a year ago

We do the alphabetization on the "from" part. So from baz import... would be before from foo import.... I can imagine it's a matter of preference though and there's probably no real 'right' answer.

Simplecode Over a year ago

Any official documentation for this ?

Tom Apr 4 at 12:53

While this is old, ordering by the from part makes more sense. This way people unfamiliar with the code can look for the package in the alphabetical order. They would not know which function from package you are using.

Alec Munro · Accepted Answer · 2011-09-10 20:19:14Z

5

PEP8 has a section on Imports (that I can't link directly).

Basically, for organizing, here's what you want to do:

Imports should be grouped in the following order:

1. standard library imports
2. related third party imports
3. local application/library specific imports

You should put a blank line between each group of imports.

Oh, I believe PyDev for Eclipse has an "organize imports" command.

answered Sep 10, 2011 at 20:19

Alec Munro

2231 silver badge6 bronze badges

Comments

Lawrence D'Oliveiro · Accepted Answer · 2011-09-11 10:31:31Z

4

I wouldn’t worry about it. Thnk of the imports as being there not to be read as such, but to provide a cross-reference: when someone comes across an unqualified identifier further down, and wonders where it came from, they can just search backwards for the first occurrence, and if it’s not defined locally in the file, they should hit a mention in an import statement which will tell them where it came from.

Conversely, you can check which imports are unused by searching forwards from the import to see if there are any mentions of the imported identifier; if not, then it should be safe to remove it.

Note this doesn’t work with wildcard imports. Don’t use wildcard imports.

answered Sep 11, 2011 at 10:31

Lawrence D'Oliveiro

2,8141 gold badge17 silver badges13 bronze badges

1 Comment

Niklas Rosencrantz Over a year ago

Good. I could also use a design pattern to organize which file does what and then it will become more clear and less duplicates.

Ethan Furman · Accepted Answer · 2011-09-11 08:52:13Z

3

Why can't I specify the same import for multiple files at one place and I must spec the same import in both files?

To answer only this question: you can.

all_imports.py
--------------
import google.appengine as gae
import geo
import catptcha

other.py
--------
from all_imports import *

Using `from ... import *' is bad practice when handled poorly (for example, importing five modules this way) as you end up with lots of names in your module namespace and it becomes difficult to track down where a variable/function/class came from.

When handled properly (such as in this example ;) it can be a great tool (where did gae come from? Oh, it must be in all_imports... I'll check there...).

answered Sep 11, 2011 at 8:52

Ethan Furman

70.1k21 gold badges174 silver badges251 bronze badges

Collectives™ on Stack Overflow

Making python imports more structured?

6 Answers 6

3 Comments

1 Comment

4 Comments

Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

3 Comments

1 Comment

4 Comments

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related