Reading a file using a relative path in a Python project

Question

Say I have a Python project that is structured as follows:

project
    /data
        test.csv
    /package
        __init__.py
        module.py
    main.py

__init__.py:

from .module import test

module.py:

import csv

with open("..data/test.csv") as f:
    test = [line for line in csv.reader(f)]

main.py:

import package

print(package.test)

When I run main.py I get the following error:

 C:\Users\Patrick\Desktop\project>python main.py
Traceback (most recent call last):
  File "main.py", line 1, in <module>
    import package
  File "C:\Users\Patrick\Desktop\project\package\__init__.py", line 1, in <module>
    from .module import test
  File "C:\Users\Patrick\Desktop\project\package\module.py", line 3, in <module>
    with open("../data/test.csv") as f:
FileNotFoundError: [Errno 2] No such file or directory: '../data/test.csv'

However, if I run module.py from the package directory, I don’t get any errors. So it seems that the relative path used in open(...) is only relative to where the originating file is being run from (i.e __name__ == "__main__")? How can deal with this, using relative paths only?

As a sidenote, quoting from PEP8: “Relative imports for intra-package imports are highly discouraged. Always use the absolute package path for all imports.” Here, from package.module import test. — spectras
– spectras, Commented Nov 4, 2016 at 6:05
This should be a duplicate of How do I get the path and name of the python file that is currently executing?, as well as (as I originally closed it) open() gives FileNotFoundError / IOError: '[Errno 2] No such file or directory'. The issue is the same, and it's extremely common. — Karl Knechtel
– Karl Knechtel, Commented Sep 11, 2023 at 17:12

Abdul Aziz Barkat · Accepted Answer · 2023-10-26 08:03:54Z

243

Relative paths are relative to current working directory. If you do not want your path to be relative, it must be absolute.

But there is an often used trick to build an absolute path from current script: use its __file__ special attribute:

from pathlib import Path

path = Path(__file__).parent / "../data/test.csv"
with path.open() as f:
    test = list(csv.reader(f))

This requires python 3.4+ (for the pathlib module).

If you still need to support older versions, you can get the same result with:

import csv
import os.path

my_path = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(my_path, "../data/test.csv")
with open(path) as f:
    test = list(csv.reader(f))

[2020 edit: python3.4+ should now be the norm, so I moved the pathlib version inspired by jpyams' comment first]

edited Oct 26, 2023 at 8:03

Abdul Aziz Barkat

22.3k9 gold badges104 silver badges94 bronze badges

answered Nov 4, 2016 at 5:54

spectras

13.6k2 gold badges34 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

jpyams Over a year ago

Of course, if you're using Python 3.4+, you can just use pathlib.Path: Path(__file__).parent.resolve()

aksh1618 · Accepted Answer · 2019-03-07 19:01:55Z

52

For Python 3.4+:

import csv
from pathlib import Path

base_path = Path(__file__).parent
file_path = (base_path / "../data/test.csv").resolve()

with open(file_path) as f:
    test = [line for line in csv.reader(f)]

answered Mar 7, 2019 at 19:01

aksh1618

2,59927 silver badges46 bronze badges

Comments

programmer · Accepted Answer · 2020-08-20 06:45:51Z

5

This worked for me.

with open('data/test.csv') as f:

answered Aug 20, 2020 at 6:45

programmer

2594 silver badges15 bronze badges

2 Comments

joanis Over a year ago

This probably only works when you're running the code from the parent directory of data/, but not in general, so it's not likely to reliable solution.

joanis Over a year ago

In fact, it's not an answer to the question at all, since OP specifically asked about a solution that would work when run from anywhere rather than just from a specific folder.

Peter Mortensen · Accepted Answer · 2023-02-12 05:30:29Z

4

My Python version is Python 3.5.2 and the solution proposed in the accepted answer didn't work for me. I've still were given an error

FileNotFoundError: [Errno 2] No such file or directory

when I was running my_script.py from the terminal. Although it worked fine when I run it through Run/Debug Configurations from the PyCharm IDE (PyCharm 2018.3.2 (Community Edition)).

Solution:

instead of using:

my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path

as suggested in the accepted answer, I used:

my_path = os.path.abspath(os.path.dirname(os.path.abspath(__file__))) + some_rel_dir_path

Explanation:

Changing os.path.dirname(__file__) to os.path.dirname(os.path.abspath(__file__)) solves the following problem:

When we run our script like that: python3 my_script.py the __file__ variable has a just a string value of "my_script.py" without path leading to that particular script. That is why method dirname(__file__) returns an empty string "". That is also the reason why my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path is actually the same thing as my_path = some_rel_dir_path. Consequently FileNotFoundError: [Errno 2] No such file or directory is given when trying to use open method because there is no directory like "some_rel_dir_path".

Running script from PyCharm IDE Running/Debug Configurations worked because it runs a command python3 /full/path/to/my_script.py (where "/full/path/to" is specified by us in "Working directory" variable in Run/Debug Configurations) instead of justpython3 my_script.py like it is done when we run it from the terminal.

edited Feb 12, 2023 at 5:30

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Oct 6, 2019 at 13:22

kaspiotr

7061 gold badge10 silver badges12 bronze badges

1 Comment

spectras Over a year ago

It is indeed a caveat. It's so uncommon to run script explicitly that you seem to be the first to notice (usually you would put #!/usr/bin/env python at the top, mark it executable and run it as ./myscript.py). The pathlib.Path version does not have this gotcha though as is probably a better option since python3.4.

Peter Mortensen · Accepted Answer · 2023-02-12 05:31:03Z

0

Try

with open(f"{os.path.dirname(sys.argv[0])}/data/test.csv", newline='') as f:

edited Feb 12, 2023 at 5:31

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Aug 17, 2020 at 20:10

Axel Schneider

772 bronze badges

Comments

Ahmad Ismail · Accepted Answer · 2023-02-12 07:52:23Z

0

I was surprised when the following code worked.

import os

for file in os.listdir("../FutureBookList"):
    if file.endswith(".adoc"):
        filename, file_extension = os.path.splitext(file)
        print(filename)
        print(file_extension)
        continue
    else:
        continue

So, I checked the documentation and it says:

Changed in version 3.6: Accepts a path-like object.

path-like object:

An object representing a file system path. A path-like object is either a str or...

I did a little more digging and the following also works:

with open("../FutureBookList/file.txt") as file:
   data = file.read()

edited Feb 12, 2023 at 7:52

answered Oct 1, 2020 at 20:23

Ahmad Ismail

14.5k7 gold badges74 silver badges122 bronze badges

1 Comment

joanis Over a year ago

This is not really an answer to the question, which is about making such an open statement work from somewhere else than a specific directory.

Collectives™ on Stack Overflow

Reading a file using a relative path in a Python project

6 Answers 6

1 Comment

Comments

2 Comments

1 Comment

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

1 Comment

Comments

2 Comments

1 Comment

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related