More efficient way of iterating over a text-file in python?

Question

What I have attempted to do is read the file and store the information in separate lists. For this program a small example file is used to first ensure the code works but this program will deal with photos in the 100,000's and this doesn't seem like the best way to index the file for optimal efficiency. This is what I have so far:

with open('a_example.txt') as example_file:    
    content = [i.strip() for i in example_file.readlines()]
    
    number_of_photos = content[0]
    del content[0] #remove the numphoto info
    for j in content:
        orientation_of_photo.append(j[0])
        number_of_tags.append(j[2])

What efficiency are you worrying about? CPU, RAM, DiskIO, ...? What's wrong with the code? Who complains and why? Ask that person what efficiency would be acceptable. — Thomas Weller
– Thomas Weller, Commented Feb 10, 2021 at 18:39
Stop using .example_file.readlines(), you can iterate directly over file objects, content = [i.strip() for i in examplefile.readlines()] although you are working with small files, this won't really affect things, since the bulk of the time will be spent on I/O — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Feb 10, 2021 at 18:42

OneCricketeer · Accepted Answer · 2021-02-10 18:42:13Z

2

The only reason I can tell you are indexing the file is to get the first line.

You can use next() to get this, then continue on with the loop

with open('a_example.txt') as example_file:    
    number_of_photos = next(example_file).strip()
    for line in example_file:
        j = line.strip()
        orientation_of_photo.append(j[0])
        number_of_tags.append(j[2])

answered Feb 10, 2021 at 18:42

OneCricketeer

193k20 gold badges147 silver badges277 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

juanpa.arrivillaga Over a year ago

maybe, orientation, _, num_tags = line.strip()

OneCricketeer Over a year ago

Maybe, but I would think "number of tags" should be more than a single digit/character

Collectives™ on Stack Overflow

More efficient way of iterating over a text-file in python?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related