Using a generator in a while loop and evaluate after every yield

Question

I'm using Python to open some files in a CAD program. Since the program will crash when I open too many files at once, I want my script to stop opening files from a list I generated when the sum of thier filesize exceeds a certain value.

Here is what I have so far:

I'm converting the log file to a list. It contains the filepaths seperated by commas:

fList = []

with open('C:/Users/user/Desktop/log.txt', 'r') as f:
    fList = f.read().split(',')
    with suppress(ValueError, AttributeError):
        fList.remove('')
    fcount = len(fList)

This is the Generator that I use to Iterate over the partList:

def partGenerator(partList):
    for file in partList:
        yield file

Here I try to loop over the files while the sum of thier size is smaller than 2500000 bite:

count = 0
progression = 0
storage = 0

while storage < 2500000:
    for file in partGenerator(fList):
        name = os.path.basename(file)
        storage += os.path.getsize(file)
        print(f'Auslastung: {storage} bite / 2500000 bite')
        oDoc = oApp.Documents.Open(file)

        progression += 1
        percent = round(100 * progression / fcount)
        print(f'Fortschritt: {progression} / {fcount} ({percent} %) - {name}')

What happens is, that the files open propperly in the CAD Software, but they don't stop after the while condition is exceeded. My guess is, that the while condition is evaluated after the list runs out of entries and not after every entry like I what to.

Help on the correct syntax would be great!

What I'm looking for ultimately:

I would like to use this script in a way that it opens some files and whenever I manualy close one in the CAD program, It opens the next one from my list until the list is exhausted.

What's the point of partGenerator? You can iterate over the list directly, no need for a generator. — Aran-Fey
– Aran-Fey, Commented Mar 7, 2019 at 12:56
I make small changes to the files manually so I can not have the script close the files itself. — J.H
– J.H, Commented Mar 7, 2019 at 12:59
I want the script to pause the iteration over the list when the filesize of the previously opend files exceeds a value. My guess was, that it's only possible with a generator — J.H
– J.H, Commented Mar 7, 2019 at 13:02
Sorry, please don't alter questions after receiving answers that address your initial problem. If you have more questions after solving the first, post a new question. — Martijn Pieters
– Martijn Pieters, Commented Mar 7, 2019 at 13:17

Martijn Pieters · Accepted Answer · 2019-03-07 13:21:16Z

2

Your while condition is never checked, no, because the for loop never lets Python check. That the for loop takes elements from a generator function is neither here nor there.

You need to check inside your for loop if your condition still holds:

for file in partGenerator(fList):
    name = os.path.basename(file)
    storage += os.path.getsize(file)
    if storage >= 2500000:
        # wait for input before continuing, then reset the storage amount
        input("Please close some files to continue, then press ENTER")
        storage = 0

Python doesn't check while conditions until the full suite (series of statements) in the block under the while ...: statement has completed running or executes a continue statement, so a while condition really isn't suitable here.

In the above example I used the low-tech input() function to ask whomever is running the script to press ENTER afterwards. It'll depend on what oDoc.Documents actually offers as an API to see if you could use that to detect that files have been closed.

If you wanted to use a generator function, have it track file sizes. You can even have it read those from the CSV file. I'd use the csv module to handle splitting and progress, by the way:

import csv


def parts(logfile):    
    with open(logfile, newline='') as f:
        reader = csv.reader(f)
        files = [column for row in reader for column in row if column]
    fcount = len(files)
    storage = 0
    for i, filename in enumerate(files):
        storage += os.path.getsize(file)
        if storage >= 2500000:
            input("Please close some files to continue, then press ENTER")
            storage = 0
        print(f'Auslastung: {storage} bite / 2500000 bite')
        yield file
        print(f'Fortschritt: {i} / {fcount} ({i / fcount:.2%}) - {name}')

then just use

for file in parts('C:/Users/user/Desktop/log.txt'):
    oDoc = oApp.Documents.Open(file)

Note that the absolute number of files open is what your OS limits on, not how large those files are.

edited Mar 7, 2019 at 13:21

answered Mar 7, 2019 at 12:57

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

J.H Over a year ago

Thank you for the answer! I thought about using an if condition aswell, but then the script stops after meeting the size-condition right? I want the script to pause if possible until i close enought files so it can open the next ones until the list is exhausted

Martijn Pieters Over a year ago

@J.H: then don't exit, pause. Use input('Size limit reached, want to continue?')

Martijn Pieters Over a year ago

@J.H: Yes, your if you use a while condition the loop exits when the condition is no longer true, that's the point of such a loop. Don't use a while loop if you don't want to exit the loop.

J.H Over a year ago

Ok I think I got it. Thanks for the input!

J.H · Accepted Answer · 2019-03-07 14:06:37Z

With the input from Martijn Pieters I came up with something that works perfectly for me. I'm a noob in programming so it took me a while to understand the problem. Here is what woked just fine in the end:

fList = []

with open('C:/Users/jhoefler/Desktop/log.txt', 'r') as f:
    fList = f.read().split(',')
    with suppress(ValueError, AttributeError):
        fList.remove('')
    fcount = len(fList)

count = 0
progression = 0

for file in fList:

    name = os.path.basename(file)
    if oApp.Documents.Count < 10:
        oDoc = oApp.Documents.Open(file)
    else:
        pCount = oApp.Documents.LoadedCount
        fCount = oApp.Documents.LoadedCount
        while fCount == pCount:
            time.sleep(1)
            pCount = oApp.Documents.LoadedCount
        oDoc = oApp.Documents.Open(file)

    progression += 1
    percent = round(100 * progression / fcount)
    print(f'Fortschritt: {progression} / {fcount} ({percent} %) - {name}')

I'm sure there is a more elegant way to solve the problem, but it workes for my needs just fine.

Collectives™ on Stack Overflow

Using a generator in a while loop and evaluate after every yield

Here is what I have so far:

What I'm looking for ultimately:

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Here is what I have so far:

What I'm looking for ultimately:

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related