Loops in python does not loop through every line in file

Question

I am trying to loop through every line in a text file and perform some actions. Right now I have a text file which contains this:

--- small modified --- #line 1
1,2,3                  #line 2
4,5,6                  #line 3
--- big modified ---   #line 4
7;8;9                  #line 5
10;11;12               #line 6

I am trying to parse line 2,3 into one file, and lines 5,6 into another file but right now, only lines 2 and 3 gets written into the file and idk why the "elif" statement is not run. I can't solve the logic error and would appreciate if someone could help me out.

Below is my code:

def convert_json(fileName):
    with open(fileName,'r') as file:
        for line in file:
            if 'modified' and 'small' in line:
                for li in file:
                    fields1 = li.split(',')
                    if len(fields1) >= 3:
                            smallarr.append({
                            "a": fields1[0],
                            "b": fields1[1],
                            "c": fields1[2]
                                })
                            with open('smalljson.txt','w+') as small_file:
                                json.dump(smallarr, small_file)
                    else:
                        pass

            elif 'modified' and 'big' in line:
                for li in file:
                    fields2 = li.split(';')
                    if len(fields2) >= 3:
                            bigarr.append({
                            "w1": fields2[0],
                            "w2": fields2[1],
                            "w3": fields2[2],
                                })
                            with open('big.txt','w+') as big_file:
                                json.dump(bigarr, big_file)
                    else: 
                        pass



            else:
                print 'test'

Update: THis is my current code, I am able to do it but only for lines 2 and lines 5, other than s second for-loop i cannot think of another way to loop through the lines

def convert_json(fileName):
with open(fileName,'r') as file:
    for line in file:
        #if 'modified' in line and 'small' in line:
        if 'modified' in line and 'Small' in line:
            fields1 = next(file).split(',')
            if len(fields1) >= 3:
                smallarr.append({
                "a": fields1[0],
                "b": fields1[1],
                "c": fields1[2]
                })
                with open('smalljson.txt','w+') as small_file:
                    json.dump(smallarr, small_file)
            else:
                pass



        elif 'modified' in line and 'big' in line:
            fields2 = next(file).split(';')
            if len(fields2) >= 3:
                bigarr.append({
                "w1": fields2[0],
                "w2": fields2[1],
                "w3": fields2[2],
                })
                with open('bigwater.txt','w+') as big_file:
                    json.dump(bigarr, big_file)
            else:
                pass

        else:
            print 'test'

What have you tried to do to fix this? Have you tried inserting debug messages? Or stepping the program through with your IDE's debugger (if there is one)? — Okx
– Okx, Commented Feb 6, 2017 at 16:58
I tried playing around the loops, when i insert a break statement after the first else statement, only line 2, line 5 and 6 gets written. Line 3 is not written. — decemberrobot
– decemberrobot, Commented Feb 6, 2017 at 17:02
I'm pretty sure your inner for-loop will iterate through the entire file exhausting the iterator you are sharing across loops. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Feb 6, 2017 at 17:06
@juanpa.arrivillaga any suggestions how to stop the inner for-loop from doing that ? — decemberrobot
– decemberrobot, Commented Feb 6, 2017 at 17:07
You can break out. But you should be doing this with exactly 1 for-loop. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Feb 6, 2017 at 17:08

Dmitry Shilyaev · Accepted Answer · 2017-02-06 17:43:20Z

Your parsing logic need to be changed. Here is what code looks like, use it for reference in future improvements.

def file_parser(self):
    file_section = 0

    smallarr = []
    bigarr = []
    with open('data.txt') as in_file:
        for in_line in in_file:
            in_line = in_line.strip()

            if 'small' in in_line:
                file_section = 1
                continue
            elif 'big' in in_line:
                file_section = 2
                continue

            if file_section == 1:
                fields1 = in_line.split(',')
                if len(fields1) >= 3:
                    smallarr.append({
                        "a": fields1[0],
                        "b": fields1[1],
                        "c": fields1[2]
                    })
            elif file_section == 2:
                fields2 = in_line.split(';')
                if len(fields2) >= 3:
                    bigarr.append({
                        "w1": fields2[0],
                        "w2": fields2[1],
                        "w3": fields2[2],
                    })

    with open('small.txt', 'w+') as small_file:
        json.dump(smallarr, small_file)

    with open('big.txt', 'w+') as big_file:
        json.dump(bigarr, big_file)

Input data:

--- small modified ---
1,2,3
4,5,6
--- big modified ---
7;8;9
10;11;12

small.txt

[{"a": "1", "c": "3", "b": "2"}, {"a": "4", "c": "6", "b": "5"}]

big.txt

[{"w3": "9", "w2": "8", "w1": "7"}, {"w3": "12", "w2": "11", "w1": "10"}]

Thank you so much for your help. It is a working program now :)

caot · Accepted Answer · 2017-02-06 17:00:17Z

1

change

elif 'modified' and 'big' in line:

into

elif 'modified' in line and 'big' in line:

answered Feb 6, 2017 at 17:00

caot

3,3681 gold badge37 silver badges44 bronze badges

3 Comments

decemberrobot Over a year ago

I tried, does not seem to work. Still the "elif" statement is not executed

caot Over a year ago

change your logic, and remove "for li in file:"

juanpa.arrivillaga Over a year ago

Your first if condition suffers from the same defect.

daragua · Accepted Answer · 2017-02-06 20:29:05Z

There are a few issues in your code.

Firstly you are repeating yourself. The big and small cases don't vary enough to justify the code duplication.

Secondly, while I understand what you're trying to do with next(file), you'd still need to loop that instruction in some way to go get the next lines. But wait, you're already doing that exactly with for line in file.

Finally, at each loop, you're reopening the same file and redumping an ever increasing array. This is wasteful IO. If you're trying to stream from file into bigwater.txt and smalljson.txt and not storing too much stuff in memory, this is the wrong approach since json.dump can't be used to stream data.

Here's my take at it:

def convert_json(fileName):
    big = []
    small = []
    with open(fileName,'r') as file:
        for line in file:
            line = line.strip()
            if line.startswith("--"):
                if "big" in line:
                    array = big
                    keys = ["w1", "w2", "w3"]
                    sep = ";"
                else:
                    array = small
                    keys = ["a", "b", "c"]
                    sep = ","
                continue

            values = line.split(sep)
            # todo: make sure sizes match
            mapping = dict(zip(keys, values))
            array.append(mapping)

    with open('smalljson.txt','w') as small_file:
        json.dump(small, small_file)
    with open('bigwater.txt','w') as big_file:
        json.dump(big, big_file)

Collectives™ on Stack Overflow

Loops in python does not loop through every line in file

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related