Writing from a large list to a text file in Python

Question

Something strange happens when I try to write from a ordinary list, containing 648470 string-values, to a text file.

    textFile = open('bins.txt', 'w')

    for item in bigList:
        print item # This prints fine
        textFile.write(item)

    textFile.close()

The text file grows rapidly in file size and is filled with all kind of symbols, not the intended ones... Even if I just write a small span of the content of bigList the text file gets corrupted. If I do the exact same thing with a much smaller list, there's no problem. Is the large size of the list causing this problem? The output of print(bigList[:10]) is

['167', '14444', '118', '22110', '118', '8134', '82', '8949', '7875', '171']

@Smajjk Are you opening the file like textFile = open(path, 'w')? And are you sure you are writing text, and not numbers into each line? — john_science
– john_science, Commented Feb 7, 2013 at 18:36
I forgot to mention, an extract from the list would be good too - as given by Python if you do print(bigList[:20]), for example. — Gareth Latty
– Gareth Latty, Commented Feb 7, 2013 at 18:44
I notice you're opening this was 'a', not 'w'. What was in the file before? And—not to be patronizing, but I've made this kind of mistake before—are you sure you're looking at the data you appended, rather than the garbage that was already there? In fact, are you sure you appended anything at all (since you don't close the file, it may never flush the buffer to disk)? — abarnert
– abarnert, Commented Feb 7, 2013 at 19:01
What program do you use to view the file? The file consists of one extremely long line; perhaps that triggers a bug in the program. Try another way of viewing the file. — Janne Karila
– Janne Karila, Commented Feb 7, 2013 at 20:31

LtWorf · Accepted Answer · 2013-02-07 19:01:22Z

1

It works absolutely fine to me.

In your code you are forgetting to close the file, and also, since you open the file in append mode, my guess is that you have some garbage in the file that was there and you forgot to remove.

Also, keep in mind that the write in that way will not separate the numbers in any way.

answered Feb 7, 2013 at 19:01

LtWorf

7,6786 gold badges36 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Smajjk Over a year ago

I changed from append mode to write mode and also closes the file. When I write using textFile.write(item) I do still not get the long line of numbers I am expecting but a mess of symbols, looks almost like there are written on top of eachother! But when I use writeFile.write("%s\n" % item) it works?!

LtWorf Over a year ago

What version of python and OS are you using?

Smajjk Over a year ago

xubuntu 12.10 and Python 2.7.3

LtWorf Over a year ago

Are you sure the list only contains string objects?

LtWorf Over a year ago

try this to check: reduce(lambda x,y: x and y, [isinstance(i,str) for i in bigList])

|

Steven · Accepted Answer · 2013-02-07 19:44:57Z

0

It is possible that the file is having problems writing because some of the list objects are not strings. Try:

textFile = open('bins.txt', 'w')

for item in bigList:
    print item # This prints fine
    textFile.write(str(item))

textFile.close()

However I cannot see your list so I do not know for certain if this is an actual problem.

answered Feb 7, 2013 at 19:44

Steven

1951 gold badge3 silver badges10 bronze badges

1 Comment

Steven Over a year ago

Hmm. I'm not sure why this wouldn't write then... It's odd.

Collectives™ on Stack Overflow

Writing from a large list to a text file in Python

2 Answers 2

7 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related