Python: Write a list to a text file & text file size limit

Question

Sorry for the lengthy description of the problem definition.

I'm new to python. Started learning about a month back. Today i was trying to write a list into a text file. Though i got the answer in this site and used the same method. And it worked great. But i encountered with another problem while writing the list into the text file.

The length of my list is '2372'. When I write the list into text file ,In a specific format required for me, I found only few list members were written. (Format is : First four list members in first row. Likewise, group of 4 list members in rest of the rows.) I found only 1938 list members are written into the text file. The size of the file found out to be 25KB.

I also tried to write each list member in new row, then also the size of text file found to be 25KB and this time even less list members were written compared to above case.

Is there a file size limitation to the text file in which list is written ? Kindly help me to solve this issue.

** The snippet of python code **

Lenlist = 2372

filptr = open(tempfile, 'w+')
count=0
for x in range(0,Lenlist):
  filptr.write("%s\t" % line[x])
  count+=1
  if count>3:
   if (count%4==0):
     filptr.write("\n")

The maximum file size is determined by your OS, but it's unlikely you can fit python on a filesystem that won't allow files larger than 25K; there's no way that's relevant. — Wooble
– Wooble, Commented Sep 25, 2012 at 13:27
@prasanna21 -- Then that's probably your problem. You need to close() a file to guarantee that it is flushed appropriately (or, better use a context manager (with statement)). — mgilson
– mgilson, Commented Sep 25, 2012 at 13:32
@mgilson : U r absolutely correct. After using close() statement, the text file is written with all the list members. From next time ill be careful while using the file handlers. Thanks. — prasanna21
– prasanna21, Commented Sep 26, 2012 at 6:43

Martijn Pieters · Accepted Answer · 2012-09-25 13:44:43Z

2

It'll be easier to use iterate over your list:

with open(tempfile, 'w+') as filptr:
    for count, l in enumerate(line):
        filptr.write("%s\t" % l)
        if count and count % 4 == 0:
             filptr.write("\n")

The use of the with statement ensures that the file is closed properly when the loop is done. Using enumerate let's us eliminate having to use a manual counter.

Note that now your lines always have a \t extra tab at the end. You can use a itertools.izip_longest() trick to grab 4 items at a time from your list and write those together with a '\t'.join():

from itertools import izip_longest

per_four = izip_longest(*([iter(line)] * 4), fillvalue='')

with open(tempfile, 'w+') as filptr:
    for entries in per_four:
        filptr.write("\t".join(entries) + '\n')

Here I use izip_longest to take an item from the list 4 times each time we iterate over it, until the original list has been exhausted. To make sure we have 4 items at the end no matter what the length of the list, we use an empty string as padding:

>>> line = [1, 2, 3, 4, 5]
>>> for four in izip_longest(*([iter(line)] * 4), fillvalue=''):
...     print four
... 
(1, 2, 3, 4)
(5, '', '', '')

edited Sep 25, 2012 at 13:44

answered Sep 25, 2012 at 13:27

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

mgilson Over a year ago

Why not use enumerate while you're at it?

Martijn Pieters Over a year ago

I was still at it, indeed. :-)

mgilson Over a year ago

Also, careful using l as a variable name. People will start quoting PEP-8 at you :)

jterrace Over a year ago

also has a trailing \t on each line, not sure if that matters, as the OP has it too

Martijn Pieters Over a year ago

@jterrace: Offered an option that does away with the extra tab.

|

Ashwini Chaudhary · Accepted Answer · 2012-09-25 13:29:05Z

0

something like this, using a generator:

lis=[1,2,3,,,,,]  #your list
with open(filename,'w') as f:
    for y in (lis[x:x+4] for x in range(0,len(lis),4):
       f.write("%s\n",y)

as:

In [34]: lis=range(20)

In [35]: [lis[x:x+4] for x in range(0,len(lis),4)]
Out[35]: 
[[0, 1, 2, 3],
 [4, 5, 6, 7],
 [8, 9, 10, 11],
 [12, 13, 14, 15],
 [16, 17, 18, 19]]

answered Sep 25, 2012 at 13:29

Ashwini Chaudhary

252k60 gold badges478 silver badges519 bronze badges

Comments

Antimony · Accepted Answer · 2012-09-25 13:29:13Z

0

The problem is that file writes are buffered, so they aren't written to disk immediately. If you exit without closing the file, they may not be written at all (especially problematic in Jython and Pypy due to lack of deterministic GC).

If you've got Python 2.6 or higher, you can use a with statement to automatically close the file when you're done.

answered Sep 25, 2012 at 13:29

Antimony

39.8k13 gold badges103 silver badges115 bronze badges

Collectives™ on Stack Overflow

Python: Write a list to a text file & text file size limit

3 Answers 3

7 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

7 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related