I only need to read the first line of a huge file and change it.
Is there a trick to only change the first line of a file and save it as another file using Python? All my code is done in Python and would help me to keep consistency.
The idea is to not have to read and then write the whole file.
shutil.copyfileobj() should be much faster than running line-by-line. Note from the docs:
Note that if the current file position of the [from_file] object is not 0, only the contents from the current file position to the end of the file will be copied.
Thus:
from_file.readline() # and discard
to_file.write(replacement_line)
shutil.copyfileobj(from_file, to_file)
to_file? just w?If you want to modify the top line of a file and save it under a new file name, it is not possible to simply modify the first line without iterating over the entire file. On the bright side, as long as you are not printing to the terminal, modifying the first line of a file is VERY, VERY fast even on vasy large files.
Assuming you are working with text-based files (not binary,) this should fit your needs and perform well enough for most applications.
import os
newline = os.linesep # Defines the newline based on your OS.
source_fp = open('source-filename', 'r')
target_fp = open('target-filename', 'w')
first_row = True
for row in source_fp:
if first_row:
row = 'the first row now says this.'
first_row = False
target_fp.write(row + newline)
'\r\n' is almost never correct in any context. If you want to read/write text files with python use 'rt' or 'wt' mode.
wt and rt appear to be Python 3-specific implementations. generally, it's best to write code which is python 2.5+ and 3 compatible if at all possible. Python 2.7 Docs: docs.python.org/2.7/library/functions.html#open Python 3.3 Docs: docs.python.org/3.3/library/functions.html#openAn alternate solution that does not require iterating over the lines that are not of interest.
def replace_first_line( src_filename, target_filename, replacement_line):
f = open(src_filename)
first_line, remainder = f.readline(), f.read()
t = open(target_filename,"w")
t.write(replacement_line + "\n")
t.write(remainder)
t.close()
Unless the new line is the same length as the old line, you can not do this. If it is, you could solve this problem through a mmap.
The sh module worked for me:
import sh
first = "new string"
sh.sed("-i", "1s/.*/" + first + "/", "file.x")
The solution i would use is to use create a file missing old first line
from_file.readline() # and discard
shutil.copyfileobj(from_file, tail_file)
then create a file with the new first line
then use the following to concatenate the newfirstline file and tail_file
for f in ['newfirstline.txt','tail_file.txt']:
with open(f,'rb') as fd:
shutil.copyfileobj(fd, wfd, 1024*1024*10
Here is the working example of "Nacho" answer:
import subprocess
cmd = ['sed', '-i', '-e', '1,1s/.*/' + new_line + '/g', 'filename.txt']
subprocess.call(cmd)
