How to delete an error bad line in file.csv?
import pandas as pd
df = pd.read_csv(r"G:\file.csv", delimiter=',', error_bad_lines=False)
print(df.head())
My file is like this
column1, column2, column3
test, test, test
test2, test2, test2, te,st2
test3, test3, test3
te,st4, test4, te,st4, test4
I want to delete this line from file.csv and record to error.txt
test2, test2, test2, te,st2
te,st4, test4, te,st4, test4
Here is a way to do it without pandas.
import csv
error_liens=[]
good_lines=[]
with open(r'D:\test.csv', 'r') as file:
reader = csv.reader(file)
#if the first line is header
header=0
for row in reader:
if header==0:
header=1
error_liens.append(",".join(row))
good_lines.append(",".join(row))
continue
#checking if number of columns is not equal to 3 (bad lines)
if len(row)!=3:
error_liens.append(",".join(row))
else:
good_lines.append(",".join(row))
#write the error list to error file
f = open("error.csv", "w")
f.write("\n".join(error_liens))
f.close()
#write the good_lines list to correct lines file
f = open("good.csv", "w")
f.write("\n".join(good_lines))
f.close()
not sure if you can get a separate bad lines files using pandas.
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 1455: character maps to <undefined>
with open(r'D:\test.csv', 'r') as file: with with open(r'D:\test.csv', 'r', encoding='utf-8') as file: for encoding issueUnicodeEncodeError: 'charmap' codec can't encode characters in position 49470-49474: character maps to <undefined> in this line --> 22 f.write("\n".join(error_liens))f=open("error.csv","w",encoding="utf-8") while opening the file before f.write() for both the files.