Reading in Excel file with corrupt data using PYTHON

Question

I am trying to read in a table from a .CSV file which should have 5 columns. But, some rows have corrupt data..making it more than 5 columns.

How do I reject those rows and continue reading further ?

*Using

temp = read_table(folder + r'\temp.txt, sep=r'\t')

Just gives an error and stops the program*

I am new to Python...please help Thanks

are you sure you want a raw string for the delimiter? can you post your read_table function? — dm03514
– dm03514, Commented Dec 10, 2012 at 2:40
my delimiter is tab....that's how tab is used as a delimiter. The file is corrupted in some places. So, instead of 5 columns, there are 15...i need to reject the rows with more than 5 columns — Rahul Bhatia
– Rahul Bhatia, Commented Dec 10, 2012 at 2:43

timc · Accepted Answer · 2012-12-10 02:53:32Z

2

Look into using Python's csv module.

Without testing the damaged file it is difficult to say if this will do the trick however the csvreader reads a csv file's rows as a list of strings so you could potentially check if the list has 5 elements and proceed that way.

A code example:

out = []
with open('file.csv', 'rb') as csvfile:
    reader = csv.reader(csvfile, delimeter='    ')
    for row in reader:
        if len(row) == 5:
            out.append(row)

edited Dec 10, 2012 at 2:53

answered Dec 10, 2012 at 2:48

timc

2,19415 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

timc Over a year ago

I'm not totally familiar with the .lvm format but it seems that it is very similar to csv so it would be worth trying my suggestion. Failing that if you can open the .lvm file in a text editor and paste some sample data here that might help me work it out.

Rahul Bhatia Over a year ago

13/11/2012 7:31:59 AM t1= 17.1600, c1=-0.00034, s= 0.0000, lat=37 52.9112 S, lon=147 58.2424 E, hms=203149, dmy=121112 13/11/2012 7:32:00 AM t1= 17.1598, c1=-0.00036, s= 0.0000 13/11/2012 7:32:01 AM t1= 17.1598, c1=-0.00034, s= 0.0000, lat=37 52.9111 S, lon=147 58.2423 E, hms=203151, dmy=121112 IT IS SPACE DELIMITED

timc Over a year ago

What happens when you try my suggestion?

Rahul Bhatia Over a year ago

first, it is not a CSV file...when i use the read function...f = open(folder + r'/FLNTU_RAW_12-11-13_0731_000001.lvm','rb') ln = f.read().....the row(len) is always 1

timc Over a year ago

Read my suggestion properly. It doesn't really matter if it is a csv file or not I would still try to use the csv module. Using open() to open the file doesn't work in the same way that using the csv module does.

|

Collectives™ on Stack Overflow

Reading in Excel file with corrupt data using PYTHON

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related