ignore "blank" (unfilled) row with python csv reader

Question

everybody.

I can't find a pythonic way to ignore "blank" lines in a CSV. I use quotes because I'm talking about lines that look like '','','','','' Here is a CSV (blank lines could be random):

id,name,age
1,alex,22
3,tiff,42
,,
,,
4,john,24

Here is the code:

def getDataFromCsv(path):
    dataSet = []
    with open(unicode(path), 'r') as stream:
        reader = csv.reader(stream, delimiter=',')
        reader.next() # ignoring header
        for rowdata in reader:
            # how to check here?
            dataSet.append(rowdata)
    return dataSet

Here is similar questions that I've been reading, but different to this in particular: python csv reader ignore blank row

You can use if any(x for x in rowdata): dataSet.append(rowdata) — dekim
– dekim, Commented Jan 16, 2018 at 22:56
BTW, change to next(reader) to be compatible with Python3 also — tonypdmtr
– tonypdmtr, Commented Jan 16, 2018 at 23:01
together with schwobaseggl's @dekim solution works, exacly as expected. Thanks — alete
– alete, Commented Jan 17, 2018 at 1:41
@tonypdmtr good call, I was thinking only in 2.7.x, but I'll have that in mind for now on. — alete
– alete, Commented Jan 17, 2018 at 1:43

user2390182 · Accepted Answer · 2018-01-16 22:53:48Z

13

You can use any to check if any column in the row contains data:

for rowdata in reader:
    # how to check here?
    if any(x.strip() for x in rowdata):
        dataSet.append(rowdata)

answered Jan 16, 2018 at 22:53

user2390182

73.7k6 gold badges71 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

alete Over a year ago

Works as expected, it's clear and it taught me something. Thanks!

grego Over a year ago

Useful! If I understand correctly, this line if any(x.strip() for x in rowdata): can be read as: "if there are any values left after stripping all string values in the row, then the row has data and should be added to the dataSet'

grego Over a year ago

For those trying to scrub out empty strings from rows is you used csv.DictReader: remember to specify that x is teh lsit of values from your row dictionary, i.e. use this line: if any(x.strip() for x in list(row.values())):

user2390182 Over a year ago

@grego "if there are any values left after stripping all string values in the row" -- that's almost correct: 1. ... any values left that aren't the empty string ... 2. any stops at the first truthy element, i.e. not necessarily all elements get stripped, but only as many as it takes.

vt220 · Accepted Answer · 2021-06-21 13:56:20Z

0

Danger zone.. Maybe reviving an old thread..

Why not use a filter? Then there are no memory issues for large csv files, I think.

Something like:

for data in filter(any, reader):
    print(data)

edited Jun 21, 2021 at 13:56

answered Jun 21, 2021 at 13:46

vt220

11 bronze badge

Comments

Patrick O'Connor · Accepted Answer · 2018-01-16 22:53:41Z

-1

What about:

if len(rowdata) > 0:
    dataSet.append(rowdata)

Or am I missing a part of your question?

answered Jan 16, 2018 at 22:53

Patrick O'Connor

3341 silver badge6 bronze badges

1 Comment

user2390182 Over a year ago

This won't work as these "empty" rows are still 3 long. You must test if all of the 3 strings in the row are empty.

tommy.carstensen · Accepted Answer · 2019-03-19 14:43:51Z

-1

You can use the built-in function any:

for rowdata in reader:
    # how to check here?
    if not any(row):
        continue
    dataSet.append(rowdata)

answered Mar 19, 2019 at 14:43

tommy.carstensen

9,66215 gold badges70 silver badges113 bronze badges

Comments

T3metrics · Accepted Answer · 2019-05-08 16:30:05Z

-1

with open(fn, 'r') as csvfile:
    reader = csv.reader(csvfile)
    data = [row for row in reader if any(col for col in row)]

open CSV file
instantiate csv.reader() object
use a list comprehension to:
- iterate over CSV rows
- iterate over columns in the row
- check if any column in the row has a value and if so, add to the list

edited May 8, 2019 at 16:30

answered Mar 27, 2019 at 19:10

T3metrics

1852 silver badges12 bronze badges

2 Comments

chb Over a year ago

Hi, welcome to Stack Overflow. When answering a question that already has many answers, please be sure to add some additional insight into why the response you're providing is substantive and not simply echoing what's already been vetted by the original poster. This is especially important in "code-only" answers such as the one you've provided.

undetected Selenium Over a year ago

While this code may answer the question, providing additional context regarding how and/or why it solves the problem would improve the answer's long-term value.

Collectives™ on Stack Overflow

ignore "blank" (unfilled) row with python csv reader

5 Answers 5

4 Comments

Comments

1 Comment

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

4 Comments

Comments

1 Comment

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related