Read csv file into list and convert the strings into integers Python

Question

I am trying to read in a CSV file and then returns all the data into a list. I first tried to read in the csv file using csv.reader by using the following code:

import csv
with open(fileName, 'r') as f:
    next(f)
    data = csv.reader(f)
    dataList = list(data)

and I now have a list that looks like:

[['123', '234', '456', '567']
['345', '3456', '5678', '678']
['2345', '4567', '45678', '56789']
...]

I noticed that the numbers are stored as strings in the list, so I created a new list by newList = [int(i) for i in dataList] and I got this error message TypeError: int() argument must be a string, a bytes-like object or a number, not 'list'

so I tried the following code I saw on Youtube:

with open('fileName', 'r') as f:
    next(f)
    data = csv.reader(f)
    data_lst = []
    for line in reader:
        data_lst.append([int(line)])

and I get this error message:

ValueError                                Traceback (most recent call last)
<ipython-input-123-9fbefdb892ab> in <module>
      3     data = csv.reader(f)
      4     data_lst = []
----> 5     for line in reader:
      6         data_lst.append([int(line)])

ValueError: readline of closed file

Does anyone know how to convert the strings in the list into integers?

Many Thanks!

Sample csv file

    number1    number2    number3    number4
0    123        456        567        5678
1    4567       3456       6789       2345   
....

I need to read this csv file and that store the data in a list so that each row in the csv file is a list like [[123, 456, 567, 5678], [4567, 3456, 6789, 2345]...]

It seems the problem is that you have a nested list, do: [[int(i) for i in l] for l in dataList] — Dani Mesejo
– Dani Mesejo, Commented Oct 24, 2019 at 1:02

Grismar · Accepted Answer · 2019-10-24 01:40:07Z

1

The answer @brunoto provided is correct, I'd prefer this solution (closer to yours):

with open('fileName', 'r') as f:
    next(f)
    data = csv.reader(f)
    data_lst = []
    for line in reader:
        data_lst.append([int(x) for x in line])

Note that all that was needed was a change to the final line, looping over the elements of each line and turning them into int individually, instead of trying to turn the whole line (a record, i.e. a list) into a single int.

answered Oct 24, 2019 at 1:40

Grismar

32.4k6 gold badges43 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Laksh Matai · Accepted Answer · 2019-10-24 01:13:47Z

0

This is happening because python3 is reading and writing files in binary. So you can either convert bytes data in to string and continue, or use pandas to read the data which will mostly read your numbers as integers.

import pandas as pd

df = pd.read_csv("filename")

rows = df.values.tolist()

Pandas gives you more freedom to perform operations on rows. You can also use something like below to convert column's datatype into int

df[column_name]=df[column_name].astype(int)

answered Oct 24, 2019 at 1:13

Laksh Matai

1761 silver badge5 bronze badges

1 Comment

Grismar Over a year ago

This is not actually true - using open(), Python opens files in text mode, so reading is fine. The error of OP is actually in properly accessing the list.

brunoto · Accepted Answer · 2019-10-24 01:45:52Z

It's because you have one list inside another one. Like you told us:

dataList = [['123', '234', '456', '567'],['345', '3456', '5678', '678'],['2345', '4567', '45678', '56789']]

In your code you're iterating over the inner lists, so you just need to iterate over the strings inside the inner lists. Like this:

int_list = []
    for inner_list in dataList:
        for string in inner_list:
            int_list.append(int(string))

or, with single line:

int_list = [int(string) for inner_list in dataList for string in inner_list]

That will have the output bellow:

[123, 234, 456, 567, 345, 3456, 5678, 678, 2345, 4567, 45678, 56789]

@edit

If you want a list of list of values, as the input, the code bellow will work.

outer_out_list = []
for inner_list in dataList:
    innet_out_list = []
    for string in inner_list:
        innet_out_list.append(int(string))
    outer_out_list.append(innet_out_list)

Output:

[[123, 234, 456, 567], [345, 3456, 5678, 678], [2345, 4567, 45678, 56789]]

Collectives™ on Stack Overflow

Read csv file into list and convert the strings into integers Python

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related