write lists to pandas dataframe to csv, read dataframe from csv and convert to lists again without having strings

Question

Originally I had a list of list and each list contains tuples of strings (from some computations). I want to save them for later, so I don't have to do all the computations again and just read the csv.

 L = [l1,l2,...]
 l1 = [('a','b'), ('c','d'),...]
 l2 = [('e','f'), ('g','h'),...]...

I converted it to a pandas data frame:

 import pandas as pd
 df = pd.DataFrame(L)
 df.to_csv('MyLists.csv', sep=";")

So each list l is saved as a row in the csv. Some time later I want to use the list saved in the csv again. So I imported pandas again and did:

readdf = pd.read_csv('MyLists.csv', delimiter = ";")
newList = readdf.values.tolist()

The problem is that every tuple is a string itself now, i.e. every list in newList looks as follows:

l1 = ['('a','b')', '('c', 'd')',...]

When I look at the csv with a text editor, it looks correct, somehow like:

('a','b');('c','d');...

I tried to read it directly with:

import csv

newList = []
with open('MyLists.csv') as f:    
    reader = csv.reader(f, delimiter=";")
    for row in reader:
        newList.append(row)

But the problem is the same. So how can I get rid of the extra " ' "?

jezrael · Accepted Answer · 2018-01-14 15:45:54Z

3

I think you need convert strings to tuples, because data in csv are strings:

import ast

l1 = [('a','b'), ('c','d')]
l2 = [('e','f'), ('g','h')]
L = [l1,l2]

df = pd.DataFrame(L)
print (df)
        0       1
0  (a, b)  (c, d)
1  (e, f)  (g, h)

df.to_csv('MyLists.csv', sep=";")

readdf = pd.read_csv('MyLists.csv', delimiter = ";", index_col=0)
newList = readdf.applymap(ast.literal_eval).values.tolist()
print (newList)
[[('a', 'b'), ('c', 'd')], [('e', 'f'), ('g', 'h')]]

But I think better is use pickle for save your data - use to_pickle / read_pickle:

df.to_pickle('MyLists.pkl')

edited Jan 14, 2018 at 15:45

answered Jan 14, 2018 at 15:17

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

write lists to pandas dataframe to csv, read dataframe from csv and convert to lists again without having strings

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related