How to handle lists as single values in csv with Python

Question

I am handling a csv import and got troubles with a value that should be in list form but is read as string.

One of the csv rows looks like the following:

    ['name1', "['name2', 'name3']"]

As you can see the value in the second column is a list but is read as a string. My problem is that I need to iterate through that list and the length of that list can vary from row to row.

I am wondering where the problem lies. Can csv read not handle a list? Is there a way to turn that string in the second column into a list rather than using regex? Here is the code that I am running:

    import csv
    import os

    content = []
    file_path = os.path.abspath(file)
    if os.path.exists(file_path):
      with open(file_path, 'rb') as csvfile:
        csvreader = csv.reader(csvfile, delimiter = ',')
      for row in csvreader:
        content.append(row)
      for row in content[1:5]:
        print row
      print row[0], row[1]
      for name in row[1]:
        print name

The output row looks as above but when iterating though row[1] it does not iterate though the list of names but through every single character. Anyone got an idea? Thanks in advance for any help!

No, the csv module cannot turn a string into a list, can you add a sample of what your input looks like? How did you end up with a list written to the file? — Padraic Cunningham
– Padraic Cunningham, Commented Jul 8, 2015 at 8:55
Thanks @PadraicCunningham for quick answer. In a different application it made sense to save multiple string values in one cell to csv. Rows look like in the first example ['name1', "['name2', 'name3']"]. Probably there is a better way to handle this? — Justus Niemzok
– Justus Niemzok, Commented Jul 8, 2015 at 9:12
why is the structure ['name1', "['name2', 'name3']"].? How do you end up with nested lists? — Padraic Cunningham
– Padraic Cunningham, Commented Jul 8, 2015 at 9:33
@PadraicCunningham I'll give some more context. csv shows interactions between different users. So full line looks the following: [date, text, id, 'name1', "['name2', 'name3']"] meaning that user1 mentions both user2 and user3. At time of creating csv it seemed like most convenient solution — Justus Niemzok
– Justus Niemzok, Commented Jul 8, 2015 at 10:14
I think maybe storing dicts or maybe use pandas would be a good idea, what are you doing with the data? — Padraic Cunningham
– Padraic Cunningham, Commented Jul 8, 2015 at 10:18

Community · Accepted Answer · 2020-06-20 09:12:55Z

3

An easy way to convert string to list is using ast.literal_eval function.

Example -

>>> import ast
>>> s = "['name2', 'name3']"
>>> s
"['name2', 'name3']"
>>> l = ast.literal_eval(s)
>>> l
['name2', 'name3']
>>> type(l)
<class 'list'>

From ast documentation -

ast.literal_eval(node_or_string)

Safely evaluate an expression node or a Unicode or Latin-1 encoded string containing a Python literal or container display. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None.

But if your complete csv looks like that, you should consider using json to parse the csv, rather than csv module.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Jul 8, 2015 at 8:56

Anand S Kumar

91.5k18 gold badges196 silver badges179 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Abhijit · Accepted Answer · 2015-07-08 08:55:07Z

1

Considering your second list item is a valid python data type, you can use ast.literal_eval to parse the string

>>> import ast
>>> ast.literal_eval("['name2', 'name3']")
['name2', 'name3']

So in your particular case,you may want to do the following

  .............
  row[1] = ast.literal_eval(row[1])
  print row[0], row[1]
  for name in row[1]:
    print name

answered Jul 8, 2015 at 8:55

Abhijit

64k20 gold badges143 silver badges209 bronze badges

Comments

itzMEonTV · Accepted Answer · 2015-07-08 08:56:44Z

1

Try this, using literal_eval to convert string to corresponding class

from ast import literal_eval
for name in literal_eval(row[1]):
   print name

or

for name in eval(row[1]):
   print name

answered Jul 8, 2015 at 8:56

itzMEonTV

20.4k4 gold badges44 silver badges53 bronze badges

Comments

vks · Accepted Answer · 2015-07-08 09:00:28Z

0

x=['name1', "['name2', 'name3']"]
import re
print [ast.literal_eval(i) if re.match(r"\[.*?\]",i) else i for i in x ]

OUTPUT: ['name1', ['name2', 'name3']]

You can use ast.literal_eval and re to convert string in list to list and leave others as they are.

answered Jul 8, 2015 at 9:00

vks

68.1k11 gold badges96 silver badges132 bronze badges

Collectives™ on Stack Overflow

How to handle lists as single values in csv with Python

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related