I am trying to read a CSV file with the read_csv method. The column item_number needs to be read as a list.
Example : 'item_number' : ['123456']
df = pd.read_csv(filepath, sep =',', dtype = {'item_number' = list})
dtype = {'item_number' = list} ->does not work, while when I write it as dtype = {'item_number' = str} , it does work. Tried using converters but want to do it with dtype.
I don't think list is a valid pandas dtype. You can read it in as a string and subsequently convert it to a list
file_str = '''1|[123456]
2|[345678]'''
df = pd.read_csv(io.StringIO(file_str), sep='|', header=None)
df[2] = df[1].apply(lambda x: list(x[1:-1]))
print(df)
# 0 1 2
#0 1 [123456] [1, 2, 3, 4, 5, 6]
#1 2 [345678] [3, 4, 5, 6, 7, 8]
I think you mean by?
df = pd.read_csv(filepath, sep =',', dtypes = {'item_number': list})
But I am not sure with your dataset if that would work or not, so I guess you need:
df = pd.read_csv(filepath, sep =',', dtypes = {'item_number': pd.eval})
Edit:
Try:
df = pd.read_csv(filepath, sep =',')
df['item_number'] = df['item_number'].str[1:-1].map(list)
