How can I replace all empty list [] values to None in the following df:
import pandas as pd
d = pd.Timestamp.now()
df = pd.DataFrame({'col1': ['a', 'b', 'c'],
'col2': [1, list(), 3],
'col3': ['x', 'y', list()],
'col4': [d, d, d]})
The solution:
df[df.applymap(lambda x: x == [])] = None gives TypeError: Cannot do inplace boolean setting on mixed-types with a non np.nan value
This is tricky! Because of the way pandas (and numpy) coerce lists, vectorizing this may actually be impossible. However, I might be wrong.
A clean, simple solution would be to use applymap to check if each value is [], and then replace using that mask:
df[df.applymap(lambda x: x == [])] = None
Output:
>>> df
col1 col2 col3
0 a 1 x
1 b None y
2 c 3 None
Try where
df.where(df.astype(bool),None,inplace=True)
df
Out[419]:
col1 col2 col3
0 a 1 x
1 b None y
2 c 3 None
Solved this by using applymap with boolean mask with:
import numpy as np
from collections.abc import Iterable
df[df.applymap(lambda x: len(x) == 0 if isinstance(x, Iterable) else False)] = np.nan