import pandas as pd
input_thing = [
[{'foo': 1, 'bar': 'a', }],
[{'foo': 2, 'bar': 'b', }],
]
print(input_thing)
How can I get a nice pandas dataframe out of the data displayed above?
I.e. that the columns foo and bar are nicely displayed on the outside
pd.DataFrame(input_thing)
Only will generate a dataframe with a single column 0
The desired output would be:
pd.concat([
pd.DataFrame(input_thing[0]),
pd.DataFrame(input_thing[1])
])
If every sublist in input_thing has one element:
df = pd.DataFrame([i[0] for i in input_thing])
print(df)
Prints:
foo bar
0 1 a
1 2 b
EDIT:
df = pd.DataFrame(input_thing)
print(df[0].apply(pd.Series))
Prints:
foo bar
0 1 a
1 2 b
pd.Series
df.apply(pd.Series) is very slow compared to list comprehension as it builds Series object for each item in the list which adds way too much overhead. Your better off using list comprehension.timeit test.pd.DataFrame(chain.from_iterable(input_thing)) too. Anyways nice one.Just replace your pd.concat with a for loop:
import pandas as pd
input_thing = [
[{'foo': 1, 'bar': 'a', }],
[{'foo': 2, 'bar': 'b', }],
]
df = pd.DataFrame([i[0] for i in input_thing])
print(df)
Then it will get what you want:
foo bar
0 1 a
1 2 b
This is what you desire.
