75 questions
0
votes
3
answers
138
views
How to normalize a JSON Object that has nested arrays and lists in python pandas
I have the following json data and i've been trying to flatten it out into a single row. I tried a few methods like explode() and json_normalize(data, max_level=3), flatten_json. The code does not ...
1
vote
0
answers
48
views
split multi-valued cell (with delimeter) to multiple rows with copied data [duplicate]
I have an Excel table (export from MS MDE, to be precise) of vulnerabilities along with the machines on which they occur. One vulnerability can occur on hundreds of devices and I would like to ...
- 57
0
votes
2
answers
70
views
Adding quotes to list objects to format as a dictionary pyspark
I have a column of my dataframe data that contains date and value information in a long string. The column, we will call x for this purpose, is formatted as such:
x = "{date1:val1, date2:val2, ...
- 13
1
vote
0
answers
30
views
Google sheets equivalent of Explode in python? [duplicate]
I have a table with one of the columns containing a list of comma delimited values in google sheets. How do I transform each element of a list-like to a row, replicating index values.
Python ...
0
votes
0
answers
46
views
Parse, transpose and insert a new row [duplicate]
I have a large spreadsheet and need to parse, transpose and insert the parsed values into a new row.
My sample data:
What I need to do is format the sheet to read as below:
I tried
TOCOL(A2:(...
- 33
0
votes
1
answer
98
views
Separate excel columns with different lengths into multiple rows using Python
I am attempting to use python to loop through rows in an excel spreadsheet and separate data in a row into multiple rows if the column in question has multiple newlines or carriage returns.
In the ...
0
votes
0
answers
46
views
Performance improvement of multiple s.str.split calls
DataFrame has the following columns
NCT Number object
Study Title object
Study URL object
Acronym ...
2
votes
1
answer
80
views
itertools.product in dataframe
Inputs:
arr1 = ["A","B"]
arr2 = [[1,2],[3,4,5]]
Expected output:
short_list
long_list
0
A
1
1
A
2
2
B
3
3
B
4
4
B
5
Current output:
short_list
long_list
0
A
[1, 2]
1
A
[3, 4, 5]...
- 1,051
0
votes
2
answers
79
views
Parsing or exploding a list of dictionaries in a dataframe
I have a dataframe with lists of nested dictionaries that want to unpack.
I need to get the date and price from the priceHistory and the items listed in both WaterConservation and EnergyEfficient. The ...
- 15
0
votes
1
answer
54
views
xplode data in google sheet [duplicate]
I got a table like this
Category is a multiple selection, the count simply is the number of occurrences. I want to tranform to this table
.
In python, i can use xplode function. But u want using ...
1
vote
2
answers
697
views
How do you deal with multiple values in a single cell separated by comma?
Student
Fruits
Foo
Apple,Banana
Bar
Orange
Baz
Lemon,Orange
How do I quantify/analyze the column Fruits with multiple values in a single cell separated by comma?
I created multiple helper columns, ...
- 57
-1
votes
3
answers
178
views
Explode command not working while calling excel or csv file
i have been working with this explode command. please here is my code fyr.
import pandas as pd
import os
os.getcwd()
os.chdir('C:/Users/nick/Documents/Python') # NOTE csv_filepath
df= pd.read_excel('...
- 35
0
votes
2
answers
239
views
Google Sheets Splitting Text to Rows and Duplicate Surrounding Rows Entries
enter image description here
I want to change the data from table on the left to table on the right by using formula or function.
I already try some formulas but it only work for one column.
All help ...
- 3
1
vote
1
answer
188
views
Pandas groupby mean and value_counts but using another column of counts
I need a memory-efficient way of exploding a dataframe and then grouping by a column to get either normalized value_counts or the mean depending on dtype.
I have a dataframe similar to this:
key ...
- 291
2
votes
3
answers
171
views
Distributing Values in a Pandas dataframe with different time intervals
I have a pandas data frame which has the data sorted into quarters like this: ( I am showing a sample data)
df
id qtr value
a Q01 100
a Q02 130
a Q03 160
a Q04 100
b ...
- 884
1
vote
0
answers
24
views
I'm trying to explode a string in python pandas by a delimiter but it isn't working [duplicate]
df = pd.DataFrame({'index': 1 , 2],
'artist': ['Elton John', 'Biffy Clyro'],
'song': ['Rocket Man', 'Bubbles']
'id': ['B12, U6, Z5', 'H91, D293, ...
- 11
1
vote
2
answers
187
views
exploding deeply nested json (json flatten) / flattening pandas df
I have the data coming via REST api with nested json, Trying to explode the response but its flatteing in only the first level. Need to explode the nested part also.
First step im converting data into ...
- 61
1
vote
1
answer
237
views
Pandas pivot + date slicing: group by periods of time with partial overlap
I am trying to find a way to 'pivot' my pandas dataframe, but keeping my index by sliced dates. The end goal is to create a range for each index in which each attributes and their values are matched.
...
- 375
0
votes
2
answers
157
views
In PySpark, how do I get word frequency in a column when a row can contain multiple words?
Assume a two column PySpark DataFrame with 3 rows:
["Number"] [ "Keywords"}
1 Mary had a little lamb
2 A little lamb is white
3 Mary ...
0
votes
1
answer
106
views
With pandas, how can I split a JSON into multiple lines, and then split each line into multiple columns?
Given this JSON:
sample = [
{
"id": 1,
"name": "Tiago",
"activities": [
{
"task_id": 1,
...
- 2,398
0
votes
1
answer
30
views
Better way to duplicate rows based on two columns, merging those columns into a single column
I have the following Padas DF...
I.e.
,resultset_id,resultsetrevision_id,injection_id,injection_acqmethod_id,injection_damethod_id
0,8c502f71-9965-43c9-b3be-e7988a2fc89e,023c8953-565e-4953-991a-...
- 2,512
0
votes
2
answers
56
views
How to cancat one column values into another column pandas?
I have a dataframe, it has two columns, one is called 'PTM_loc' and another one is called 'PTM_types'
data['PTM_loc']
0 24
1 24;30
2 22
3 19;20;66
4 16;30;50
data['...
- 41
0
votes
1
answer
81
views
Applying explode on a pandas dataframe results in the error 'ValueError: column must be a scalar'
Here the dataframe before attempting to explode it:
Acc # Match Length Gene Identity (%)
0 CP034360.1 [312] [b4172] [88.462]
1 ...
- 304
-2
votes
1
answer
79
views
How to explode Python Pandas Dataframe and merge strings from other dataframe?
Dataframe1 has a lot of rows and columns of data. One column is Text. Certain rows in Text column have strings and some strings include within the strings this {ExplodeEList2}
How to explode (expand) ...
- 91
0
votes
1
answer
87
views
How to explode Python Pandas Dataframe based on string and criteria
How to turn StringDataFrame:
String
Jon likes {ExplodeAnimals}.
Jon eats {ExplodeFruit}.
Into this:
String
Jon likes Cats.
Jon likes Dogs.
Jon likes Tigers.
Jon likes Llamas.
Jon eats Apples.
Jon eats ...
- 91
0
votes
1
answer
86
views
Explode Array and many sub-array(s) in Pyspark
Explode array of structs to columns in pyspark
Should be transformed to
How can I do this ?
I am unable to resolve from hits arrays to ajax array. Is there any way to explode second array from the ...
0
votes
1
answer
298
views
Explode list of tuples in pandas
I am trying to explode a dataframe where one column consists of tuples. While it works well in cases where the tuples are of type string, but it does not work with when the tuples are integers.
...
0
votes
1
answer
141
views
How to explode in Pandas dataframe rows with comma-separated numbers while leaving unchanged rows with single numbers and no commas?
In a Dataframe, there are two columns (From and To) with rows containing multiple numbers separated by commas and other rows that have only a single number and no commas. How to explode into their own ...
- 91
1
vote
1
answer
174
views
pandas pre-filter an exploded list
I have a list column in a pandas dataframe that needs to be exploded and filtered based on another list.
df_have = pd.DataFrame({'user': ['emp_1', 'emp_2', 'emp_3', 'admin'],
'...
- 135
1
vote
1
answer
459
views
Convert specific list of strings cells into multiple rows and keep the other columns
I have pandas dataframe that looks like this:
label
pred
gt
label1
val1
val11
label2
['str1', str2']
['str1', 'str3', 'str4']
label3
foo
box
And I want to convert label2 row where I have lists of ...
- 985
1
vote
1
answer
2k
views
How to explode pandas dataframe columns and create new column with the key
I have pandas daraframe where I have multiple column which has list of dictionary. Ex:
col1 col2 combine_col1 combine_col2
val1 val2 [{'x1':'v1','x2':'v2'}] [{'x3':'v3','x4':...
- 961
2
votes
3
answers
262
views
Python - How to extract values from different list of dictionaries in the rows of the dataframe
I have data frame as like df:
id features
100 [{'city': 'Rio'}, {'destination': '2'}]
110 [{'city': 'Sao Paulo'}]
135 [{'...
- 568
0
votes
3
answers
481
views
Python: How to explode two columns and set prefix
I have a DataFrame as follows:
time asks bids
2022-01-01 [{'price':'0.99', 'size':'32213'}, {'price':'0.98', 'size':'23483'}...
- 785
0
votes
3
answers
6k
views
input to function explode should be array or map type, not struct
I have the following data.
data = [
[
"2022-12-12",
["IND", "u1", [["auction_1", [[1,20], [2,12]]], ["auction_2", [[1,5], [2,7]]]]]...
- 2,661
0
votes
1
answer
164
views
Pandas Object Column with Mixed Types list and string - dowstream impact on explode
I have a chunk of code that has worked for some time, and now, for some reason my use of .explode has stopped working as expected. After doing some digging, it appears that while the values look like ...
- 40.4k
0
votes
1
answer
150
views
Explode column of objects python pandas dataframe [duplicate]
I am trying to explode a column to create new rows within pandas data frame. What would be the best approach to this?
Input:
SKU
Quantity
Name
YY-123-671
5
drawer
YY-345-111-WH,YY-345-111-RD,YY-345-...
- 827
1
vote
1
answer
89
views
dataframe: parse a column containing list of dicts: Traceback ValueError: cannot reindex on an axis with duplicate labels
I have one column(called 'data') in a dataframe which looks like this, each row has a list of dicts, starting with 2022-01-04, ended today, for example the 1st row is {'value': 18.76, 'date': '2022-01-...
- 477
-1
votes
1
answer
164
views
exploding a multi dictionnary columns
I have a data that contains +15 columns all of them with dictionnary as values. all of the dictionnary has the same keys but different values depending on th column and the key of course. i need to ...
0
votes
1
answer
49
views
transform a list into rown in pandas dataframe
I recuperated my data from TMDB and i've reached to a dataframe that contains:
id (tmdb movie id), nameperson(nameof the each member of the cast), knownfor (movies they participated) and popularity (...
1
vote
1
answer
281
views
How to drop the last row of an exploded dataframe
I have a large dataframe, but to make this easy it looks something like this one below
A B C
0 [a, b, c] 1 22
1 [d, e] 2 45
2 [f, g] 3 32
3 [h, i] 4 64
4 [j, k, ...
0
votes
0
answers
80
views
Explode() function is leading to cannot handle a non-unique multi-index! error
I have a dataframe with columns which has list values:
df = pd.DataFrame({'var1': ['[2.0, 2.0, 2.0, 2.0]', '[1.0, 5.0]', '[2.0, 2.0, 2.0, 2.0]', '[6.0]', '[2.0, 5.0]', '[2.0, 2.0, 2.0, 2.0]', '[3.0]', ...
- 541
0
votes
1
answer
48
views
Explode data in Python [closed]
I have data in the table which I wanted to explode
Input:
| Col 1 | Col 2 | Col 3 |
| -------- | ---------|---------|
| [A,B,C,D]| Value 1 | Value 2 |
Expected Output: The 'col 1' is ...
- 143
1
vote
2
answers
373
views
How to create a formula that will repeat a range by splitting one column and joining with the other column?
I have 2 columns, first name and surname. I want to split a cell that contains multiple values and then combine it with the cell on the right. I have no idea how to do this using a formula, please ...
- 21
1
vote
3
answers
88
views
Pandas explode/split lists and reasign values
I have this code
df = pd.DataFrame({'an':{0: 'f', 1: 'i', 2:'-' , 3:'-' , 4:'f' , 5:'c,f,i,j' , 6:'c,d,e,j'},
'nv':{0: [-1], 1: [-1], 2: ['-'], 3:['-'] , 4:[-2] , 5:[-2,-1,-3,-1] , ...
- 571
0
votes
1
answer
823
views
Explode multiple columns in CSV with varying/unmatching element counts using Pandas
I'm trying to use the explode function in pandas on 2 columns in a CSV that have varying element counts. I understand that one of the limitations of a multi-explode currently is that you can't have ...
- 3
-1
votes
1
answer
188
views
How to explode a struct index in pandas series
I have a pandas Series where the index is a StructType. I want to explode this index into the structfields.
index
value
(2005-07-03, beta0)
0.997160
(2005-07-03, beta1)
0.037629
(2005-07-03, beta2)
0....
0
votes
2
answers
1k
views
Using groupby() on an exploded pandas DataFrame returns a data frame where indeces are repeated but they have different attributes
I am working with a dataset found in kaggle (https://www.kaggle.com/datasets/shivamb/netflix-shows) which has data regarding different productions on Netflix. I am looking to answer the following ...
0
votes
1
answer
425
views
How to convert multiple Json responses into one dataframe in python
I need to create a dataframe for the output generated from the API response response.json() my output looks like this:
{'output':[[2,3,4]]}
{'output':[[6,3,7]]}
I have tried to normalize it by using ...
1
vote
1
answer
316
views
Call pandas explode on one column, and divide the other columns accordingly
I have a dataframe like the one below
d = {"to_explode": [[1, 2, 3], [4, 5], [6, 7, 8, 9]], "numbers": [3, 2, 4]}
df = pd.DataFrame(data=d)
to_explode numbers
0 [1, 2, ...
- 378
0
votes
1
answer
1k
views
Pandas - explode a column and set a specific value to a column for replicated rows
I would like to explode a column Col1 of a dataframe and for all the replicated rows, set a specific value z for a given column Col2.
For example if my dataframe df is:
Col1
Col2
Col3
[A,B,C]
x
y
I ...
- 13