Create nested loop in a dynamic way in python

Question

I have a dataframe that looks like this: foo = pd.DataFrame({'a': [1,2,3], 'b': [4,5,6], 'c': [6,7,8]}) and a list of columns list_of_columns = ['a','b'] of foo.

The list_of_columns are dynamically selected by the user, so it can be ['a','b'] but it can also be ['a','c'] or ['c'] or ['a','b','c'] etc

I would like for every column in the list_of_columns to create (nested) for loops and query the dataframe in the following way:

In case list_of_columns = ['a','b'] the the loop would be like this:

for a in foo.a.unique():
    for b in foo.b.unique():
        print(foo.query(f'a=={a} and b=={b}'))

In case list_of_columns = ['a'] the the loop would be like this:

for a in foo.a.unique():
    print(foo.query(f'a=={a}'))

In case list_of_columns = ['a','b','c'] the the loop would be like this:

for a in foo.a.unique():
    for b in foo.b.unique():
        for c in foo.c.unique():
            print(foo.query(f'a=={a} and b=={b} and c=={c}'))

Is there a way to programmatically achieve that in python ?

recursive function with a single loop. You remove 'a' from list and call this function recursively with smaller list for every element in 'a'. When the list is empty print — matszwecja
– matszwecja, Commented Jul 20, 2022 at 8:47

Dani Mesejo · Accepted Answer · 2022-07-20 08:59:24Z

2

One approach using itertools.product, to handle the "nested" loops:

import pandas as pd
from itertools import product

foo = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6], 'c': [6, 7, 8]})

list_of_columns = ['a', 'b']
for p in product(*(foo[col].unique() for col in list_of_columns)):
    query = " and ".join(f"{c}=={u}" for c, u in zip(list_of_columns, p))
    print(foo.query(query))
    print("--")

Output

   a  b  c
0  1  4  6
--
Empty DataFrame
Columns: [a, b, c]
Index: []
--
Empty DataFrame
Columns: [a, b, c]
Index: []
--
Empty DataFrame
Columns: [a, b, c]
Index: []
--
   a  b  c
1  2  5  7
--
Empty DataFrame
Columns: [a, b, c]
Index: []
--
Empty DataFrame
Columns: [a, b, c]
Index: []
--
Empty DataFrame
Columns: [a, b, c]
Index: []
--
   a  b  c
2  3  6  8
--

edited Jul 20, 2022 at 8:59

answered Jul 20, 2022 at 8:51

Dani Mesejo

62.2k6 gold badges56 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

mozway · Accepted Answer · 2022-07-20 08:51:47Z

Essentially, it looks like you want to loop over the unique combinations?

But as you then query for valid ones, you obtain a lot of empty DataFrames. If you do not need those, a much simpler and more efficient version would be:

for _,g in foo.groupby(list_of_columns):
    print('---')
    print(g)

output:

---
   a  b  c
0  1  4  6
---
   a  b  c
1  2  5  7
---
   a  b  c
2  3  6  8

In comparison, the output of your nested loop:

---
   a  b  c
0  1  4  6
---
Empty DataFrame
Columns: [a, b, c]
Index: []
---
Empty DataFrame
Columns: [a, b, c]
Index: []
---
Empty DataFrame
Columns: [a, b, c]
Index: []
---
   a  b  c
1  2  5  7
---
Empty DataFrame
Columns: [a, b, c]
Index: []
---
Empty DataFrame
Columns: [a, b, c]
Index: []
---
Empty DataFrame
Columns: [a, b, c]
Index: []
---
   a  b  c
2  3  6  8

Collectives™ on Stack Overflow

Create nested loop in a dynamic way in python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related