Optimize Django for loops with filter inside the loop

Question

If i have model like this

class User(models.Model):
    name = models.CharField(max_length=255)
    organization = models.ForeignKey(
        "organization.Organization", on_delete=models.CASCADE
    )  
    school = models.ForeignKey(
        school, on_delete=models.CASCADE,)

And then I receive and input of list of lists of user ID. Like this:

list_of_lists_of_user_id = [  [11,21,23] ,  [12, 324] , [909,2,12,444,242] , ....... ]

I want to return the same kind of list but not with just ID, instead I want the other value like "name" and "school" too. So I want to return the list of queryset of User's name and School I would have to loop through like this:

return_list_of_query_set = []
for group in list_of_lists_of_user_id:
            query_set = User.values("name","school").filter(
                id__in=group
            )

            return_list_of_query_set.append(query_set)

How would I optimize this for loop and don't make 1000s queries

sp1rs · Accepted Answer · 2019-07-24 03:55:19Z

3

First flatten your list

>>> import itertools
>>> list_of_lists_of_user_id = [  [11,21,23], [12, 324], [909,2,12,444,242], ... ]
>>> merged = list(itertools.chain(*list2d))
>>> # merged = [11, 21, 23, 12, 324, 909, ...]

Then run your query on the entire merged list.

users = User.objects.filter(
    pk__in=merged
).values("id", "name", "school")

# And then map the `user_ids` with `name` and `school`.
user_mapping = {user['id']: user for user in users}

Once you get the user_mapping of id and user info. Create a new list.

return_list_of_info = []

for user_ids in list_of_lists_of_user_id:
    return_list_of_info.append(
         list(user_mapping.get(user_id) for user_id in user_ids)
    )

answered Jul 24, 2019 at 3:55

sp1rs

7868 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

rabbit.aaron · Accepted Answer · 2019-07-24 03:47:04Z

1

import itertools

User.objects.filter(
    pk__in=itertools.chain.from_iterable( # this will flattern your list
        list_of_lists_of_user_id
    )
).values("name", "school")

doc here: https://docs.python.org/3.6/library/itertools.html#itertools.chain.from_iterable

answered Jul 24, 2019 at 3:47

rabbit.aaron

2,64719 silver badges27 bronze badges

Collectives™ on Stack Overflow

Optimize Django for loops with filter inside the loop

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related