Python itertools.groupby multiple values

Question

I have records returned from a database that look like this:

region       month_taken         total_att num_classes
Colorado    2013-01-01 00:00:00.000 78485   4648
Colorado    2013-02-01 00:00:00.000 71769   4162
Midwest     2013-01-01 00:00:00.000 110508  7101
Midwest     2013-02-01 00:00:00.000 103545  6410

I'm attempting to get them into lists as so:

Total_att:

[{"data": [78485, 71769], "name": "Colorado"}, {"data": [110508, 103545], "name": "Midwest"}]

num_classes:

[{"data": [4648, 4162], "name": "Colorado"}, {"data": [7101, 6410], "name": "Midwest"}]

I discovered itertools.groupby which does what I want, but I'm having difficulty doing it with more than one list of values (for lack of a better term).

totalResults = []            
for key, location in groupby(rows, lambda k: k[0]):
    totalRow = dict()
    totalRow['name'] = key
    totalRow['data'] = [x[2] for x in location]
    totalResults.append(totalRow)

Great, that gets me my total_att list, but then I do an entire additional groupby loop to create the "num_classes" list, which seems ridiculous. I saw this in the docs, but honestly I'm not quite sure what it means or how to handle my problem if I converted it to a list:

The returned group is itself an iterator that shares the underlying iterable with groupby(). Because the source is shared, when the groupby() object is advanced, the previous group is no longer visible. So, if that data is needed later, it should be stored as a list:

So, how can I create my lists without doing multiple for key, location in groupby(rows, lambda k: k[0]):?

I hope that is clear, but am happy to provide more information as necessary.

Note that instead of a lambda function, you should use operator.itemgetter(0). (If you use gnibbler's answer, then you can store the itemgetter into a variable and use it twice to save creating the same thing twice). — Gareth Latty
– Gareth Latty, Commented Mar 16, 2013 at 13:57
You can use itertools.tee to create 2 independent iterators from a single iterable. — nymk
– nymk, Commented Mar 16, 2013 at 14:12

John La Rooy · Accepted Answer · 2013-03-16 14:00:32Z

4

totalResults = [] 
totalClasses = []           
for key, location in groupby(rows, lambda k: k[0]):
    location = list(location)
    totalResults.append(dict(name=key, data=[x[2] for x in location]))
    totalClasses.append(dict(name=key, data=[x[3] for x in location]))

edited Mar 16, 2013 at 14:00

answered Mar 16, 2013 at 13:54

John La Rooy

306k54 gold badges378 silver badges514 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ilmoi Over a year ago

You sir are a lifesaver. I spent ages looking for a solution to this. Thank you so much!

Collectives™ on Stack Overflow

Python itertools.groupby multiple values

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related