Nested list comprehension with split() in python

Question

Long story short, I am trying to work with some crazy data I collected yesterday. A quick nested for loop would do this real easy but right now, I just want to get my nested list comprehension to work. There are a bunch of post on this subject and I am not sure whether it is just me or the because of Friday evening, I can't get it to work. My list comphrension is in format:

[ [str(j[0], sanitize(j[1]) for j in i.split(',')] for i in b]

where b is ['string, float\t\n', 'string, float\t\n'] or b might be a file object or whatever, doesn't matter a heck lot.

I could simply do something like

for i in b:
    out.append(str(j[0]), sanitize(j[1])

But, it should be possible by list comprehension. IT SHOULD!!! And, somehow, I am not getting it right.

Now, on to what I have done so far...

>>> b
['08:54:55, 0.031250\n', '08:55:57, 0.031250\n']

>>> [i.split(',') for i in b]
[['08:54:55', ' 0.031250\n'], ['08:55:57', ' 0.031250\n']]

What I want:

[['08:54:55', 0.031250], ['08:55:57', 0.031250]]

A multiple list comprehension should do the trick:

Something like:

[ [j[0], j[1]] for j in i.split(',') for i in b] # the cleaning/str/float would be something like: str(j[0]), float(j[1])

BUT, this is what I get.

>>> [ [j[0], j[1]] for j in i.split(',') for i in b]
[['0', '8'], ['0', '8'], [' ', '0'], [' ', '0']]
>>> 
>>> [ [j[1]] for j in i.split(',') for i in b]
[['8'], ['8'], ['0'], ['0']]

Any attempt to work on j[0] or j[1] does not pan out.

To figure out what was going on, I did this:

>>> [ [ pprint(type(j)) for j in i.split(',')] for i in b]
<type 'str'>
<type 'str'>
<type 'str'>
<type 'str'>
[[None, None], [None, None]]

If I don't mess with j[0] or j[1],

>>> [ [ j for j in i.split(',')] for i in b]
[['08:54:55', ' 0.031250\n'], ['08:55:57', ' 0.031250\n']]

The example with pprint(type(j)) means even if I were to write an external function, it would not do the trick. I would simply be getting an empty list back from the list comprehension.

What am I doing wrong? A lambda might do the trick?

AChampion · Accepted Answer · 2016-11-20 05:02:43Z

4

You can create a second for comprehension to extra the split():

>>> [[t, sanitize(v)] for i in b for t, v in [i.split(',')]]
[['08:54:55', 0.031250], ['08:55:57', 0.031250]]

answered Nov 20, 2016 at 5:02

AChampion

30.5k4 gold badges63 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

James T Over a year ago

That works great. Wrapping i.split(',') in [ ] appears to do the job.

Ry- · Accepted Answer · 2016-11-20 05:02:31Z

1

If you write out a proper, working loop:

out = []

for i in b:
    j = i.split(',')
    out.append((j[0], sanitize(j[1])))

you’ll notice that you need somewhere to store the result of i.split(',') to avoid repeating it. This isn’t possible inline in one list comprehension. You can write a function:

def convert_thing(thing):
    time, value = thing.split(',')
    return time, sanitize(value)

out = [convert_thing(x) for x in b]

or generate intermediate values (don’t do this):

out = [(time, sanitize(value)) for time, value in (x.split(',') for x in b)]
# this bit is a generator expression              ^^^^^^^^^^^^^^^^^^^^^^^^^

answered Nov 20, 2016 at 5:02

Ry-♦

226k56 gold badges496 silver badges504 bronze badges

5 Comments

James T Over a year ago

After going through AChampion's answer (see below), it seems that you can actually store result when doing list comprehension. Instead of a simple, a.split(','), I had to do [a.split(',')]. Took me a while to realize that.

James T Over a year ago

In [24]: [ [j[0], j[1]] for j in [i.split(',')] for i in b] Out[24]: [['08:55:57', ' 0.03\n'], ['08:55:57', ' 0.03\n']] In [25]: [ [j[0], float(j[1])] for j in [i.split(',')] for i in b] Out[25]: [['08:55:57', 0.03], ['08:55:57', 0.03]]

Ry- Over a year ago

@JamesT: That’s just a two-loop list comprehension, about the same as the snippet in my answer annotated with “don’t do this”. The “don’t do this” applies equally to it.

James T Over a year ago

ah just realized it. And for reference, the one I typed there does not work - throws a NameError. It was working only because ipython was saving a previous value of i, @AChampion works though. However, when you said "don't do this", what is the reason? Other then it being exceedingly difficult to read for most part. The time complexity is O(m*n) either way, not sure about the memory. Haven't done actual timeit. Is it more to do with the python programming conventions (preserving simplicity aside) then any inherent issue?

Ry- Over a year ago

@JamesT: It’s because it’s hard to read. Anyone who looks at it is going to do a double-take and wonder if that’s what you really meant.

Collectives™ on Stack Overflow

Nested list comprehension with split() in python

2 Answers 2

1 Comment

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related