Python Nested List Comprehension with If Else

Question

I was trying to use a list comprehension to replace multiple possible string values in a list of values.

I have a list of column names which are taken from a cursor.description;

['UNIX_Time', 'col1_MCA', 'col2_MCA', 'col3_MCA', 'col1_MCB', 'col2_MCB', 'col3_MCB']

I then have header_replace;

{'MCB': 'SourceA', 'MCA': 'SourceB'}

I would like to replace the string values for header_replace.keys() found within the column names with the values.

I have had to use the following loop;

headers = []
for header in cursor.description:
    replaced = False
    for key in header_replace.keys():
        if key in header[0]:
            headers.append(str.replace(header[0], key, header_replace[key]))
            replaced = True
            break

    if not replaced:
        headers.append(header[0])

Which gives me the correct output;

['UNIX_Time', 'col1_SourceA', 'col2_SourceA', 'col3_SourceA', 'col1_SourceB', 'col2_SourceB', 'col3_SourceB']

I tried using this list comprehension;

[str.replace(i[0],k,header_replace[k]) if k in i[0] else i[0] for k in header_replace.keys() for i in cursor.description]

But it meant that items were duplicated for the unmatched keys and I would get;

['UNIX_Time', 'col1_MCA', 'col2_MCA', 'col3_MCA', 'col1_SourceA', 'col2_SourceA', 'col3_SourceA', 
'UNIX_Time', 'col1_SourceB', 'col2_SourceB', 'col3_SourceB', 'col1_MCB', 'col2_MCB', 'col3_MCB']

But if instead I use;

[str.replace(i[0],k,header_replace[k]) for k in header_replace.keys() for i in cursor.description if k in i[0]]

@Bakuriu fixed syntax

I would get the correct replacement but then loose any items that didn't need to have an string replacement.

['col1_SourceA', 'col2_SourceA', 'col3_SourceA', 'col1_SourceB', 'col2_SourceB', 'col3_SourceB']

Is there a pythonesque way of doing this or am I over stretching list comprehensions? I certainly find them hard to read.

I’d stick with the loops, and avoid the list comprehensions. My rule of thumb is that if you’re doing more than one thing in the comprehension, it’s probably best expanded into a loop with .append. — sneeu
– sneeu, Commented Feb 18, 2014 at 10:20
cursor.description holds a list of strings. header[0] holds a single character. Is that on purpose? — Bach
– Bach, Commented Feb 18, 2014 at 10:21
@HansZauber yes sorry I will use them to write the headers of a csv file using csv.writer(), normally I just do writer.writerow([i[0] for i in cursor.description]) — Dave Anderson
– Dave Anderson, Commented Feb 18, 2014 at 10:21
@sneeu: or a generator that yields the things that should end up in the list, even simpler than creating an empty list and appending to it — RemcoGerlich
– RemcoGerlich, Commented Feb 18, 2014 at 10:37

Bakuriu · Accepted Answer · 2014-02-18 10:33:56Z

8

[str.replace(i[0],k,header_replace[k]) if k in i[0] for k in header_replace.keys() for i in cursor.description]

this is a SyntaxError, because if expressions must contain the else part. You probably meant:

[i[0].replace(k, header_replace[k]) for k in header_replace for i in cursor.description if k in i[0]]

With the if at the end. However I must say that list-comprehension with nested loops aren't usually the way to go. I would use the expanded for loop. In fact I'd improve it removing the replaced flag:

headers = []
for header in cursor.description:
    for key, repl in header_replace.items():
        if key in header[0]:
            headers.append(header[0].replace(key, repl))
            break
    else:
        headers.append(header[0])

The else of the for loop is executed when no break is triggered during the iterations.

I don't understand why in your code you use str.replace(string, substring, replacement) instead of string.replace(substring, replacement). Strings have instance methods, so you them as such and not as if they were static methods of the class.

edited Feb 18, 2014 at 10:33

answered Feb 18, 2014 at 10:25

Bakuriu

103k23 gold badges208 silver badges236 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Dave Anderson Over a year ago

Thanks, didn't know about the else in a for loop that can be really useful. What is the purpose of the _?

Bakuriu Over a year ago

@DaveAnderson I use tuple unpacking to avoid using header[0] all the time. But I just changed it because that would work only if cursor.description was a pair, but since it can have any number of elements I decided to roll that back. The symbol _ doesn't have any particular meaning. It's just often used to mean "a variable that I'm not going to use, but is required". This convention was probably taken from Haskell where it is a special pattern that means "any value matches".

Michael · Accepted Answer · 2014-02-18 10:29:04Z

1

If your data is exactly as you described it, you don't need nested replacements and can boil it down to this line:

l = ['UNIX_Time', 'col1_MCA', 'col2_MCA', 'col3_MCA', 'col1_MCB', 'col2_MCB', 'col3_MCB']
[i.replace('_MC', '_Source')  for i in l]

>>> ['UNIX_Time',
>>>  'col1_SourceA',
>>>  'col2_SourceA',
>>>  'col3_SourceA',
>>>  'col1_SourceB',
>>>  'col2_SourceB',
>>>  'col3_SourceB']

answered Feb 18, 2014 at 10:29

Michael

7,8061 gold badge41 silver badges64 bronze badges

3 Comments

Dave Anderson Over a year ago

Unfortunately the common MC and Source isn't guaranteed it could be anything from M00 through to MFF and the replacement values can be almost anything.

aisbaa Over a year ago

in that case I would advice to write specialized string replace function M_replace(source_string)

Michael Over a year ago

In that case I'd go with regular expressions instead of a huge dictionary of replacements.

georg · Accepted Answer · 2014-02-18 10:21:25Z

0

I guess a function will be more readable:

def repl(key):
    for k, v in header_replace.items():
        if k in key:
            return key.replace(k, v)
    return key

print map(repl, names)

Another (less readable) option:

import re
rx = '|'.join(header_replace)
print [re.sub(rx, lambda m: header_replace[m.group(0)], name) for name in names]

answered Feb 18, 2014 at 10:21

georg

216k57 gold badges324 silver badges401 bronze badges

Collectives™ on Stack Overflow

Python Nested List Comprehension with If Else

3 Answers 3

2 Comments

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related