6

Problem Statement

I would like to apply a list of functions fs = [ f, g, h ] sequentially to a string text=' abCdEf '

Something like f( g( h( text) ) ).

This could easily be accomplished with the following code:

# initial text
text = '  abCDef   '

# list of functions to apply sequentially
fs = [str.rstrip, str.lstrip, str.lower]

for f in fs:
    text = f(text)

# expected result is 'abcdef' with spaces stripped, and all lowercase
print(text)

Using functools.reduce

It seems that functools.reduce should do the job here, since it "consumes" the list of functions at each iteration.

from functools import reduce

# I know `reduce` requires two arguments, but I don't even know
# which one to chose as text of function from the list
reduce(f(text), fs)

# first interaction should call
y = str.rstrip('   abCDef   ')  --> '    abCDef' 

# next iterations fails, because tries to call '   abCDef'() -- as a function

Unfortunately, this code doesn't work, since each iteration returns a string istead of a function, and fails with TypeError : 'str' object is not callable.

QUESTION: Is there any solution using map, reduce or list comprehension to this problem?

Improve this question
2
  • You don't want a list, so why are you even considering list comprehensions? Commented May 12, 2022 at 23:39
  • 2
    This could easily be accomplished with the following code So do that. Seems like you have answered your own question. (If there's some reason why you can't do it this way, say so!) Commented May 12, 2022 at 23:46

4 Answers 4

Reset to default
8

reduce can take three arguments:

reduce(function, iterable, initializer)

What are these three arguments in general?

  • function is a function of two arguments. Let's call these two arguments t and f.
  • the first argument, t, will start as initializer; then will continue as the return value of the previous call of function.
  • the second argument, f, is taken from iterable.

What are these three arguments in our case?

  • the iterable is your list of function;
  • the second argument f is going to be one of the functions;
  • the first argument t must be the text;
  • the initializer must be the initial text;
  • the return of function must be the resulting text;
  • function(t, f) must be f(t).

Finally:

from functools import reduce

# initial text
text = '  abCDef   '

# list of functions to apply sequentially
fs = [str.rstrip, str.lstrip, str.lower]

result = reduce(lambda t,f: f(t), fs, text)

print(repr(result))
# 'abcdef'
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

7

Here's an alternative solution, which allows you to compose any number of functions and save the composed function for reuse:

Quick and dirty

import functools as ft

def compose(*funcs):
    return ft.reduce(lambda f, g: lambda x: f(g(x)), funcs)

Usage:

In [4]: strip_and_lower = compose(str.rstrip, str.lstrip, str.lower)

In [5]: strip_and_lower('  abCDef   ')
Out[5]: 'abcdef'

In [6]: strip_and_lower("  AJWEGIAJWGIAWJWGIWAJ   ")
Out[6]: 'ajwegiajwgiawjwgiwaj'

In [7]: strip_lower_title = compose(str.title, str.lower, str.strip)

In [8]: strip_lower_title("     hello world  ")
Out[8]: 'Hello World'

Note that the order of functions matters; this works just like mathematical function composition, i.e., (f . g . h)(x) = f(g(h(x)) so the functions are applied from right to left.

With type annotations

If you want static analysis to work, you'll need to add type annotations. We can use generics and a closure instead of un-type-able lambdas:

import functools as ft
from typing import Callable, TypeVar


T = TypeVar("T")

Composable = Callable[[T], T]


def compose(*funcs: Composable[T]) -> Composable[T]:
    def compose_f_and_g(f: Composable[T], g: Composable[T]) -> Composable[T]:
        def f_of_g(x: T) -> T:
            return f(g(x))

        return f_of_g

    return ft.reduce(compose_f_and_g, funcs)


strip_and_lower = compose(str.rstrip, str.lstrip, str.lower)

Result in vscode:

function composition properly typed

The editor will also warn you if you attempt to compose incompatible functions:

incompatible function composition

Improve this answer

3 Comments

I couldn't appreciate more your answer. I was thinking in (f . g. h)(x) when I first realized this problem. This lambda² is super. Thanks for sharing the knowledge.
Despíte the elegance, I think that's why reduce and lambda functions are a nightmare to code maintenance. Even today I feel it is difficult to grasp this solution.
I've added a typed version, which may help! It would certainly turn this into a an easily maintainable utility (I agree that nested lambdas are not good for readability).
2

You can try this:

import functools
text = '  abCDef   '
fs = [str.rstrip, str.lstrip, str.lower]
text = functools.reduce(lambda store, func: func(store), fs, text)
print(text)

I think you have misunderstood how reduce works. Reduce reduces an iterable into a single value. The callback function can take two arguments, a store and a element.

The reduce function first creates a store variable. Then, looping through the iterable, it calls the function with the store variable and the current element, and updating the store to the returned value. Finally, the function returns the store value. The final argument is what the store variable starts with.

So in the snippet, it loops through the function array, and calls the respective function on it. The lambda will then return the processed value, updating the store.

Improve this answer

Comments

2

Since you also asked for a map solution, here is one. My values contains single-element iterables, values[0] has the original value and values[i] has the value after applying the first i functions.

text = '  abCDef   '
fs = [str.rstrip, str.lstrip, str.lower]

values = [[text]]
values += map(map, fs, values)
result = next(values[-1])

print(repr(result))  # prints 'abcdef'

But I wouldn't recommend this. I was mostly curious whether I can do it. And now I'll try to think of how to avoid building that auxiliary list.

Ok I found a way without that auxiliary list, taking only O(1) memory. It entangles maps with tee and chain to a self-feeding iterator and uses a deque to drive it and get the final element. Obviously just for fun. Here's output of alternatingly running my above solution and the new solution three times each. It shows the result and the peak memory usage. I repeated the list 10000 times to drive the memory usage up.

result = 'abcdef'   3,126,760 bytes   map_with_black_magic
result = 'abcdef'      12,972 bytes   map_with_blacker_magic
result = 'abcdef'   3,031,048 bytes   map_with_black_magic
result = 'abcdef'      12,476 bytes   map_with_blacker_magic
result = 'abcdef'   3,031,048 bytes   map_with_black_magic
result = 'abcdef'       8,052 bytes   map_with_blacker_magic

Code (Attempt This Online!):

def map_with_black_magic(text, fs):
    values = [[text]]
    values += map(map, fs, values)
    return next(values[-1])

def map_with_blacker_magic(text, fs):
    parts = [[[text]]]
    values = chain.from_iterable(parts)
    it1, it2 = tee(values)
    parts.append(map(list, map(map, fs, it1)))
    return deque(it2, 1)[0][0]

from itertools import tee, chain
from collections import deque
import tracemalloc as tm

text = '  abCDef   '
fs = [str.rstrip, str.lstrip, str.lower] * 10000

for func in [map_with_black_magic, map_with_blacker_magic] * 3:
    tm.start()
    result = func(text, fs)
    memory = tm.get_traced_memory()[1]
    tm.stop()
    print(f'{result = !r}{memory:12,} bytes  ', func.__name__)
Improve this answer

2 Comments

Nine months later: did you manage to avoid building that auxiliary list?
@Stef I don't think so. If I did, I probably would've posted it, no matter how awful. I just thought about it again have a vague idea, I might try again...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.