Multidimensional non-linear optimization in python

Question

I have several different blocks in my Python-based program, with each block representing a non-linear function f(x, l) with x representing a class containing several different parameters (here labeled as k, l and m). The function is acting on those parameters. Those functions (here called A, B, C etc.) can be chained to each other, such that I can get constructions such as

x_0->A(l_1)->B(l_2)->C(l_3)->x_1

i.e. I take an initial class x_0, put it into A together with a specific value l_1, use the modified class x_A as input in B, obtain a new result class x_B, and so on, until I get x_1 as final result. Of course, I also can re-use those functions, creating structures like

x_0->A(l_1)->B(l_2)->A(l_3)->C(l_4)->x_1

Due to the nonlinearities involved I can not change the order in this arrangement.
Now my aim is to optimize one (or several, depending on my aim) of the parameters in class x. I now could try around by changing the order of the involved function blocks, the amount of the used function blocks and the additional parameters l_n, but this is quite cumbersome due to the nature of brute-forcing. Moreover, as soon as one or more of the input parameters k, l or m changes, I have to repeat that process.

Therefore, are there functions/methods which are already available in Python which I could use here?

A short example could be:

def function_A(x_0, l_0):
    return x_0 * np.exp(-l_0)

def function_B(x_0, l_0):
    return x_0 * np.cos(l_0)

def function_C(x_0, l_0):
    return np.power(x_0, l_0)

def fit_functions(init_val, function_list, l_list):
    cur_val = init_val
    for i, elem in enumerate(function_list):
        cur_val = elem(cur_val, l_list[i])

    return cur_val

init_val = 1
ret_val = fit_functions(init_val, [function_A, function_C, function_B, function_A], [1.1, 2.4, 0.2, 1]) #Example call of fit_functions

Now I would like to maximize ret_val while using between 1 and 5 function blocks, consisting of function_A to function_C. Neither the order nor the amount of blocks is fixed, ideally this should be up to the optimization approach

What optimizations are you trying to do specifically? Your question seems very general and it's hard to answer based only on a description. Could you provide MWE? — fgoudra
– fgoudra, Commented Dec 3, 2020 at 16:23
@fgoudra: I tried to add a short example, even though it is only a very reduced example based on my application. I hope that improves the question — arc_lupus
– arc_lupus, Commented Dec 4, 2020 at 8:48
yes that is much clearer in my opinion! I understand what you're trying to do. — fgoudra
– fgoudra, Commented Dec 4, 2020 at 15:12

Dharman · Accepted Answer · 2020-12-04 17:06:32Z

There are many ways to optimize a function. In your case, I would suggest to recast your problem and optimize independently for each number of functions to use. In other words optimize a first time when using only one function, then 2 and three, etc.

For each of these optimization, optimize the order/type of functions used with initial parameters. I.e.: you want to optimize a new function that takes into argument the sequence of functions to use (in float form since optimization algorithms work in real space usually).

# exemple for 4 functions
def func_to_optimize(f1, f2, f3, f4, l1, l2, l3, l4):
    # get what functions to use since f1, f2,... are floats
    if f1 < 1:
        f1func = function_A
    elif f1 > 1 and f1 < 2:
        f1func = function_B
    # etc...
    # since we want to maximize, you minimize the inverse
    return 1 / function_to_fit([init_val, [f1func, f2func, f3func, f4func], [l1, l2, l3, l4])

And then, you can use scipy.optimize.minimize function to compute the minimal value of this function with the algorithm of choice. This function would be the most "standard" way to do this. You can also try to look for more advanced modules that use other methods to optimize if this does not work (like genetic algorithms and whatnot).

from scipy.optimize import minimize

minimize(func_to_optimize,   # function to optimize
         [0, 0, 0, 0],       # initial guess, here it would correspond to 4 times function_A
         args=[1.1, 2.4, 0.2, 1],  # the other "fixed args" of the function (here your l parameters)
         bounds=[[0, 4]] * 4,  # the function arguments bounds. I.e.: if you have 5 different function to fit, bound each arguments that specify a function between 0 and 4.
         )

Collectives™ on Stack Overflow

Multidimensional non-linear optimization in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related