Using python scipy.optimize.minimize with function that evaluates both value and gradient [duplicate]

Question

I want to find local minima of a function (of several variables) using python. The set of gradient-based optimization methods described in scipy.optimize.minimize seem like a good way to start.

I can compute the value of the function as well as the gradient of this function. As a matter of fact, when I evaluate the function, I basically get the gradient for free. Is there a way to leverage this property to minimize the number of function calls using scipy.optimize.minimize ?

I'm only referring to methods that use gradient based optimization (say BFGS for instance).

More precisely, how can I plug in a single python function that computes both the value of my mathematical function and the value of its gradient into scipy.optimize.minimize ?

Instead of this :

res = minimize(fun, x0, method='BFGS', jac=grad_fun,options={'disp': True})

I would like something like this :

res = minimize(fun_and_grad, x0, method='BFGS', options={'disp': True})

Thank you !

Sorry ! I think this is a duplicate ! stackoverflow.com/questions/37734430/… I'm not sure what to do about the question though. — G. Fougeron
– G. Fougeron, Commented Feb 16, 2021 at 10:54
Does the duplicate answer your question? Then it can be closed as duplicate. — a_guest
– a_guest, Commented Feb 16, 2021 at 12:14
Yes, although the answer my question has attracted is also relevant and interesting, and not present in the previous one. — G. Fougeron
– G. Fougeron, Commented Feb 16, 2021 at 15:36

a_guest · Accepted Answer · 2021-02-16 12:12:39Z

3

You can use a custom class that caches the gradient and then returns it when requested:

class Wrapper:
    def __init__(self):
        self.cache = {}

    def __call__(self, x, *args):
        fun, grad = compute_something(x)
        self.cache['grad'] = grad
        return fun

    def jac(self, x, *args):
        return self.cache.pop('grad')


wrapper = Wrapper()
res = minimize(wrapper, x0, jac=wrapper.jac, method='BFGS', options={'disp': True})

answered Feb 16, 2021 at 12:12

a_guest

36.7k15 gold badges75 silver badges137 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

G. Fougeron Over a year ago

Yes, caching is definitely clever. Although in this very case, the code would likely break down as many of the algorithms from minimize do not necessarily call gradients and function on the same evaluation points.

Collectives™ on Stack Overflow

Using python scipy.optimize.minimize with function that evaluates both value and gradient [duplicate]

1 Answer 1

1 Comment

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Linked

Related