How can I precompute constants for frozen random variable in SciPy?

Question

I have defined a custom continuous distribution, with implemented _pdf and _cdf methods. For these I need to compute expensive constants (given the parameters), namely the normalization constant for the PDF and the integration constant for the CDF. These constants are computed every time any of the functions of the (frozen) random variable are evaluated, which takes a lot of time.

I was wondering if there is a way to precompute or cache/memoize these expensive constant for frozen random variables in SciPy?

Here is some minimal code of an example distribution with parameters a and b defined by non-normalized PDF f(x, a, b) and its antiderivative F(x, a, b). The expensive functions are the norm N(a,b) and the integration constant C(a,b):

from scipy.stats import rv_continuous


class Example_gen(rv_continuous):

    def _norm(self, a, b):
        """Expensive function"""
        return N(a, b)

    def _C(self, a, b):
        """Expensive function"""
        return C(a, b)

    def _pdf(self, x, a, b):
        return f(x, a, b) / self._norm(a, b)

    def _cdf(self, x, a, b):
        return (F(x, a, b) + self._C(a, b)) / self._norm(a, b)


Example = Example_gen()

I'm too dull but I can't help if you don't show at least some code. — gboffi
– gboffi, Commented Dec 22, 2023 at 8:35

Severin Pappadeux · Accepted Answer · 2023-12-24 03:50:26Z

1

How about caching them locally? You could have a dictionary, with key being tuple of values.

Along the lines (untested)

class Example_gen(rv_continuous):

    _n_cache = dict()
    _C_cache = dict()

    def _norm(self, a, b):
        """Expensive function"""
        key = (round(a,5), round(b,5))
        v = _n_cache.get(key) 
        if v is None:
            v = N(a, b)
            _n_cache[key] = v              
        
        return v

    def _C(self, a, b):
        """Expensive function"""
        key = (round(a,5), round(b,5))
        v = _C_cache.get(key) 
        if v is None:
            v = C(a, b)
            _C_cache[key] = v              
        
        return v

You could initialize cache dictionaries by, say, reading some JSON files or pickles.

answered Dec 24, 2023 at 3:50

Severin Pappadeux

20.4k4 gold badges45 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

ev-br · Accepted Answer · 2023-12-22 21:09:49Z

0

There isn't. Of course, if the specific distribution you think about defines some internal functions, you came monkey-patch them to return precomputed values. But this is certainly not supported by the framework, and you're on your own if you do that.

answered Dec 22, 2023 at 21:09

ev-br

26.3k9 gold badges68 silver badges84 bronze badges

Collectives™ on Stack Overflow

How can I precompute constants for frozen random variable in SciPy?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related