Is there a way to retrieve the weights from a GPflow GPR model?

Question

I do not necessarily need the explicit weights. However, I have two issues that may be solved using the weights:

I would like to compile and send a trained model to a third party. I would like to do this without sending the training data and without the third party having access to the training data.
I would like to be able to predict new mean values without calculating new variances. Currently predict_f calculates both the mean and the variance, but I only use the mean. I believe I could speed up my prediction significantly if I didn't calculate the variance.

I could resolve both of these issues if I could retrieve the weights from the GPR model after training. However, if it is possible to resolve these tasks without ever dealing with explicit weights, that would be even better.

STJ · Accepted Answer · 2021-10-27 19:15:08Z

1

It's not entirely clear what you mean by "explicit weights", but if you mean alpha = Kxx^{-1} y where Kxx is the evaluation of k(x,x') and y is the vector of observation targets, then you can get that by using the Posterior object (see https://github.com/GPflow/GPflow/blob/develop/gpflow/posteriors.py), which you get by calling posterior = model.posterior(). You can then access posterior.alpha.

Re 1.: However, for predictions you still need to be able to compute Kzx the covariance between new test points and the training points, so you will also need to provide the training locations and kernel hyperparameters.

This also means that you cannot rely on this to keep your training data secret, as the third party could simply compute Kxx instead of Kzx and then get back y = Kxx @ alpha. You can avoid sharing exact (x,y) training set pairs by using a sparse approximation (this would remove "individual identifiability" at least). But I still wouldn't rely on it for privacy.

Re 2.: The Posterior object already provides much faster predictions; if you only ask for full_cov=False (marginal variances, the default), then you're at worst about a factor ~3 or so slower than predicting just the mean (in practice, I would guesstimate less than 1.5x as slow). As of GPflow 2.3.0, there is no implementation within GPflow of predicting the mean only.

answered Oct 27, 2021 at 19:15

STJ

1,54710 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

partyphysics Over a year ago

Thank you, the alphas* are exactly what I wanted. However, when I try this with GPR it returns only the y values. This code seems to be the culprit: github.com/GPflow/GPflow/blob/…

partyphysics Over a year ago

The covariance matrix needs to be inverted and multiplied by y

STJ Over a year ago

@partyphysics good catch, that's weird; can you please open an issue on Github?

Collectives™ on Stack Overflow

Is there a way to retrieve the weights from a GPflow GPR model?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related