1

I'm trying to create a composite covariance function to model my data. Specifically I want to create a kernel which is weighted between @covSEard & @covRQard. For ex: I want to give a 30% weight to @covSEard & 70% to @covRQard, something like 0.3*@covSEard + 0.7*@covRQard

I tried the following options.

Option 1 - Using the @covSum & @covProd functionalities

input_dimensions = 4; % number of input dimensions
seed = 1234; % seed for reproducibility
rng(seed);
X = rand(100, input_dimensions); % sample input data generation
y = rand(100, 1); % sample output data generation

addpath(genpath(PATH_TO_GPML_TOOLBOX)); % adding gpml toolbox to the path

meanFunc = {@meanZero}; % using a zero mean
meanHyp = {[]}; % zero mean has no hyper parameters

kernel_weight = 0.3; % weight of covSEard kernel, 0.7 will be the weight for covRQard kernel

% defining the covariance function as a weighted sum of covSEard & covRQard
covFunc = {'covSum', {{'covProd', {'covConst','covSEard'}}, {'covProd', {'covConst','covRQard'}}}};

% variables to define the hyperparameters for the above kernel
sf=2; L=rand(input_dimensions,1); al=2; 

% Covariance function for a constant function. The covariance function is parameterized as:
% k(x,z) = sf^2
% 
% The scalar hyperparameter is:
% hyp = [ log(sf) ]
covHyp = {log([sqrt(kernel_weight); L; sf; sqrt(1-kernel_weight); L; sf; al])};

likFunc = {@likGauss}; % Using a gaussian likelihood
likHyp = {-1}; % Likelihood hyper parameter initialization

infMethod= @infGaussLik; % Using Gaussian inference
iters = -300; % Number of iterations for Bayesian Optimization


% Defining the hyper parameter struct
hyp.lik = cell2mat(likHyp(1));
hyp.cov = cell2mat(covHyp(1));
hyp.mean = cell2mat(meanHyp(1));

% Defining mean, covariance and likelihood functions
mF = meanFunc{1,1}; cF=covFunc; lF=likFunc{1,1};
hyp2vfe = minimize(hyp, @gp, iters, infMethod, mF, cF, lF, X, y); % Optimization of hyperparameters / Training
[nll, ~] = gp(hyp2vfe, infMethod, mF, cF, lF, X, y); % Negative Log Likelihood calculation

Here, I'm trying to use the @covConst kernel which has only the signal variance hyper-parameter which I'm forcing to be equal to the weight (Ex: 0.3 for @covSEard & 0.7 for @covRQard in this case.
However, when I try to optimize the above kernel's hyper parameters, even the weights (which are actually the hyper parameters of the @covConst kernel are also getting modified.

Option 2 - Using the @covSum functionality and repeating each kernel n times based on weight

For example, if I want to give 1:2 weights to @covSEard & @covRQard respectively, I would do the following

replace the following lines in the above code

covFunc = {'covSum', {{'covProd', {'covConst','covSEard'}}, {'covProd', {'covConst','covRQard'}}}};
with
covfunc = {'covSum', {'covSEard', 'covRQard', 'covRQard'}};

&

covHyp = {log([sqrt(kernel_weight); L; sf; sqrt(1-kernel_weight); L; sf; al])};
with
covHyp = {log([L; sf; L; sf; al; L; sf; al])};

However, in this case the number of hyper parameters would increase linearly and also I'm not sure if this is the right way to do things.

I want to know what is the correct way to create such a covariance function in GPML. Please suggest.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.