Linear regression with constrained intercept

Question

I do want to make a constrained linear regression with the intercept value to be like: lowerbound<=intercept<=upperbound.

I do know I can constrain the coefficients with some python libraries but couldn't find one where I can constrain the intercept.

What I want is to get the best solution that fits to my data points with the minimal possible error under the constraint where the intercept is in the range I defined.

How can I do it in python?

@seed the question was changed to ask about a range for the intercept, and no longer asks about a fixed value. — James Phillips
– James Phillips, Commented Nov 2, 2019 at 15:41
Scipy's curve_fit will accept bounds. For example to set a upper bound only on a parameter, that parameter's bound would be [-numpy.inf, upper bound]. Note that if bounds are used for curve_fit, the initial parameter estimates must all be within the specified bounds. — James Phillips
– James Phillips, Commented Nov 2, 2019 at 15:45

James Phillips · Accepted Answer · 2019-11-04 16:25:55Z

1

Here is an example of using curve_fit with parameter bounds. In this example parameter "a" is unbounded, parameter "b" is bounded and the fitted value is within those bounds, and parameter "c" is bounded and the fitted value is at a bound.

import numpy
import matplotlib
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit

xData = numpy.array([5.0, 6.1, 7.2, 8.3, 9.4])
yData = numpy.array([ 10.0,  18.4,  20.8,  23.2,  35.0])


def standardFunc(data, a, b, c):
    return a * data + b * data**2 + c


# some initial parameter values - must be within bounds
initialParameters = numpy.array([1.0, 1.0, 1.0])

# bounds on parameters - initial parameters must be within these
lowerBounds = (-numpy.Inf, -100.0, -5.0)
upperBounds = (numpy.Inf, 100.0, 5.0)
parameterBounds = [lowerBounds, upperBounds]

fittedParameters, pcov = curve_fit(standardFunc, xData, yData, initialParameters, bounds = parameterBounds)

# values for display of fitted function
a, b, c = fittedParameters

# for plotting the fitting results
xPlotData = numpy.linspace(min(xData), max(xData), 50)
y_plot = standardFunc(xPlotData, a, b, c)

plt.plot(xData, yData, 'D') # plot the raw data as a scatterplot
plt.plot(xPlotData, y_plot) # plot the equation using the fitted parameters
plt.show()

print('fitted parameters:', fittedParameters)

UPDATE: per the comments, here is a multivariate fitting example:

import numpy
from scipy.optimize import curve_fit

X1 = (-1.0, -2.2, -3.3, -4.4, -5.5, -6.7)
X2 = (21.0, 22.2, 23.3, 24.4, 25.5, 26.7)
X3 = (51.0, 52.2, 53.3, 54.4, 55.5, 56.7)
all_X_data = numpy.array([X1, X2, X3])

Y = (11.1, 12.1, 13.1, 14.1, 15.1, 16.1)


# function to be fitted
def modelFunction(data, a, b, c, offset):
    f = (data[0] * a) + (data[1] * b) + (data[2] * c) + offset
    return f


# some initial parameter values
# these might be estimated from scatterplots
initialParameters = (1.0, 1.0, 1.0, 1.0)

# perform the fit
fittedParameters, pcov = curve_fit(modelFunction, all_X_data, Y, initialParameters)

print('a, b, c, offset:', fittedParameters)

edited Nov 4, 2019 at 16:25

answered Nov 2, 2019 at 20:22

James Phillips

4,6673 gold badges16 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Henry A. Over a year ago

@jamesPhililips many thanks man, this might work for 2-dimensional regression, but I do have a multivariate linear regression, I was in fact using the linearregression() and it works just how I expected but it doesn't allowed me to set up constraints on the intercept. Is it there a way for when several independent variables are required in the function?.

James Phillips Over a year ago

curve_fit can be used with multivariate data, I can give an example if it might be useful to you.

Henry A. Over a year ago

tht will be useful

James Phillips Over a year ago

See the section marked UPDATE in my answer for the multivariate fitting example.

Collectives™ on Stack Overflow

Linear regression with constrained intercept

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related