Linear regression function by hand in Python

Question

I have written the code below to define a function for regression analysis, however, it does not work.

import numpy as np
import pandas as pd

def linear_regress(y, X):
    X_trp = np.transpose(X)
    b = np.linalg.inv(X_trp@X)@X_trp@y
    e = y - X@b
    e_trp = np.transpose(e)
    sigma_sqr = (e_trp@e) / (X.shape[0] - X.shape[1])
    var_b = sigma_sqr@(np.linalg.inv(X_trp@X))
    SE = np.sqrt(var_b)
    z_score = 1.96
    upper = b+SE*z_score
    lower = b-SE*z_score
    CI = [lower, upper]
    results = {"Regression Coefficients": b,
              "Standard Error (SE)": SE,
              "95% Confidence Interval": CI}
    return results

It returns when I define array and run the function:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-17-8307755b9bbc> in <module>
----> 1 linear_regress(indt, dept)

<ipython-input-16-a6336ec1997c> in linear_regress(y, X)
      8     e_trp = np.transpose(e)
      9     sigma_sqr = (e_trp@e) / (X.shape[0] - X.shape[1])
---> 10     var_b = sigma_sqr@(np.linalg.inv(X_trp@X))
     11     SE = np.sqrt(var_b)
     12     z_score = 1.96

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 2 is different from 1)

Can you please help me to figure out what the problem is?

Thanks for your help!

Agni · Accepted Answer · 2021-04-25 18:58:59Z

1

It is difficult to tell what you are doing wrong without some proper idea of your data. But I am guessing you are trying to implement the Normal Equation to get weight. If that is the case, this seems way too complicated for that, here is a simpler approch.

def LinearRegression(X, y):
    X_t = X.T
    theta = np.dot(np.linalg.inv(np.dot(X.T, X)), np.dot(y,X))
    mean_sqr_error = np.mean((y - np.dot(theta,X.T))**2)
    return theta, mean_sqr_error

answered Apr 25, 2021 at 18:58

Agni

2412 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

alperyildirim Over a year ago

Thank you for your help. I'll try what you have written, and I guess I need to do some readings on econometrics with matrix algebra. :)

Collectives™ on Stack Overflow

Linear regression function by hand in Python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related