Implementing AIC score for multiple linear regression manually

Question

I have implemented a multiple linear regression class by hand and right now I am working on the metrics methods. I have tried to calculate the AIC and BIC scores manually, but the results weren't correct. The reason is that I didn't use the Log Likelihood function but went with the SSE approach. Could you please suggest me how to change the implementation to compute the full AIC and BIC scores?

Here is how my method looks like right now:

  def AIC_BIC(self, actual = None, pred = None):
    if actual is None:
      actual = self.response
    if pred is None:
      pred = self.response_pred

    n = len(actual)
    k = self.num_features

    residual = np.subtract(pred, actual)
    RSS = np.sum(np.power(residual, 2))

    AIC = n * np.log(RSS / n) + 2 * k
    BIC = n * np.log(RSS / n) + k * np.log(n)

    return (AIC, BIC)

And please try to give me a manual approach and not a library call. Thanks!

Please review stackoverflow.com/questions/7458391/… as my old notes say that it has discussion of the AIC calculation for multiple regression. — James Phillips
– James Phillips, Commented Nov 21, 2019 at 12:37

OwlOnMoon · Accepted Answer · 2019-11-21 22:43:43Z

1

I managed to solve the issue with the help of @James. Here is how my new implementation looks like:

 def LogLikelihood(self):
     n = self.num_obs
     k = self.num_features
     residuals = self.residuals

     ll = -(n * 1/2) * (1 + np.log(2 * np.pi)) - (n / 2) * np.log(residuals.dot(residuals) / n)

     return ll

  def AIC_BIC(self):
    ll = self.LogLikelihood()
    n = self.num_obs
    k = self.num_features + 1

    AIC = (-2 * ll) + (2 * k)
    BIC = (-2 * ll) + (k * np.log(n))

    return AIC, BIC

I implemented a log-likelihood calculation and used it in the formula which can be found on Wikipedia.

answered Nov 21, 2019 at 22:43

OwlOnMoon

993 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Implementing AIC score for multiple linear regression manually

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related