Seems like a basic question, but I need to use feature scaling (take each feature value, subtract the mean then divide by the standard deviation) in my implementation of linear regression with gradient descent. After I'm finished, I'd like the weights and regression line rescaled to the original data. I'm only using one feature, plus the y-intercept term. How would I change the weights, after I get them using the scaled data, so that they apply to the original unscaled data?
2
-
1And your question is ...?alko– alko2014-01-16 17:39:20 +00:00Commented Jan 16, 2014 at 17:39
-
Edited to make question clearer.Cartesian Theater– Cartesian Theater2014-01-16 17:54:11 +00:00Commented Jan 16, 2014 at 17:54
Add a comment
|
2 Answers
Suppose your regression is y = W*x + b with x the scaled data, with the original data it is
y = W/std * x0 + b - u/std * W
where u and std are mean value and standard deviation of x0. Yet I don't think you need to transform back the data. Just use the same u and std to scale the new test data.
3 Comments
Mohd Shahril
@lennon310, sorry for troubling, but can you show me the intuitive on how above equation can unscale the data back?
p.mathew13
@lennon310 Could you please tell how it would work on new test data when using same u and std.
ptyshevs
@p.mathew13 when you rescale coefficients back, you no longer need mu and std and just use unscaled data to get the prediction.
If your feature is x, your scaled feature is z = (x - μ) / σ. The result of the linear regression provides you w and b in the equation y = w z + b.
If you want the un-scaled values of w and b, you just need to replace z with (x - μ) / σ:
y = w z + b
y = w (x - μ) / σ + b
y = w/σ x - w μ/σ + b
y = [w/σ] x + [- w μ/σ + b]
The parameters m and q of your un-scaled linear regression y = m x + q are
m = w/σ
q = b - w μ/σ
If you want to scale both the features and the target, you can find the un-scaled parameters from the equation
(y - μ_y) / σ_y = w (x - μ_x) / σ_x + b
m = w σ_y / σ_x
q = b σ_y + μ_y / σ_y - w μ_x σ_y / σ_x
