Is the given code for gradient descent updating the parameters sequentially or simultaneously?

Question

I'm new to machine learning and I have been learning gradient descent algorithm. I believe this code uses simultaneous update, even though it looks like sequential update. Since the values of partial derivative are calculated before updating either w or b, that is, from the original w and b, the algorithm thus applied on individual w,b is being applied from original values. Am I wrong?


dj_dw=((w*x[i]+b-y[i])*x[i])/m
dj_db=(w*x[i]+b-y[i])/m
w=w-a*dj_dw
b=b-a*dj_db

The language is python3.

x and y are training set.

w and b are parameters that the algorithm is being applied on. I'm using gradient descent algorithm for linear regression.

dj_dw is the partial differentiation of squared mean error cost function with respect to w. Same goes for dj_db.

Apologies for any errors, I'm new.

I tried cross checking using gemini and chatgpt and they said it was sequential, hence the confusion

I don't think this is answerable with so little code, you did not even mention what programming language is this. — Dr. Snoopy
– Dr. Snoopy, Commented Jun 5, 2024 at 15:49
Still it is too vague, I tend to say it is parallel but all depends on what the variables mean and how they were computed. — Dr. Snoopy
– Dr. Snoopy, Commented Jun 5, 2024 at 18:32

Mustufa Abbas · Accepted Answer · 2024-10-02 13:52:29Z

0

It's simultaneous because both w and b are updated before the next iteration, meaning w and b both have updated values when the next differentiation occurs.

answered Oct 2, 2024 at 13:52

Mustufa Abbas

112 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Is the given code for gradient descent updating the parameters sequentially or simultaneously?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related