least square estimation in python using numpy and scipy

Question

Both scipy and numpy has least square estimation but I got slightly confused after reading the documentation.

So, my problem is classical regression where I am trying to find the best matrix transformation between two pairs of values. So something of the form:

Ax = b

Here, I know x and b. A has 9 unique components and x and b are 3D vectors. So, I need at least 3 points. So, 3 non-collinear x and b and I have them and I can create a 3x3 concatenated x and b vectors.

However, from the documentation, I see that it is designed for solving systems when A and b are known. So we solve for x. Assuming A is invertible, this would mean something like:

x = A(-1)b          (-1) indicating the inverse or pseudo inverse.

In my case, the solution becomes

A = b x(-1)

I was wondering if I can still somehow use the built in numpy machinery for my setup.

The pseudo inverse of a vector x is the matrix inv(t(x)*x)*t(x) where t(x) is x transposed and * is matrix product. All of that can be done in python. There's also functions like numpy.linalg.lstsq() and modules like StatsModel. There's also other stack overflow Q&A on this topic in python... chances that this is a duplicate question are pretty high. — Paul
– Paul, Commented Jan 13, 2015 at 12:34

gg349 · Accepted Answer · 2015-01-13 12:42:02Z

3

Look at this:

Ax = b
x^TA^T = b^T

where A^T indicates the transpose of A. Now define the symbols Ap=x^T and Xp = A^T and bp=b^T and your problem becomes:

Ap Xp = bp

that is exactly in the form that you can treat with least squares

answered Jan 13, 2015 at 12:42

gg349

22.8k5 gold badges58 silver badges65 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

least square estimation in python using numpy and scipy

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related