1

As a way to learn Python/NumPy I’m programming my SAS IML dissertation of 20 years ago in Python 2.7.11. The method involves monte carlo simulations with multiple loops inside the outer loop. I’ve been successful with a simple example. I’m at the point where I need to accumulate statistics from each iteration and I need guidance because I’m a little (or a lot) confused about how and when to use lists, arrays, and matrices and how to convert one to the other despite repeatedly looking a docs and examples.

If I was printing a report, I want a table looking like this (delimiter can be a space)

MSEa     Ca MSEb     Cb 
1421 7  1184     3
 925     2  1077 4  

so I can choose the smallest MSE and C pairing in each MSE column.

What I have is

MSEV
[matrix([[ 1421]]), 7, matrix([[ 1184]]), 3, matrix([[ 925]]), 2, matrix([[ 1077]]), 4]
type(MSEPCV)
<type 'list'>

MSE is a scalar from matrix math, MSEV is initiated as MSEV = [] {type(MSEPCV) }, MSEV is created from MSE {type(MSE) } and COUNT {type(COUNT) }

I thought it would be easiest to make everything a list and tried tolist() but the squackets remain.

2
  • A list does have square brackets [] - that's just what a list is. I'm not entirely certain that you need a matrix, but I'm unfamiliar with your problem space Commented Aug 4, 2016 at 20:16
  • probably better not use mix arrays and matrices as they have different semantics. Commented Aug 4, 2016 at 20:36

3 Answers 3

1

You have somehow ended up with a list containing a mixture of single-element matrices and ordinary numbers. If you cannot avoid this, then you could always clean up the list for printing using something like the following:

def extract_number(x):
    try:
        return x.item(0)    # returns first item in x, if x is a matrix
    except AttributeError:
        return x            # if x is not a matrix or array, just return x

print([extract_number(x) for x in MSEV])

This will output the list in the standard way that Python displays lists

[1421, 7, 1184, 3, 925, 2, 1077, 4]

To display it more nicely you can look into string formatting and do something like this:

print("""MSEa     Ca MSEb     Cb 
{:4}{:6}{:6}{:7}
{:4}{:6}{:6}{:7}""".format(*msev))
Sign up to request clarification or add additional context in comments.

Comments

0

I’m a little (or a lot) confused about how and when to use lists, arrays, and matrices and how to convert one to the other

Since you tagged it numpy, I am going to assume you mean numpy.array and numpy.matrix. Then, in short, when to use

  • matrices: basically, never. np.matrix is an endless source of confusion. Better use arrays, use dot for matrix multiplication, or @ operator if you can use python 3.5+ only.

  • arrays: use them when you know the dimensions and sizes (shape, in numpy parlance) in advance.

  • lists: use lists when you need to append/remove elements.

As to how to convert things: given a list, lst, np.asarray(lst) is a numpy array. Given an array, arr, arr.tolist() is a list. Given a matrix, np.asarray conveerts it into an array, and if your matrix is 1-by-1, m[0, 0] is a scalar.

Comments

0

I'm not sure where your matrix comes from, and why the mix of matrix and numbers, but I'll try to process it.

For ease of copy-n-paste I'll define matrix (I've loaded numpy as np). I'm working in a Ipython session:

In [373]: matrix=np.matrix

In [375]: alist=[matrix([[ 1421]]), 7, matrix([[ 1184]]), 3, matrix([[ 925]]), 2, matrix([[ 1077]]), 4]

In [376]: alist
Out[376]: 
[matrix([[1421]]),
 7,
 matrix([[1184]]),
 3,
 matrix([[925]]),
 2,
 matrix([[1077]]),
 4]

I'll run it through a list comprehension with a ifelse expression that extracts the element from the matrices. I could also have defined a simple helper function as Stuart did:

In [379]: newlist=[x[0,0] if isinstance(x,np.matrix) else x for x in alist]
In [380]: newlist
Out[380]: [1421, 7, 1184, 3, 925, 2, 1077, 4]

now turn it into an array - and use reshape to make it 2 rows.

In [381]: Marray=np.array(newlist).reshape(2,-1)
In [382]: Marray
Out[382]: 
array([[1421,    7, 1184,    3],
       [ 925,    2, 1077,    4]])

np.savetxt is the numpy function to write csv style files:

In [386]: np.savetxt('test.txt',Marray)
In [387]: cat test.txt
1.421000000000000000e+03 7.000000000000000000e+00 1.184000000000000000e+03 3.000000000000000000e+00
9.250000000000000000e+02 2.000000000000000000e+00 1.077000000000000000e+03 4.000000000000000000e+00

Oops, default format is float; change that to integer:

In [388]: np.savetxt('test.txt',Marray, '%d')

and look at the resulting file (just like doing cat in a linux shell)

In [389]: cat test.txt
1421 7 1184 3
925 2 1077 4

and with a header line:

In [392]: np.savetxt('test.txt',Marray, '%d',header='MSEa Ca MSEb Cb')

In [393]: cat test.txt
# MSEa Ca MSEb Cb
1421 7 1184 3
925 2 1077 4

Format can be refined, but that gives the idea.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.