Appending to a specific row in a numpy array

Question

I am trying to append a number to a particular row of a 2D numpy array. I did the following but it doesn't work right:

onspiketimes=np.array([[] for i in range(9)])
neurs = [3, 4, 8, 2]
onspiketimes[neurs]=2
>>> onspiketimes
array([], shape=(9, 0), dtype=float64)

As you can see that nothing got appended,onspiketimes still remains empty. However I could do this with list of lists as below:

>>> onspiketimes= [[] for i in range(9)]
>>> for items in neurs:
        onspiketimes[items].append(2)
>>> for items in neurs:
        onspiketimes[items].append(3)
>>> onspiketimes
[[], [], [2, 3], [2, 3], [2, 3], [], [], [], [2, 3]]

This is a contrived example, I can't really compare numpy and lists here but in my real applications for loops will really effect the speedup. I was hoping to solve this using array slicing in numpy.

You don't want to deal with jagged numpy arrays. What are you trying to do with this? — user3483203
– user3483203, Commented Aug 10, 2018 at 23:10
I am trying to store neurons firing times with list of lists or jagged numpy arrays, each sub list denotes a neuron and elements within that sub list denote that particular neuron's firing times. — pythonic
– pythonic, Commented Aug 10, 2018 at 23:13
do you thin I could use map here to append to a specific list if i just stick with list of lists? — pythonic
– pythonic, Commented Aug 10, 2018 at 23:16
If you want to use numpy, I would initialize the array onspiketimes to something larger than you would anticipate needing. Then you can fill it in as you go. Ideally, you would add all the data at once rather than in increments if you want to speed up the process. — busybear
– busybear, Commented Aug 10, 2018 at 23:19
What is the actual shape of your real data? How many total rows, and what are the min/max/mean numbers of elements per row? — John Zwinck
– John Zwinck, Commented Aug 10, 2018 at 23:36

hpaulj · Accepted Answer · 2018-08-11 01:05:24Z

To do the same thing with an array, you have to make an object dtype array, and initialize it with [].

In [580]: alist = [[] for _ in range(10)]
In [581]: arr = np.empty(10, object)
In [582]: arr[:]= alist
In [583]: arr
Out[583]: 
array([list([]), list([]), list([]), list([]), list([]), list([]),
       list([]), list([]), list([]), list([])], dtype=object)
In [584]: neurs = [3, 4, 8, 2]
In [585]: for item in neurs:
     ...:     arr[item].append(2)
     ...:     
In [586]: for item in neurs:
     ...:     arr[item].append(3)
     ...:     
In [587]: 
In [587]: arr
Out[587]: 
array([list([]), list([]), list([2, 3]), list([2, 3]), list([2, 3]),
       list([]), list([]), list([]), list([2, 3]), list([])], dtype=object)

In [588]: arr.tolist()
Out[588]: [[], [], [2, 3], [2, 3], [2, 3], [], [], [], [2, 3], []]

Iterating on an object array is a little slower than iterating on a list (but faster than iterating on the rows of a 2d array.

Doing something similar with a 2d numeric array:

In [589]: arr2 = np.zeros((10,5),int)
In [590]: arr2[neurs, 0] = 2
In [591]: arr2[neurs, 1] = 3
In [592]: arr2
Out[592]: 
array([[0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [2, 3, 0, 0, 0],
       [2, 3, 0, 0, 0],
       [2, 3, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [2, 3, 0, 0, 0],
       [0, 0, 0, 0, 0]])

This will be faster, but the result isn't a ragged array.

What you are doing with lists can't be done (exactly) faster with arrays. It doesn't fit the multidimensional array model that has many fast compiled building blocks.

You ask about map. That is similar to a list comprehension, in function and speed. It produces a new list. It shouldn't be used for side-effects, like appending to values to an existing list.

Daniel Ong · Accepted Answer · 2018-08-11 01:10:39Z

1

NumPy arrays are stored in a contiguous block of memory so appending new elements into an array the way you've described would require allocating a new, bigger block of memory, copying over the existing elements in the array, and constructing the new, modified array. This is inefficient and undesirable. For what you're trying to do in your example, a list of lists is much better than a NumPy array.

answered Aug 11, 2018 at 1:10

Daniel Ong

2831 silver badge12 bronze badges

Collectives™ on Stack Overflow

Appending to a specific row in a numpy array

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related