How to merge numpy arrays stored in a nested list in python

Question

I have a list of numpy arrays and want to split it into chunks based on the length of array. This is my list:

import numpy as np
nums=[[np.array([[1.]]), np.array([[2.]])],\
      [np.array([[3.]]), np.array([[5.]])],\
      [np.array([[3.], [3.]]), np.array([[8.], [9.]])],\
      [np.array([[8.], [9.]]), np.array([[1.], [2.]])],\
      [np.array([[5.]]), np.array([[1.]])]]

Each sublist has always two arrays with similar length. First two sublists should be merged because the length of their arrays is the same (1). I take only unique arrays. The third and fourth sublists should be merged because of the same length (2). These two sublists have a common array (np.array([[8.], [9.]])) and I only take unique arrays. Last sublist is also another split. In reality I have several sublist but I want to used the same rule for merging them. Finally, I want to get my list as:

arri=[[np.array([[1.]]), np.array([[2.]]),\
       np.array([[3.]]), np.array([[5.]])],\ # first chunk
      [np.array([[3.], [3.]]), np.array([[8.], [9.]]),\
       np.array([[1.], [2.]])],\ # second chunk
      [np.array([[5.]]), np.array([[1.]])]] # third chunk

I tried the following code but it was not successful:

arri=[]
for i in range (len (nums)-1):
    if (len(nums[i][0]) == len(nums[i+1][0]) and
        any((nums[i+1][0] == x).all() for x in nums[i])):
        arri.append ([nums[i], nums[i+1]])

In advance, I do appreciate any help.

Dear @Mateen Ulhaq, Thanks for your attention. I have written it in my question. I want to merge sublists which the length of their arrays is the same. I start from the first sublist and check the length its array. then I go to the next sublist, if the lengths of both are the same, I merge uniques arrays. Then I go for the third, if the length is the same as previous, i merged it to them. If not, I go forwards and check the length of next one and so on, — Link_tester
– Link_tester, Commented May 14, 2021 at 12:11
Dear @Steve, I have written the expected output as arri in my question. It has three sublists. First sublist is created by merging unique arrays of first two sublists of input. Second sublist of arri is created by merging unique arrays of third and fourth sublists of input. Last one is also simply the last sublist of input. The problem with my try is that it cannot export the last sublist and still I cannot get unique arrays. — Link_tester
– Link_tester, Commented May 14, 2021 at 12:26

Mateen Ulhaq · Accepted Answer · 2021-05-14 12:49:15Z

1

Sounds like you want to groupby and then flatten:

from itertools import groupby

def flatten(xss):
    return [x for xs in xss for x in xs]

key = lambda x: (x[0].size, x[1].size)

results = [flatten(g) for _, g in groupby(nums, key)]

To further keep only the unique arrays in each row, one can make use of dict:

unique_results = [
    list({tuple(x.reshape(-1)): x for x in xs}.values()) for xs in results
]

Which gives:

[
 [  array([[1.]]), array([[2.]]), array([[3.]]), array([[5.]])     ],
 [  array([[3.], [3.]]), array([[8.], [9.]]), array([[1.], [2.]])  ],
 [  array([[5.]]), array([[1.]])                                   ],
]

edited May 14, 2021 at 12:49

answered May 14, 2021 at 12:27

Mateen Ulhaq

27.9k21 gold badges121 silver badges155 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Link_tester Over a year ago

Dear @Mateen Ulhaq, this is exactly what I want. The only remained issue is that I want to have only unique arrays in each sublist. Can this be added to your solution? Thanks in advance for any feedback.

Mateen Ulhaq Over a year ago

One can filter results afterwards. The way to do this depends on how big the arrays are... if performance isn't an issue, you can use dict's unique key property: unique_results = [list({tuple(x.reshape(-1)): x for x in xs}.values()) for xs in results]

Link_tester Over a year ago

to do so, I face the error: unhashable type: 'list'

Link_tester Over a year ago

Sorry, where should I put this reshape(-1)? Thanks for your help.

Link_tester Over a year ago

Thanks for being that much supportive. I do appreciate it.

Collectives™ on Stack Overflow

How to merge numpy arrays stored in a nested list in python

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related