NumPy recfunctions join_by TypeError

Question

I encounter a TypeError when I attempt to join a 'uint16' field to a structured array in NumPy 1.11 or 1.12 (Python 3.5).

import numpy as np
from numpy.lib import recfunctions as rfn
foo = np.array([(1,)],
               dtype=[('key', int)])
bar = np.array([(1,np.array([1,2,3]))],
               dtype=[('key', int), ('value', 'uint16', 3)])
rfn.join_by('key', foo, bar)

This is the error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/anaconda3/lib/python3.5/site-packages/numpy/lib/recfunctions.py", line 986, in join_by
    output.sort(order=key)
  File "/home/user/anaconda3/lib/python3.5/site-packages/numpy/ma/core.py", line 5420, in sort
    sidx = self.filled(filler).argsort(axis=axis, kind=kind,
  File "/home/user/anaconda3/lib/python3.5/site-packages/numpy/ma/core.py", line 3668, in filled
    fill_value = _check_fill_value(fill_value, self.dtype)
  File "/home/user/anaconda3/lib/python3.5/site-packages/numpy/ma/core.py", line 470, in _check_fill_value
    fill_value = np.array(_recursive_set_fill_value(fill_value, ndtype),
  File "/home/user/anaconda3/lib/python3.5/site-packages/numpy/ma/core.py", line 436, in _recursive_set_fill_value
    output_value.append(np.array(fval, dtype=cdtype).item())
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

The same problem does not occur if I use a 'float16'.

import numpy as np
from numpy.lib import recfunctions as rfn
foo = np.array([(1,)],
               dtype=[('key', int)])
bar = np.array([(1,np.array([1,2,3]))],
               dtype=[('key', int), ('value', 'float16', 3)])
rfn.join_by('key', foo, bar)

Is this just a bug? Or is there some way to prevent this problem?

Looks like a bug to me - you should report it on the bug tracker — Eric
– Eric, Commented Jun 26, 2017 at 22:59
The bug is visible more simply as bar.view(np.ma.MaskedArray).sort() — Eric
– Eric, Commented Jun 26, 2017 at 23:04
@Eric - Ah... so it is probably closely related to this bug. — eatcrayons
– eatcrayons, Commented Jun 26, 2017 at 23:10

Eric · Accepted Answer · 2017-07-01 15:37:54Z

2

This is a bug. This PR partially fixes it, but it seems you've stumbled across a can of worms relating to np.ma and subdtypes.

As for why it worked for float16 - None was being coerced into nan (a questionable feature), rather than erroring.

edit: PR is merged, this will be fixed in numpy 1.14

edited Jul 1, 2017 at 15:37

answered Jun 27, 2017 at 0:15

Eric

98.1k54 gold badges257 silver badges389 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

eatcrayons Over a year ago

Thanks for looking into this! I'll see if I can find a workaround for now.

eatcrayons Over a year ago

Although not a specific answer to this question, this post provided me with enough information to develop a workaround using merge_arrays.

Collectives™ on Stack Overflow

NumPy recfunctions join_by TypeError

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related