Keep only numerical information from numpy array of strings

Question

I want to keep only numbers from an numpy array of strings, which are not necessarily valid. My code looks looks like the following:

age = train['age'].to_numpy() # 200k values
set(age)
# {'1', '2', '3', '7-11', np.nan...} 

age  = np.array(['1', '2', '3', '7-11', np.nan])

Desired output: np.array([1, 2, 3]). Ideally, '7-11' would be 7, however, that's not simple and is a tolerable loss.

np.isfinite(x) gives "ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''"

x = [num for num in age if isinstance(num, (int, float))] returns []

This is in essence a follow-up to the following question, stackoverflow.com/q/11620914/5728614 — Ali Pardhan
– Ali Pardhan, Commented Jan 21, 2022 at 2:11
Please provide the desired output, which I assume is an array with just [1, 2] in it? Should return any integers and floating point numbers, but what about a string that is a number, e.g. '1'. Should that be included as an integer type? A little more context about edge cases, etc would be helpful — frederick-douglas-pearce
– frederick-douglas-pearce, Commented Jan 21, 2022 at 2:26
Maybe this SO answer is helpful? If you are going to have to evaluate x like shown, with mixed types including strings, then the output x array will all be strings, and you can use isnumeric to filter the array for numbers, see docs — frederick-douglas-pearce
– frederick-douglas-pearce, Commented Jan 21, 2022 at 2:29
@frederick-douglas-pearce I have made a large edit and corrected the data types. I attempted one of the solutions to the answer you linked. I will give isnumeric a try — Ali Pardhan
– Ali Pardhan, Commented Jan 21, 2022 at 2:58

frederick-douglas-pearce · Accepted Answer · 2022-01-21 08:43:38Z

2

Here's an option that will split strings on '-' first, and only take the first value, so '7-11' is converted to 7:

age = np.array(['1', '2', '3', '7-11', np.nan])
age_int = np.array([int(x[0]) for x in np.char.split(age, sep='-') if x[0].isdecimal()])

Output: array([1, 2, 3, 7])

There is a more efficient way to do this if you don't care about cases like '7-11':

age_int2 = age[np.char.isdecimal(age)].astype(int)

Output2: array([1, 2, 3])

edited Jan 21, 2022 at 8:43

answered Jan 21, 2022 at 8:27

frederick-douglas-pearce

5064 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

BoomBoxBoy · Accepted Answer · 2022-01-21 03:07:51Z

1

You could do something like the following

for pos, val in enumerate(age):
    try:
        new_val = int(val)
    except:
        new_val = np.nan
    age[pos] = new_val

age = age[age!="nan"].astype(int)

print(age)
> array([1, 2, 3])

answered Jan 21, 2022 at 3:07

BoomBoxBoy

1,8951 gold badge9 silver badges25 bronze badges

2 Comments

Ali Pardhan Over a year ago

After a small change, which I edited in, this code solves my problem. Thank you! I am going to wait a few days before accepting this, in case there is a nice one-liner or 'better' solution.

BoomBoxBoy Over a year ago

@frederick-douglas-pearce has a great one-liner!

Collectives™ on Stack Overflow

Keep only numerical information from numpy array of strings

2 Answers 2

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related