2

I have a dataframe of columns that I would like to convert to numeric, like so:

for col in df.columns[22:33]:
   df[col] = pd.to_numeric(df[col], errors = 'coerce')

Which works great. However whenever I try to include more than a single range, like so:

for col in df.columns[22:66, 68, 69, 71, 72, 74:79, 81:86, 88, 89,91:94 ]:
    df[col] = pd.to_numeric(df[col], errors = 'coerce')

I get the error,

IndexError:  too many indices for array

Is there a way around this, I would like to not have to do it over and over for each range. Thanks.

Improve this question

3 Answers 3

Reset to default
3

this is the functionality of np.r_. You pass in array of numbers and slices, it will construct/concatenate them into the expanded array.

cols = np.r_[22:66, 68, 69, 71, 72, 74:79, 81:86, 88, 89,91:94]

array([22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38,
       39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,
       56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 68, 69, 71, 72, 74, 75, 76,
       77, 78, 81, 82, 83, 84, 85, 88, 89, 91, 92, 93])

After that, you may use cols as 

df.columns[cols]
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

this is perfect! Simple and exactly what I was looking for thanks!
@jvalenti: glad I could help
Notice that the behavior is slightly different, e.g. 22:66 results in columns 22-65 whereas it would be columns 22-66 using df.loc[:, 22:66].
1

You have to create 2 loops then. Create a list with elements consist of the ranges, then loop on it, inside this loop insert your original for loop code.

Like:

for i in range_list:
    for col in df.columns[i]:
        #do your code here.

Your range_list will have the ranges you need.

Improve this answer

3 Comments

in theory this works. the problem I had with it was that I couldn't find a way to specify my ranges that was syntaxtically consistent. I tried lists and tuples and it didn't work.
I see the problem with that but I still think its doable. Basically the ranges you must be tuples of intergers to store int values say range_list = [(22,66)...] and so on then during the second iteration call each value by df.columns[tuple first element : tuple second element] explicitly.
Or maybe you are referring to the single range input like 68,69,71? If that's the case just create a range for them. So the code will hold.
1

Assuming that your target columns represent integers names for the columns in the dataframe (per your example), you can munge the string and concatenate the result.

target_cols_str = "22:66, 68, 69, 71, 72, 74:79, 81:86, 88, 89, 91:94"
df_series = []
for col in target_cols_str.split(','):
    col = col.strip()  # Remove any whitespace.
    if ':' in col:
        lhs, rhs = col.split(':')
        df_series.append(df.loc[:, int(lhs):int(rhs)])
    else:
        df_series.append(df.loc[:, int(col)])
df2 = pd.concat(df_series, axis=1)

Applied to your example to convert the original dataframe to numeric values:

target_cols_str = "22:66, 68, 69, 71, 72, 74:79, 81:86, 88, 89, 91:94"
for col in target_cols_str.split(','):
    col = col.strip()  # Remove any whitespace.
    if ':' in col:
    lhs, rhs = col.split(':')
    for col_name in range(int(lhs), int(rhs) + 1):
        df.loc[:, col_name] = pd.to_numeric(df.loc[:, col_name], errors = 'coerce')
    else:
        df.loc[:, int(col)] = pd.to_numeric(df.loc[:, int(col)], errors = 'coerce')
Improve this answer

4 Comments

I don't quite understand. The integers are the column indexes. Will this allow me to convert the columns as well? Thanks.
If the numbers represent integer locations of the columns, just convert all loc values above to iloc. I just demonstrated conversion of the columns per the second example.
When I run this I get the error: TypeError: arg must be a list, tuple, 1-d array, or Series
@AndyL. Modified the solution to account for that.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.