Apply Feature Hashing to specific columns from a DataFrame

Question

I'm a bit lost with the use of Feature Hashing in Python Pandas .

I have the a DataFrame with multiple columns, with many information in different types. There is one column that represent a class for the data.

Example:

         col1   col2    colType
    1     1      2        'A'
    2     1      1        'B'
    3     2      4        'C'

My goal is to apply FeatureHashing for the ColType, in order to be able to apply a Machine Learning Algorithm.

I have created a separate DataFrame for the colType, having something like this:

                   colType  value
           1         'A'       1
           2         'B'       2
           3         'C'       3
           4         'D'       4

Then, applied Feature Hashing for this class Data Frame. But I don't understand how to add the result of Feature Hashing to my DataFrame with the info, in order to use it as an input in a Machine Learning Algorithm.

This is how I use FeatureHashing:

  from sklearn.feature_extraction import FeatureHasher
  fh = FeatureHasher(n_features=10, input_type='string')
  result = fh.fit_transform(categoriesDF)

How do I insert this FeatureHasher result, to my DataFrame? How bad is my approach? Is there any better way to achieve what I am doing?

Thanks!

k88 · Accepted Answer · 2018-07-09 20:42:25Z

3

I know this answer comes in late, but I stumbled upon the same problem and found this works:

fh = FeatureHasher(n_features=8, input_type='string')
sp = fh.fit_transform(df['colType'])
df = pd.DataFrame(sp.toarray(), columns=['fh1', 'fh2', 'fh3', 'fh4', 'fh5', 'fh6', 'fh7', 'fh8'])
pd.concat([df1, df], axis=1)

This creates a dataframe out of the sparse matrix retrieved by the FeatureHasher and concatenates the matrix to the existing dataframe.

answered Jul 9, 2018 at 20:42

k88

1,9442 gold badges17 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Nico Gallegos · Accepted Answer · 2017-11-23 04:52:54Z

0

I have switched to One Hot Coding, using something like this:

categoriesDF = pd.get_dummies(categoriesDF)

This function will create a column for every non-category value, with 1 or 0.

answered Nov 23, 2017 at 4:52

Nico Gallegos

42110 silver badges20 bronze badges

Collectives™ on Stack Overflow

Apply Feature Hashing to specific columns from a DataFrame

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related