'OneHotEncoder' object has no attribute 'transform'

Question

I am using Spark v3.0.0. My dataframe is:

indexer.show()
+------+--------+-----+
|row_id|    city|index|
+------+--------+-----+
|     0|New York|  0.0|
|     1|  Moscow|  3.0|
|     2| Beijing|  1.0|
|     3|New York|  0.0|
|     4|   Paris|  2.0|
|     5|   Paris|  2.0|
|     6|New York|  0.0|
|     7| Beijing|  1.0|
+------+--------+-----+

Then I want to use One hot encoding of the dataframe's column "index" and getting this error.

encoder = OneHotEncoder(inputCol="index", outputCol="encoding")
encoder.setDropLast(False)
indexer = encoder.transform(indexer)

----------------------------------------
AttributeErrorTraceback (most recent call last)
<ipython-input-32-70bbd67e6679> in <module>
      1 encoder = OneHotEncoder(inputCol="index", outputCol="encoding")
      2 encoder.setDropLast(False)
----> 3 indexer = encoder.transform(indexer)

AttributeError: 'OneHotEncoder' object has no attribute 'transform'

desertnaut · Accepted Answer · 2022-01-20 10:50:04Z

3

You need to fit it first - before fitting, the attribute does not exist indeed:

encoder = OneHotEncoder(inputCol="index", outputCol="encoding")
encoder.setDropLast(False)
ohe = encoder.fit(indexer) # indexer is the existing dataframe, see the question
indexer = ohe.transform(indexer)

See the example in the docs for more details on the usage.

edited Jan 20, 2022 at 10:50

answered Sep 22, 2020 at 14:24

desertnaut

60.8k32 gold badges155 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Dimitrios Mistriotis Over a year ago

You use indexer befoer you define it, also link is broken; had same issue

desertnaut Over a year ago

@DimitriosMistriotis Thanks Δημήτρη. I fixed the link; indexer is the already existing dataframe (see the question), so here I am simply overwriting it (nowadays I don't like the practice of overwriting though).

Dimitrios Mistriotis Over a year ago

Thanks, we were following a tutorial :) and fell in a loop. Now we can try again :)

Collectives™ on Stack Overflow

'OneHotEncoder' object has no attribute 'transform'

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related