Define function in scala using DataFrame type

Question

I would have a simple question... I try to use a function that ate a dataframe as argument and returns another dataframe

I do:

val get_nb_previous_offre : ((DataFrame) => (DataFrame)) = (arg1: DataFrame) => {
   // create new_df using arg1 (=DataFrame I pass in parameter)
      new_df    
}

I get the error:

<console>:32: error: not found: type DataFrame
   val get_nb_previous_offre : ((DataFrame) => (DataFrame)) = (arg1: DataFrame) => {

Is there a librairy I have to import to use DataFrame as a type for my function?

Raphael Roth · Accepted Answer · 2018-11-16 16:08:22Z

5

Yes you need to import this type :

import org.apache.spark.sql.DataFrame

answered Nov 16, 2018 at 16:08

Raphael Roth

27.3k19 gold badges98 silver badges152 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

DataBach Over a year ago

Alternativly this comon Scala syntax also works: import org.apache.spark.sql.{DataFrame} . Also for anyone coming here and previously coding in PySpark, make sure to stick to camel case convention to avoid exception.

Collectives™ on Stack Overflow

Define function in scala using DataFrame type

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related