Calling DataFrames inside sql query

Question

Is there a way to include dataframes inside a sql query? I'm actually trying to break a huge sql query and put in multiple dataframes and call those dataframes inside the original query.

select * from DF alike?

Ramdev Sharma · Accepted Answer · 2021-02-17 17:03:27Z

3

You can create a Temporary View using Dataframe which can be used in Spark Sql

For e.g. If you have Dataframe 'df' then you can run this to create temporary View with any name e.g. "my_table"

df.createOrReplaceTempView("my_table")

Now you can run query on top of this using following to load this as newer dataframe:

df2 = spark.sql("select * from my_table")

edited Feb 17, 2021 at 17:03

answered Jan 25, 2019 at 2:56

Ramdev Sharma

1,0141 gold badge14 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Calling DataFrames inside sql query

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related