Snowflake Table from Databricks using Python/Scala

Question

I want to create a table and load data into it in Snowflake from Databricks using Python/Scala.

Below is my code snippet. I'm getting the below error. How can I create the table first if not exists in Databricks notebook using Python or Scala and then load the data?

If so, what functions do I need to use? Below gives me an error.

df1.write.format("snowflake").options(sfOptions).option("dbtable","TEST_TABLE")
         .mode(SaveMode.Append)

Without an error message, this question is missing a minimal reproducible example. I will vote to close for now. — halfer
– halfer, Commented May 25, 2023 at 11:31

Sergiu · Accepted Answer · 2021-10-12 11:58:45Z

4

If you use Scala code then your df write should look like this:

df.write
    .format(SNOWFLAKE_SOURCE_NAME)
    .options(sfOptions)
    .option("dbtable", "t2")
    .mode(SaveMode.Append)
    .save()

If you use Python code then your df write should look like this:

df.write
    .format(SNOWFLAKE_SOURCE_NAME)
    .options(**sfOptions)
    .option("dbtable", "t2")
    .mode(SaveMode.Append)
    .save()

where:

SNOWFLAKE_SOURCE_NAME = "net.snowflake.spark.snowflake"

Observe there is a difference between the options on Scala vs Python.

answered Oct 12, 2021 at 11:58

Sergiu

4,7661 gold badge16 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Snowflake Table from Databricks using Python/Scala

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related