How can we have table grouping in Spark the same way as SQL Server?

Question

In SQL Server we have a database, inside database we have schema and under that schema we have table, like DatabaseName.SchemaName.TableName. And we also refer to Database as Catalog in SQL Server.

But in Spark, Catalog means an interface that we use to interact with Spark. I also noticed same result when I typed

spark.sql('show databases').show() and
spark.sql('show schemas').show()

Does database and schema mean same thing in Spark? If not what's the difference? In Spark, is it possible to have it the same way like SQL server has DatabaseName.Schema.TableName?

David Browne - Microsoft · Accepted Answer · 2022-09-27 00:10:06Z

1

From the docs

Please note that the usage of SCHEMAS and DATABASES are interchangeable and mean the same thing.

SHOW DATABASES

And CREATE TABLE supports only two-part names.

answered Sep 27, 2022 at 0:10

David Browne - Microsoft

90.7k7 gold badges52 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How can we have table grouping in Spark the same way as SQL Server?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related