How to count number of columns in a table in Spark SQL (Databricks)?

Question

I'm having trouble writing a statement to count the number of columns in Spark SQL. I tried using the information schema and using the table.columns, but both don't seem to work in Spark SQL. Does anyone have any suggestions?

Ajay Kharade · Accepted Answer · 2020-04-28 18:09:14Z

1

Below is couple of lines you can add to count number of columns in Spark SQL,

Pyspark Solution:

df_cont = spark.creatDataframe() // use right funtion to create dataframe based on source 
print("Number of columns:"+str(len(df_cont.columns)))

Scala Solution:

val df_cont = spark.creatDataframe() // use right funtion to create dataframe based on source 
val length = df_cont.columns.length
println("Number of columns:"+length)

answered Apr 28, 2020 at 18:09

Ajay Kharade

1,5351 gold badge17 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to count number of columns in a table in Spark SQL (Databricks)?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related